sjat/boma
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

ADR-008: expand Level 4 into the verify-service harness (ADR-017)

Browse source
This commit is contained in:
sjat 2026-06-05 13:14:12 +02:00
parent cc3337502f
commit 2df1f98153
1 changed files with 18 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
docs/decisions/008-testing.md
View file

@ -53,14 +53,25 @@ Once `askari` is operational: scripted checks from outside the network confirmin
that public-facing services respond correctly. Catches firewall and reverse proxy that public-facing services respond correctly. Catches firewall and reverse proxy
configuration issues invisible to Ansible check mode. configuration issues invisible to Ansible check mode.
### Level 4 — Service-UI acceptance (planned, not built) ### Level 4 — Service-UI acceptance (Claude-driven exploratory)
Claude drives a headless browser from `ubongo` against a *deployed* service: loads A Claude-driven exploratory check of a service's **application UI**, run as
the rendered UI, creates test users, exercises features, and hands the operator a `/verify-service <name>` on `ubongo` (ADR-017). Claude drives Chromium via the
manual test script for the rest. Catches application-level regressions that no lower `playwright` plugin against a **staging** deploy, authenticates through the real
level sees. The harness (Playwright/headless-Chromium, screenshot-back-to-Claude) is Traefik + Authentik SSO flow using a test user in the staging `test` group, then
a **separate spec**; `ubongo` is sized for it (ADR-015). Status: designed, not built executes the service's `roles/<service>/VERIFY.md` acceptance journeys *and*
(STATUS.md). free-explores — judging pass/fail, screenshotting key states. It writes a dated report
to `docs/testing/reviews/` and hands the operator a manual-test checklist for anything
it can't verify (hardware, paid/external flows, subjective judgment).
Catches application-level regressions no lower level sees ("does PhotoPrism actually
serve photos?"). Placement: after Level 2 (staging deploy), before production
promotion. Exploratory and interactive by design — *not* a deterministic CI/cron gate
(that role belongs to health checks / Uptime Kuma).
**Status:** the skill, the `VERIFY.md` template, and standards are authorable now;
running it is deferred on `ubongo` + the `playwright` plugin + Authentik + a staging
deploy (STATUS.md). Full design: ADR-017.
--- ---