Add /verify-service skill for Level 4 UI verification (ADR-017)

This commit is contained in:
sjat 2026-06-05 13:16:25 +02:00
parent 05abb3b6a5
commit 25f04002df

View file

@ -0,0 +1,65 @@
Exploratory service-UI verification (ADR-008 Level 4 / ADR-017)
Drive a browser against a **staging** deploy of a service, exercise its
`roles/<service>/VERIFY.md` acceptance journeys plus free exploration, and write a
tracked report. Argument: the service/role name (e.g. `/verify-service photoprism`).
## Prerequisites (this is forward-looking — ADR-017 dependencies)
This skill cannot run until all of these exist; if any is missing, say so and stop —
do not improvise around it:
- `ubongo` with the `playwright` Claude Code plugin (browser automation tools).
- A **staging** deploy of the target service (ADR-008 Level 2).
- Authentik (staging) for test-user provisioning + SSO.
- `roles/<name>/VERIFY.md` present.
## Process
### Phase 0 — safety gate (staging only)
Confirm the target resolves to the **staging** environment/inventory, never production.
If you cannot prove it is staging, **stop** — exploratory clicking is destructive
(ADR-002). State why you stopped.
### Phase 1 — read intent
Read `roles/<name>/VERIFY.md`: the Critical user journeys, What good looks like, Not
browser-verifiable, and Test data sections.
### Phase 2 — test user
Provision (reuse-or-create) a test user in the staging Authentik `test` group, with
ephemeral credentials held only for this run. Never use a real/production account.
### Phase 3 — drive the browser
Via the `playwright` plugin, on `ubongo`: open the service's staging URL (resolved via
boma DNS), authenticate through the real Traefik + Authentik SSO flow, then execute each
`VERIFY.md` journey — judging pass/fail and screenshotting key states — and free-explore
for anything obviously broken. Save screenshots to the git-ignored `.verify-runs/`
working dir; avoid capturing credential screens.
### Phase 4 — write the report
Save to `docs/testing/reviews/YYYY-MM-DD-<name>.md` and overwrite
`docs/testing/reviews/latest.md`. Structure:
- **One-line verdict** — e.g. "5/5 journeys passed; one manual check pending".
- **Run metadata** — date, service, staging env, test user, reviewed commit SHA.
- **Per-journey result** — pass/fail against `VERIFY.md`, with the evidence (linked
screenshot path) and any observation.
- **Free-exploration findings** — anything noticed beyond the listed journeys.
- **Manual-test checklist** — the "Not browser-verifiable" items plus anything Claude
couldn't do: numbered steps, expected result, and why it was handed off.
### Phase 5 — clean up + commit
Offer to clean up the `test`-group user (or note that the staging rebuild will).
Commit the report markdown per CLAUDE.md git conventions. **Do not** commit
`.verify-runs/` (git-ignored).
## Notes
- Reports (markdown) are committed; screenshots stay local on `ubongo` in `.verify-runs/`.
- Exploratory and interactive — this is not a deterministic CI gate.