66 lines
2.8 KiB
Markdown
66 lines
2.8 KiB
Markdown
|
Exploratory service-UI verification (ADR-008 Level 4 / ADR-017)
|
||
|
|
|
||
|
|
Drive a browser against a **staging** deploy of a service, exercise its
|
||
|
|
`roles/<service>/VERIFY.md` acceptance journeys plus free exploration, and write a
|
||
|
|
tracked report. Argument: the service/role name (e.g. `/verify-service photoprism`).
|
||
|
|
|
||
|
|
## Prerequisites (this is forward-looking — ADR-017 dependencies)
|
||
|
|
|
||
|
|
This skill cannot run until all of these exist; if any is missing, say so and stop —
|
||
|
|
do not improvise around it:
|
||
|
|
|
||
|
|
- `ubongo` with the `playwright` Claude Code plugin (browser automation tools).
|
||
|
|
- A **staging** deploy of the target service (ADR-008 Level 2).
|
||
|
|
- Authentik (staging) for test-user provisioning + SSO.
|
||
|
|
- `roles/<name>/VERIFY.md` present.
|
||
|
|
|
||
|
|
## Process
|
||
|
|
|
||
|
|
### Phase 0 — safety gate (staging only)
|
||
|
|
|
||
|
|
Confirm the target resolves to the **staging** environment/inventory, never production.
|
||
|
|
If you cannot prove it is staging, **stop** — exploratory clicking is destructive
|
||
|
|
(ADR-002). State why you stopped.
|
||
|
|
|
||
|
|
### Phase 1 — read intent
|
||
|
|
|
||
|
|
Read `roles/<name>/VERIFY.md`: the Critical user journeys, What good looks like, Not
|
||
|
|
browser-verifiable, and Test data sections.
|
||
|
|
|
||
|
|
### Phase 2 — test user
|
||
|
|
|
||
|
|
Provision (reuse-or-create) a test user in the staging Authentik `test` group, with
|
||
|
|
ephemeral credentials held only for this run. Never use a real/production account.
|
||
|
|
|
||
|
|
### Phase 3 — drive the browser
|
||
|
|
|
||
|
|
Via the `playwright` plugin, on `ubongo`: open the service's staging URL (resolved via
|
||
|
|
boma DNS), authenticate through the real Traefik + Authentik SSO flow, then execute each
|
||
|
|
`VERIFY.md` journey — judging pass/fail and screenshotting key states — and free-explore
|
||
|
|
for anything obviously broken. Save screenshots to the git-ignored `.verify-runs/`
|
||
|
|
working dir; avoid capturing credential screens.
|
||
|
|
|
||
|
|
### Phase 4 — write the report
|
||
|
|
|
||
|
|
Save to `docs/testing/reviews/YYYY-MM-DD-<name>.md` and overwrite
|
||
|
|
`docs/testing/reviews/latest.md`. Structure:
|
||
|
|
|
||
|
|
- **One-line verdict** — e.g. "5/5 journeys passed; one manual check pending".
|
||
|
|
- **Run metadata** — date, service, staging env, test user, reviewed commit SHA.
|
||
|
|
- **Per-journey result** — pass/fail against `VERIFY.md`, with the evidence (linked
|
||
|
|
screenshot path) and any observation.
|
||
|
|
- **Free-exploration findings** — anything noticed beyond the listed journeys.
|
||
|
|
- **Manual-test checklist** — the "Not browser-verifiable" items plus anything Claude
|
||
|
|
couldn't do: numbered steps, expected result, and why it was handed off.
|
||
|
|
|
||
|
|
### Phase 5 — clean up + commit
|
||
|
|
|
||
|
|
Offer to clean up the `test`-group user (or note that the staging rebuild will).
|
||
|
|
Commit the report markdown per CLAUDE.md git conventions. **Do not** commit
|
||
|
|
`.verify-runs/` (git-ignored).
|
||
|
|
|
||
|
|
## Notes
|
||
|
|
|
||
|
|
- Reports (markdown) are committed; screenshots stay local on `ubongo` in `.verify-runs/`.
|
||
|
|
- Exploratory and interactive — this is not a deterministic CI gate.
|