Add ADR-014 (sourcing technical knowledge)

Policy for how agents source authoritative knowledge and translate it to boma:
- Facts (version-specific syntax/options) → consult version-matched official docs,
  cite, and stamp `verified: subject · tool ver · source · date`. Training memory
  is the draft, never the authority.
- Judgments (best practices) → evidence to translate through boma's principles
  (extends ADR-013); never adopted because the docs/a blog say so.
- Consultation is risk-based (security / unfamiliar-or-fast-moving / can't quote a
  flag with confidence), with a "from memory, unverified" transparency backstop.
- Sources: context7 → WebFetch upstream → claude-code-guide → deep-research; match
  the PINNED version, not "latest".
- Graceful degradation: the plugin accelerators may be absent on a fresh checkout,
  so the policy commits to the principle and falls back to WebFetch/WebSearch.

CLAUDE.md gains a concise operating rule + pointer. TODO 10.4 decided. New TODO
10.7 tracks making the plugin/MCP toolchain reproducible from the repo (it is not
synced by account nor carried by git) — surfaced by a portability check this run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
sjat 2026-06-04 20:07:18 +02:00
parent 2f4218814a
commit 68a37d51f1
3 changed files with 129 additions and 1 deletions

View file

@ -164,6 +164,24 @@ Single-contributor, trunk-based (no merge requests / approval gates):
---
## Sourcing technical knowledge (ADR-014)
- **Facts vs judgments.** Version-specific facts (syntax, options, defaults) have one
authoritative answer — consult **version-matched** official docs and cite. Best
practices are *evidence to translate through boma's principles* (ADR-013), never
authority ("because the docs / a blog say so" is banned).
- **When to verify (risk-based):** required when security-relevant, the tool is
unfamiliar / fast-moving or newer than your training, or you'd assert a specific
flag/option/default you can't quote with confidence. Otherwise memory is fine — but
mark any from-memory version-specific claim **"from memory, unverified."**
- **Sources:** `context7` for library docs · upstream docs via `WebFetch` · Claude
Code/SDK/API → `claude-code-guide` agent · broad questions → `deep-research`. These
are plugins and may be absent on a fresh checkout — fall back to `WebFetch`/`WebSearch`
(core tools). Match the **pinned** version, not "latest."
- **Stamp verified facts** next to them: `verified: <subject> · <tool> <version> · <source> · <YYYY-MM-DD>`.
---
## Further reading
| Topic | File |
@ -174,6 +192,7 @@ Single-contributor, trunk-based (no merge requests / approval gates):
| Per-service security checklist | `docs/security/service-checklist.md` |
| Per-service security record (template) | `docs/security/service-security-template.md` |
| Heritage / V4 policy | `docs/decisions/013-heritage-v4.md` |
| Sourcing tech knowledge | `docs/decisions/014-knowledge-sourcing.md` |
| Toolchain choices | `docs/decisions/003-toolchain.md` |
| Docker & Compose model | `docs/decisions/004-docker-model.md` |
| Bootstrapping hosts | `docs/decisions/005-bootstrapping.md` |

View file

@ -76,9 +76,15 @@
1. ~~Policy for how we collaborate with references to baobabAnsibleV4 without misusing it.~~ DECIDED — ADR-013.
2. Policy for how we write key documents like ADRs.
3. Further development on how we we collaborate on designing the foundation for the project - seperate from how we implement new containers etc.
4. How do we make sure agents always use the latest official documentation for the technologies etc. we use?
4. ~~How do we make sure agents always use the latest official documentation for the technologies etc. we use?~~ DECIDED — ADR-014 (facts → version-matched docs, cited + stamped; best practices → translated per ADR-013; risk-based triggers; graceful fallback to WebFetch).
5. Always subagent driven?
6. When AI deploys, i.e. runs playbooks etc., should we make a methodology so that it does not have to poll all the time or review all the output. Perhaps something about the MAKE method could provide only the relevant feedback?
7. **Reproducible agent toolchain** (surfaced by ADR-014) — plugins/MCP
(`context7`, `superpowers`, `deep-research`, `claude-code-guide`) live under
`~/.claude/` per machine; they are NOT synced by account nor carried by git, so
a fresh clone lacks them. Make them reproducible from the repo:
`extraKnownMarketplaces` + `enabledPlugins` in `.claude/settings.json` + a
bootstrap note/script. Tie into control-node/AI setup (items 5 and 7).
11. **Kaizen loop** — set up ~2026-06-06 (one week from now).
1. Build `/retro`: reads `docs/FRICTION.md` + recurring `/review-repo`

View file

@ -0,0 +1,103 @@
# ADR-014 — Sourcing technical knowledge (docs and best practices)
## Context
Most work in boma is done by AI agents drawing on training memory, which is stale
and hallucination-prone for version-specific facts. The repo **pins tool versions**
(`requirements.txt`, `requirements.yml`, image tags), so the right reference is the
docs for *our pinned version*, not "the latest." The capability to fetch
authoritative knowledge already exists (`context7`, `WebFetch`/`WebSearch`, the
`deep-research` skill, the `claude-code-guide` agent) but is ungoverned. This ADR
sets the policy for sourcing knowledge and translating it to boma's aims. (Resolves
TODO 10.4.)
## Two kinds of knowledge, two rules
- **Facts** — version-specific syntax, options, defaults, behaviour. There is one
authoritative answer: the version-matched official docs. Training memory is the
*draft*, never the authority.
- **Judgments** — "best practice," how to structure or approach something. External
best practices are **evidence to translate through boma's principles**, never
adopted because the docs or a blog say so. This extends ADR-013's
translate-don't-transplant leash to all external sources: the acceptance test is
*"can this be justified from boma's principles?"* — the source is the prompt, not
the authority.
## When consulting is required (risk-based)
Routine, well-known syntax may rely on memory. Consulting a version-matched
authoritative source is **required** when any of these hold:
- it is **security-relevant**; or
- the tool is **unfamiliar or fast-moving**, or the version is **newer than the
agent's training**; or
- the agent is about to assert a **specific flag/option/default it cannot quote with
confidence**.
**Transparency backstop:** because "I'm uncertain" is an unreliable self-assessment
(agents are often confidently wrong), any version-specific claim given *from memory*
is marked **"from memory, unverified"** so it is visible and checkable. By the time
something is durable and version-specific, it has usually crossed a trigger and
should be verified, not left unverified.
## Source hierarchy
1. **Version-matched official docs**`context7` for libraries/frameworks; upstream
docs via `WebFetch` for tools. Match the pinned version.
2. **Claude Code / SDK / API questions** → the `claude-code-guide` agent.
3. **Broad or contested questions** → the `deep-research` skill (multi-source,
fact-checked).
4. Reputable references below those. **Training memory is never the authority** for a
version-specific fact.
Cite the source whenever one is consulted.
**Graceful degradation:** `context7` and the agents/skills above are *plugins* and may
be absent on a fresh checkout (see "Reproducibility" below). The always-available
fallback is `WebFetch`/`WebSearch` (core tools). This policy commits to the
**principle** — consult version-matched authoritative docs and cite — never to a
specific plugin, so it holds on a bare install.
## Capture (hybrid)
- **Facts** that can go stale get a durable, greppable stamp next to the fact (a
comment in the role/template, or a line in an ADR):
```
verified: <subject> · <tool> <version> · <source> · <YYYY-MM-DD>
```
This makes currency trackable and gives re-verification a trigger.
- **Judgments** follow ADR-013: translated to boma's terms, durable docs stay clean,
lineage stays transient (commit/conversation).
## Re-verification
A stamp binds a fact to a **pinned version**. When that version bumps (a
`requirements` change or an image upgrade), the stamps for that tool mark exactly
what to re-check. A future `/review-repo` or `/security-review` check could flag
stamps whose recorded version no longer matches what is pinned (stale-stamp
detection) — a noted enhancement, not built yet.
## Reproducibility of the toolchain
The accelerators this policy prefers (`context7`, `deep-research`, `superpowers`,
`claude-code-guide`) are **plugins under `~/.claude/`** — local per machine, **not**
synced by Claude account and **not** carried by the git repo (only `.claude/commands`,
`.claude/hooks`, `.claude/settings.json` travel). A fresh clone therefore lacks the
plugin toolchain until it is reinstalled. Making it reproducible from the repo
(`extraKnownMarketplaces` + `enabledPlugins` in `.claude/settings.json`, plus a
bootstrap step) is tracked in `docs/TODO.md` and tied to control-node/AI setup. Until
then, the graceful-degradation fallback above keeps the policy working.
## Decision
- Distinguish **facts** (consult version-matched authoritative docs; cite; stamp) from
**judgments** (translate through boma's principles per ADR-013; never authority).
- Require consultation **risk-based** (security / unfamiliar-or-fast-moving / can't
quote with confidence), with a from-memory transparency backstop.
- Commit to the principle, not a tool — degrade to `WebFetch`/`WebSearch` when plugins
are absent.
See also: ADR-013 (heritage / translate-don't-transplant), ADR-011 (version pinning),
ADR-008 (testing/verification).