Compare commits

..

No commits in common. "810e6d557bef42a7dff1d8faf3f507fa632eddde" and "2dfa8ca9d682dd8d06b6d8cf0d1c9c7c48e9cc21" have entirely different histories.

18 changed files with 79 additions and 155 deletions

View file

@ -6,7 +6,6 @@ exclude_paths:
- .venv/
- .collections/
- .scaffold/
- "**/vault.yml" # ansible-vault encrypted — not lintable YAML
# Warn only (don't fail) on these rules during initial setup
# Remove entries as the codebase matures

View file

@ -12,7 +12,7 @@ platforms:
# Project-owned image built from .docker/molecule-debian13/Dockerfile
# and hosted in the Forgejo container registry.
# Build/push with: make molecule-image / make molecule-image-push
image: forgejo.nyumbani.baobab.band/<owner>/<repo>/molecule-debian13:latest
image: git.baobab.band/<owner>/<repo>/molecule-debian13:latest
pre_build_image: true
privileged: true # required for systemd
cgroupns_mode: host

View file

@ -24,4 +24,3 @@ ignore: |
.venv/
.collections/
.scaffold/
**/vault.yml

View file

@ -61,10 +61,7 @@ Full design rationale: `docs/decisions/`
- Encrypted files are always named `vault.yml`, sitting alongside `vars.yml`
- Never put plaintext secrets in any file not named `vault.yml`
- Structure secrets as a nested map `vault.<service>.<key>` (e.g.
`vault.grafana.admin_password`); reference as `{{ vault.grafana.admin_password }}`
- Vault password comes from Vaultwarden via `rbw` (`scripts/vault-pass-client.sh`,
wired as `vault_password_file`). Unlock once per session: `rbw unlock`
- Vault password file: `.vault_pass` (gitignored — obtain via secure channel)
- To edit a vault file: `make decrypt FILE=<path>`, edit, `make encrypt FILE=<path>`
---

View file

@ -9,13 +9,12 @@ PLAYBOOK := $(VENV)/bin/ansible-playbook
GALAXY := $(VENV)/bin/ansible-galaxy
LINT := $(VENV)/bin/ansible-lint
MOLECULE := $(VENV)/bin/molecule
# Vault password is resolved via ansible.cfg (vault_password_file); no flag needed.
VAULT_ARGS :=
VAULT_ARGS := --vault-password-file .vault_pass
INVENTORY := -i inventories/production/hosts.yml
TF := terraform
TF_ENV ?= staging
MOLECULE_IMAGE := forgejo.nyumbani.baobab.band/<owner>/<repo>/molecule-debian13:latest
MOLECULE_IMAGE := git.baobab.band/<owner>/<repo>/molecule-debian13:latest
MOLECULE_DOCKERFILE := .docker/molecule-debian13/Dockerfile
.DEFAULT_GOAL := help
@ -102,13 +101,13 @@ encrypt:
ifndef FILE
$(error FILE is required: make encrypt FILE=<path>)
endif
$(ANSIBLE)-vault encrypt $(FILE)
$(ANSIBLE)-vault encrypt --vault-password-file .vault_pass $(FILE)
decrypt:
ifndef FILE
$(error FILE is required: make decrypt FILE=<path>)
endif
$(ANSIBLE)-vault decrypt $(FILE)
$(ANSIBLE)-vault decrypt --vault-password-file .vault_pass $(FILE)
# ── Molecule test image ───────────────────────────────────────────────────────

View file

@ -16,9 +16,8 @@ _Last reviewed: 2026-05-30._
| `.docker/molecule-debian13/Dockerfile` | Present — custom Molecule test image (ADR-008) |
| `docs/decisions/*`, `docs/runbooks/*` | Current and mutually reconciled |
| `Makefile`, lint config (`.ansible-lint`, `.yamllint`), `.gitignore` | Present and used |
| `git` | Initialized, trunk-based on `main`, pushed to `origin` (`forgejo.nyumbani.baobab.band:7577`). |
| `git` (local) | Initialized — trunk-based on `main`. Off-machine remote (Forgejo) being set up separately. |
| Pre-commit hooks | Configured: lint, gitleaks, vault-encryption guard. Activate with `pre-commit install` after `make setup`. |
| Vault password client | `scripts/vault-pass-client.sh` fetches the master password from Vaultwarden via `rbw` (wired as `vault_password_file`). Requires `rbw` installed + `rbw unlock`. |
| Terraform HCL (`terraform/`) | Written (proxmox VM module + envs) — but never run; see below |
## Scaffolded but empty — NOT implemented
@ -42,7 +41,7 @@ calls are empty.
| CI (Forgejo Actions) | ADR-003 / ADR-008 | Pipeline described; not implemented |
| Level 2 / 3 testing (staging, `askari` smoke) | ADR-008 | Depends on real VMs / `askari`, which don't exist yet |
| Per-service roles | ADR-004 | Model defined; no service roles built |
| Forgejo Actions CI | ADR-003 / ADR-008 | Remote is live (pushed); Actions/`act_runner` pipeline not yet built |
| Forgejo remote + CI | ADR-003 / ADR-008 | Local git is live; pushing to `git.baobab.band` and Actions CI are being set up |
## Keeping this honest

View file

@ -2,7 +2,7 @@
inventory = inventories/production/hosts.yml
roles_path = roles
collections_path = .collections
vault_password_file = scripts/vault-pass-client.sh
vault_password_file = .vault_pass
interpreter_python = auto_silent
stdout_callback = yaml
callbacks_enabled = timer, profile_tasks

View file

@ -54,13 +54,10 @@ some public-facing services — not a compliance exercise.
## Secrets management
- Ansible Vault for all secrets (API keys, passwords, certificates), structured as a
nested `vault.<service>.<key>` map (ADR-003)
- The master vault password lives in **Vaultwarden** and is fetched on demand by
`scripts/vault-pass-client.sh` (wired as `vault_password_file`) through the `rbw`
agent — never written to a plaintext file on disk. Unlock once per session with
`rbw unlock`; nothing decryptable sits at rest in the repo or working tree
- See `docs/runbooks/rotate-secrets.md` for `rbw` setup and rotation
- Ansible Vault for all secrets (API keys, passwords, certificates)
- Vault password stored outside the repo (`.vault_pass` gitignored)
- New collaborators receive vault password via a separate secure channel
- See `docs/runbooks/rotate-secrets.md` for rotation procedure
## What this baseline does not include

View file

@ -33,10 +33,8 @@ reproducible, and has no extra dependencies.
- HashiCorp Vault: powerful, but significant operational overhead for this scale
**Rationale**: Vault is built-in, requires no extra services, and works well at this
scale. Whole-file encryption makes diffs unreadable regardless of layout, so rather
than flattening we organise secrets for human lookup and clean extraction: a nested
`vault.<service>.<key>` map inside each `vault.yml`, scoped to actual secrets (see
CLAUDE.md → Secrets).
scale. The main limitation (whole-file encryption makes diffs unreadable) is mitigated
by keeping `vault.yml` files small and purposeful — only actual secrets, no structure.
---
@ -73,7 +71,7 @@ Config files: `.ansible-lint`, `.yamllint` in repo root.
## CI/CD
**Choice**: Forgejo Actions (self-hosted at forgejo.nyumbani.baobab.band) + `act_runner`
**Choice**: Forgejo Actions (self-hosted at git.baobab.band) + `act_runner`
**Not chosen**: GitHub Actions (external), Jenkins (heavy)

View file

@ -45,7 +45,7 @@ has been run).
## State backend
**Choice**: Forgejo HTTP backend (self-hosted at forgejo.nyumbani.baobab.band)
**Choice**: Forgejo HTTP backend (self-hosted at git.baobab.band)
Keeps all state on the same self-hosted stack without additional services.
Authentication uses a Forgejo personal access token via `TF_HTTP_USERNAME` and

View file

@ -150,7 +150,7 @@ IoT devices cannot initiate connections to `srv`.
| Infrastructure VMs | `<role><n>` | `dns1`, `dns2`, `proxy` |
| Hetzner VPS | `askari` | Swahili for guard/sentinel |
| Internal FQDN | `<host>.boma.baobab.band` | `dns1.boma.baobab.band` |
| Public service FQDN | `<service>.baobab.band` | `forgejo.nyumbani.baobab.band` |
| Public service FQDN | `<service>.baobab.band` | `git.baobab.band` |
---
@ -166,7 +166,7 @@ Terraform itself writes no DNS records — see ADR-009.
Public-facing services resolve to the public IP or Cloudflare proxy.
**Split-horizon**: `dns1`/`dns2` serve internal answers for any hostname that has
both a public and private face. Example: `forgejo.nyumbani.baobab.band` resolves to
both a public and private face. Example: `git.baobab.band` resolves to
`10.20.0.12` (proxy) internally and to the public IP externally.
OPNsense DNS resolver forwards `boma.baobab.band` queries to `dns1`/`dns2`.

View file

@ -62,7 +62,7 @@ configuration issues invisible to Ansible check mode.
**Source**: `.docker/molecule-debian13/Dockerfile`
**Base**: `debian:trixie-slim` (official Debian 13, Docker Hub — only external
dependency permitted here, as the base OS image is not substitutable)
**Registry**: `forgejo.nyumbani.baobab.band/<owner>/<repo-name>/molecule-debian13:latest`
**Registry**: `git.baobab.band/<owner>/<repo-name>/molecule-debian13:latest`
Build and push with:
```bash

View file

@ -108,7 +108,7 @@ rendered entirely by the Ansible `dns` role:
remains the ultimate source of truth for which hosts exist; the data simply flows
through the inventory instead of through a direct Terraform→DNS write.
- **Service, alias (CNAME), split-horizon, and non-VM records** (e.g. the OPNsense
gateway, `forgejo.nyumbani.baobab.band` → proxy) are explicit zone data in `group_vars`.
gateway, `git.baobab.band` → proxy) are explicit zone data in `group_vars`.
This dissolves the bootstrap cycle that a Terraform-managed zone would create. If
Terraform wrote records via RFC 2136, provisioning the **first** DNS server would

View file

@ -1,89 +1,71 @@
# Runbook — Vault secrets (setup & rotation)
The master vault password lives in **Vaultwarden** (item `boma-ansible-vault`) and is
fetched on demand by `scripts/vault-pass-client.sh` through the `rbw` agent. It is
never stored in a plaintext file on disk. See ADR-002.
---
## One-time — `rbw` setup on a new machine
```bash
# Install rbw (Debian 13: try apt first, else cargo)
sudo apt install rbw || cargo install rbw
# Point it at the homelab Vaultwarden and log in
rbw config set base_url https://vaultwarden.baobab.band
rbw config set email <your-vaultwarden-email>
rbw login # prompts for your Vaultwarden master password (+ 2FA if enabled)
# Unlock the agent (do this once per terminal session before vault operations)
rbw unlock
# Sanity check — should print the master vault password:
rbw get boma-ansible-vault
```
Once unlocked, `make encrypt/decrypt/check/deploy` and the pre-commit ansible-lint
hook all obtain the password automatically. If the agent is locked you'll see a
clear "run: rbw unlock" error rather than a hang.
---
# Runbook — Rotating vault secrets
## Rotating a single secret value
1. Ensure the agent is unlocked: `rbw unlock`
2. Decrypt the relevant vault file:
1. Decrypt the relevant vault file:
```bash
make decrypt FILE=inventories/production/group_vars/all/vault.yml
```
3. Edit the value (keep the nested `vault.<service>.<key>` structure).
4. Re-encrypt:
2. Edit the file and update the secret value.
3. Re-encrypt:
```bash
make encrypt FILE=inventories/production/group_vars/all/vault.yml
```
5. Commit and deploy:
4. Commit the updated vault file:
```bash
git add inventories/production/group_vars/all/vault.yml
git commit -m "Rotate <service> <key>"
git commit -m "Rotate <secret name>"
```
5. Deploy to apply the new secret to hosts:
```bash
make check PLAYBOOK=site # verify what will change
make deploy PLAYBOOK=site
```
---
## Rotating the master vault password
## Rotating the vault password
This re-keys every encrypted file from the old password to a new one. **Order
matters** — re-key the files *before* updating Vaultwarden, or the fetch script will
hand back the new password while the files are still encrypted with the old one.
This affects all encrypted files in the repo. Do this only when:
- A person with vault access leaves the project
- The password is suspected to be compromised
1. Unlock the agent: `rbw unlock` (so the script can read the current/old password).
2. Generate a new password and write it to a throwaway file *outside* the repo:
Steps:
1. Ensure you have the current vault password in `.vault_pass`.
2. Re-key all vault files:
```bash
umask 077; rbw generate --length 40 > /tmp/new_vault_pass
find . -name "vault.yml" | xargs ansible-vault rekey \
--vault-password-file .vault_pass \
--new-vault-password-file /path/to/new_password_file
```
3. Re-key all vault files (old via the fetch script, new from the temp file):
```bash
find . -path ./.venv -prune -o -name vault.yml -print \
| xargs ansible-vault rekey \
--vault-password-file scripts/vault-pass-client.sh \
--new-vault-password-file /tmp/new_vault_pass
```
4. Update the `boma-ansible-vault` item in Vaultwarden to the new password
(paste the contents of `/tmp/new_vault_pass`), then re-sync: `rbw sync`.
5. Shred the temp file: `shred -u /tmp/new_vault_pass` (or `rm -f`).
6. Verify: `make decrypt FILE=<a vault.yml>` should still work, now via the new
password. Commit the re-keyed files.
Rotate the master password when someone with access leaves, or if it is suspected
compromised.
3. Replace `.vault_pass` with the new password file.
4. Distribute the new password to all collaborators via a secure channel.
5. Commit all rekeyed vault files:
```bash
git add -A
git commit -m "Rekey all vault files"
```
---
## Access for additional people / machines
## Adding a new collaborator
Access is granted in Vaultwarden — share the `boma-ansible-vault` item (or place it
in a shared collection). Each person/machine then runs the one-time `rbw` setup
above. To revoke access, remove the share in Vaultwarden and rotate the master
password (above), since anyone who had it could have copied it.
1. Share the vault password via a secure channel (password manager, etc.)
2. The collaborator creates `.vault_pass` locally (gitignored)
3. They can now decrypt/encrypt vault files normally
## Removing a collaborator's access
Rotate the vault password as described above. There is no per-user access
control in Ansible Vault — access is binary (has the password or not).
If per-user access control becomes necessary, evaluate SOPS + age at that point.

View file

@ -1,18 +1,7 @@
$ANSIBLE_VAULT;1.1;AES256
62313835613730303334653334393033646661323865636534353061333765326239333835643139
6631393939363263313861656461303134383162336662380a333564343131323036343137383736
64666265653233636631373266396132623561363766326266353638643538303936613435333530
3864356133633663650a376362313861326263633036303664336439663030613438636339613765
63383431376636646236346435363035333036373466613066643761646237323133633866366230
38396333313238393630336263373063363538343865366432353138643663653638356438303738
66313832343436616634313734343433363362613437383963383263363666613431346663376263
62633162633962376537306262353736336435343339333266643661373538643236636631666662
39643664303137356562313061306439623239656534323065306132643833383738623261393232
63643434396165343631633063616161616430373130663830623936306339393933653437393931
37633532363264636537343165316231363130613964646635666665363136623637326561323336
30386235366261353661656231396362366263316338663135663333306434306563363464363336
35376233303939393039646261633833656337666335636333343030343435656664306433363530
36306636303530336262396664646331663834336235663236656636353833396437303636373133
64313639346164656438613066636661353736613334383734633232376335323761396634363031
65333631636337323630353165356539306531393434633163373637373739366131363734353934
34373762303661316235353162636132623736646630663438366433376639613964
---
# This file must be encrypted with Ansible Vault before committing.
# Run: make encrypt FILE=inventories/production/group_vars/all/vault.yml
#
# Example secrets (plaintext shown here for structure reference only):
#
# vault__ansible_become_password: "changeme"

View file

@ -1,35 +0,0 @@
#!/usr/bin/env bash
#
# ansible-vault password client.
#
# Prints the boma master vault password to stdout by fetching it from Vaultwarden
# via the `rbw` agent. Wired in as `vault_password_file` (ansible.cfg) and used by
# the Makefile vault targets, so every ansible-vault / ansible-playbook / lint run
# obtains the password the same way.
#
# The password lives only in Vaultwarden (encrypted at rest) and in the rbw agent's
# memory while unlocked — never in a plaintext file on disk.
#
# Unlock once per terminal session before running any vault operation:
# rbw unlock
#
# Override the Vaultwarden item name via BOMA_VAULT_ITEM if it ever changes.
#
set -euo pipefail
item="${BOMA_VAULT_ITEM:-boma-ansible-vault}"
if ! command -v rbw >/dev/null 2>&1; then
echo "vault-pass-client: 'rbw' is not installed — see docs/runbooks/rotate-secrets.md." >&2
exit 1
fi
# Only the password reaches stdout; all diagnostics go to stderr so they can never
# be mistaken for the password by ansible-vault.
if ! pw="$(rbw get "$item" 2>/dev/null)"; then
echo "vault-pass-client: could not read '$item' from Vaultwarden via rbw." >&2
echo " The agent is probably locked. Run: rbw unlock" >&2
exit 1
fi
printf '%s\n' "$pw"

View file

@ -8,9 +8,9 @@ terraform {
#
# If Forgejo's HTTP state endpoint is unavailable, remove this block entirely
# to fall back to local state on the control node.
address = "https://forgejo.nyumbani.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/production.tfstate"
lock_address = "https://forgejo.nyumbani.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/production.tfstate/lock"
unlock_address = "https://forgejo.nyumbani.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/production.tfstate/lock"
address = "https://git.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/production.tfstate"
lock_address = "https://git.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/production.tfstate/lock"
unlock_address = "https://git.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/production.tfstate/lock"
lock_method = "POST"
unlock_method = "DELETE"
}

View file

@ -8,9 +8,9 @@ terraform {
#
# If Forgejo's HTTP state endpoint is unavailable, remove this block entirely
# to fall back to local state on the control node.
address = "https://forgejo.nyumbani.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/staging.tfstate"
lock_address = "https://forgejo.nyumbani.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/staging.tfstate/lock"
unlock_address = "https://forgejo.nyumbani.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/staging.tfstate/lock"
address = "https://git.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/staging.tfstate"
lock_address = "https://git.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/staging.tfstate/lock"
unlock_address = "https://git.baobab.band/api/v1/repos/<owner>/<repo>/raw/terraform/state/staging.tfstate/lock"
lock_method = "POST"
unlock_method = "DELETE"
}