Naming collision: `playbook` is overloaded (cache vs. catalog)¶

For: ARCHER-Coder (who did the domain → playbook rename) From: Sagittarius/rig eval work — surfaced 2026-06-25 Status: ✅ RESOLVED 2026-06-25 — premise was incorrect; no rename, no collision. Decision: keep playbook = cache, keep domain = catalog. See correction below.

⟳ Correction & decision — 2026-06-25 (ARCHER-Coder, code-verified)¶

This document's premise is wrong, and the decision is "do nothing to the names." Verified against the local working tree, origin/main, and every branch (git log --all).

There was no domain → playbook rename. Not started, not committed, not on any branch. The catalog concept is — and always has been — domain: DOMAIN_REGISTRY (ARCHER.py:223), the public --do/--domain CLI flag, archer_platform/domain.py (DomainConfig, DomainPlugin Protocol), load_domain, current_domain. ~160 sites, all domain. The "you renamed Concept 2 to playbook" framing attributes a rename that never happened.

What actually happened is a different rename: Skills → Plays (#974) — it renamed the unit (skill → play: plays/ dir, PLAYS, get_play_hints, play= log field). That is almost certainly what was mistaken for a catalog rename. And #974 is the one that's genuinely incomplete in code: SKILL_CATEGORIES is still the live registry (~307 sites, including the router) and the session log still writes skill= (reverted deliberately in ecb5923 because the log writer was never updated).

Therefore there is no live collision. Three distinct names exist in the code today — play (unit) · domain (catalog) · playbook (cache) — with zero overlap. The homonym this doc warns about only materializes if someone executes a domain → playbook rename. Nobody has, and the decision below is not to.

Ground-truth grep (2026-06-25):

concept	live name in code	representative sites	state
unit — one objective (`plays/PT-*.py`)	`play` (was `skill`)	`PLAYS`, `get_play_hints`, `plays/`	#974 rename partial — `SKILL_CATEGORIES` (307) + `skill=` log field unconverted
catalog — grouping of plays	`domain`	`DOMAIN_REGISTRY`, `--do`, `DomainConfig`, `DomainPlugin`, `load_domain` (~160)	untouched; never renamed to `playbook`
cache — learned best-solution store	`playbook`	`PLAYBOOK_DB` (99), `query_playbook` (57), `archer_playbook.py`, `~/.archer_playbook.db`, `--no-playbook` (21)	entrenched; backs a production DB

Decision (User, 2026-06-25): - Do NOT rename the catalog domain. It is the public --do flag baked into every eval/CI invocation and all docs; renaming buys only AD-domain disambiguation (a ~30-site, mostly-string overload) at high churn. Not worth it. - playbook stays = cache. Both schemes below are moot — they solve a collision that doesn't exist. Scheme A would migrate 119 sites + a live DB for nothing. - The only real naming debt is finishing #974 (SKILL_CATEGORIES → PLAY_CATEGORIES; skill= → play= log field). Track separately if/when naming consistency is prioritized; not urgent, and unrelated to playbook.

Everything below this line is the original 2026-06-25 Sagittarius note, retained for the record. Its central claim (a partial domain → playbook rename in code) is superseded by the grep above — the rename was never begun, not merely unfinished.

TL;DR¶

playbook now refers to two different concepts. One must keep the name; the other must be renamed. The good news: in the code the collision hasn't fully landed yet, so the cheaper option is still on the table.

Concept 1 — the cache: the learned "best play from previous plays" store (~/.archer_playbook.db, query_playbook, _save_playbook_on_success).
Concept 2 — the catalog: what was domain — the grouping of skills/plays a module performs (penetration, etc.). You renamed this to playbook.

The conflict: Concept 1 has always been playbook in code; renaming Concept 2 to playbook makes them homonyms.

How this surfaced¶

During rig eval work (8b vs 32b on inference-heavy objectives), we hit the question: if we run a more capable model, do the cached "best answers" (Concept 1) and the hints cap/drag it? That discussion kept colliding with "playbook" the catalog (Concept 2). The ambiguity is already costing clarity in design conversations — exactly the kind that breeds real bugs.

Ground truth — current code state (grep, 2026-06-25)¶

The catalog rename is conceptual/partial — not yet in the code that matters:

concept	code symbols today	sites
unit — one skill/objective (`plays/PT-*.py`)	`play` / `PLAYS`, `get_play_hints`	~40
catalog — grouping of plays (was/is "domain")	`domain.py`, `load_domain`, `current_domain`, "Domain Loaded"	~48 — still `domain`
cache — learned best-solution store	`PLAYBOOK_DB`, `query_playbook(_intelligent)`, `_save_playbook_on_success`, `_record_playbook_failure`, `init_playbook`, `archer_playbook.py`, `~/.archer_playbook.db`, `--no-playbook`	~119 — `playbook`

So in code: catalog is still domain, cache is playbook, no collision yet. The collision is the planned/conceptual domain→playbook rename meeting the entrenched playbook=cache. The expensive half (119 cache sites) is untouched — which is why the cheaper resolution is still available.

The clean model¶

Three concepts, three names, zero overlap:

play (the unit) → grouped into a catalog → best solved attempts cached in a memory.

Only question: which word lands on which.

Two resolution schemes¶

Scheme A — finish your rename (catalog = playbook); rename the cache. - Cache playbook → a distinct term across ~119 sites + archer_playbook.py module + ~/.archer_playbook.db file + --no-playbook/--no-seed-playbook flags + the playbook/playbook_failures tables. - Bigger lift; touches the live learning system and a production DB file. - Cache-name options: recall (recommended — describes the function; recall_lookup, save_to_recall), cookbook, or casebook.

Scheme B — give the catalog a different name; cache keeps playbook. - Catalog domain → skillset or module (~48 sites). - Cheaper, and also fixes a second latent overload: domain means an AD/DNS domain everywhere else in a pentest tool (GOAD has literal domains — the eval range exercises them).

Recommendation¶

Scheme B — cheaper (~48 vs ~119 sites), avoids rewriting the runtime memory system + migrating a production DB, and fixes the domain/AD-domain overload in the same pass. Final glossary: play (unit) · skillset (catalog, was domain) · playbook (cache, unchanged).

If playbook=catalog is a committed product/external decision, then Scheme A — and pick the cache name now (recommend recall) so the term stops drifting.

Migration notes (either scheme)¶

One coordinated, scoped refactor, not a piecemeal find/replace. Tests as the guard: tests/agent/* (incl. test_no_progress_watchdog.py, the depth-guard suite), tests/core/*, tests/playbook/*, tests/run_all.py.
Disambiguate "domain" during the sweep: skill-catalog domain is in scope; AD/DNS domain (range targets, Kerberos, etc.) is not — don't rename those.
Scheme A only: decide whether to keep the ~/.archer_playbook.db filename (rename only symbols) or migrate the file. eval_harness.py references --no-playbook in ARCHER_FLAGS and --no-seed-playbook + the seed pass — update together.
Eval-side instrumentation is being added for an ablation experiment (ARCHER_PLAYBOOK_DB env override + a --with-playbook eval flag, both default-off) that touch the cache. They'll be named for whatever the cache ends up called — today playbook; under Scheme A they get swept too.

Decision requested¶

Scheme A or B?
If A: cache name — recall / cookbook / casebook?

Once decided, this becomes a scoped rename ticket (own branch, tests green, no eval work mixed in).

Naming collision: playbook is overloaded (cache vs. catalog)¶