Eval concurrency contract — verifier writes only sidecars (#1194)

Prereq for #1189 (resident verifier + 2×-rig eval). Encodes the boundary that keeps the training corpus records uncorrupted when a second process writes alongside the eval process. Builds on #1192 (atomic sidecar writes) and #1159 (in-process arm-registry lock).

The problem

ARCHER's existing eval parallelism is threads inside one eval_harness process, coordinated by in-process threading.Lock:

Lock	File it guards	Site
_ARMS_MUTEX	~/.archer_eval.arms.json (arm registry)	eval_harness.py:375
_RUN_CTX_LOCK	~/.archer_run_<ts>.json (run-context)	eval_harness.py:2869
sched_lock / results_lock	in-memory scheduler / results list	eval_harness.py:745-746

The #1189 resident verifier is a separate OS process (coder model on card 1, concurrent with the judge on card 0). A threading.Lock lives in one process's memory — it does not cross the process boundary. So none of the locks above can serialize the eval process against the verifier process. If both processes wrote the same file, they would race, and a concurrent reader (ARCHER-Live poll, the next pipeline stage, an Auditor script) could observe a torn (partial) file.

The contract — three rules

1. Per-session sidecars are per-process-owned. The verifier writes only its own <session>.verifier.json, atomically, via audit_review._atomic_write_text (temp file + os.replace, #1192). Each session's .verifier.json has exactly one writer, and its filename is distinct from every eval-owned sidecar (.tier2.json, .residual.json, .txt.gz, the run CSV). Distinct filenames ⇒ zero shared-write surface ⇒ no cross-process lock needed. Helper: audit_review.write_verifier_sidecar().

2. Shared aggregates stay single-writer (eval process only). ~/.archer_eval.arms.json, the run CSV, and the run-context JSON are written by the eval process alone. The verifier process must never open them for write. With rules 1 + 2 honored, the happy path needs no cross-process lock at all — that is the point of the design, not a gap in it.

Aggregate	Writer	In-process guard	Verifier writes it?
~/.archer_eval.arms.json	eval process (acquire/release_arm_slot)	_ARMS_MUTEX (threads)	No
run CSV <ts>.csv	eval process (once, at run end, unique path)	none needed (single write)	No
run-context ~/.archer_run_<ts>.json	eval process (_append_run_context)	_RUN_CTX_LOCK (threads)	No

The in-process threading.Locks remain correct for what they do — serializing the eval process's own worker threads. They are not, and do not need to be, cross-process locks.

3. Break-glass for genuine multi-writer. If some aggregate is ever forced to be written by more than one process, it uses audit_review._flock_update_json(path, mutate_fn) — fcntl.flock(LOCK_EX) on a lockfile + read-modify-write + temp-rename publish — never threading.Lock. This mechanism is provided and tested now, but per the WATCHLIST-aware decision on #1194 it is not retrofitted onto arms.json: converting a file that is correctly single-writer today would be speculative cost with no proven multi-writer need. The primitive exists so the contract has a sanctioned answer the day a real need appears (that conversion would be its own issue, gated by #1189 review).

Why flock and not threading.Lock (the load-bearing distinction)

threading.Lock guards threads within one interpreter; fcntl.flock is an OS-level advisory lock visible to every process on the host. Under two real processes incrementing a shared counter:

• plain write, or threading.Lock → lost updates (both read N, both write N+1) → final < 2·M.
• _flock_update_json → the exclusive lock serializes each read-modify-write → final == 2·M.

The stress test tests/core/test_verifier_concurrency_1194.py demonstrates exactly this: the flock path lands on 2·M with zero lost updates, and the negative control (same two-process loop without flock) provably loses updates. That contrast is why the break-glass helper uses flock, recorded as an executable check rather than an assertion.

Where this is encoded in code

• audit_review.write_verifier_sidecar() — rule 1 (atomic per-session sidecar; touches nothing else).
• audit_review._flock_update_json() — rule 3 (cross-process break-glass).
• Module-level contract comment in audit_review.py (by the sidecar writers) and at each shared-aggregate write site in eval_harness.py (arms registry, run CSV, run-context) — so the #1189 implementer sees the single-writer invariant at the point of the write.

Scope

1194 delivers the mechanism + documented contract + cross-process stress test. The criterion

"the #1189 verifier writes only .verifier.json / touches no shared aggregate" is enforced and verified when #1189 lands (Auditor-gated); #1194 gives #1189 the primitive and the invariant to satisfy.