Skip to content

Rig flaky-link inference findings (#1022)

Date: 2026-06-24 · Harness: scripts/flaky_link_test.py · Model: qwen3:8b · rig via autossh tunnel. Raw: raw/flaky-link-baseline.json, raw/flaky-link-fault.json. (Regenerated 2026-06-28 after the original CSL-OPS copy was lost to an uncommitted-repo reset.)

Metric is client-side end-to-end wall-clock + completion rate (NOT Ollama's server-reported tok/s, which hides the link).

Steady-state baseline (n=20, 1024-tok gens)

leg completion p50 p95 max drops
tunnel (rig via WiFi) 100% (20/20) 8.03 8.12 8.20 0
rig-local (no network) 100% (20/20) 8.02 8.03 8.03 0

Transport cost +0.09s p95 (+1%), zero drops → DEFER HARDWIRE. On a healthy link the tunnel is effectively free.

Fault behavior (5s WiFi cut mid-call)

  • In-flight call aborts (RemoteDisconnected) — rig compute wasted, result lost (Ollama cancels generation the instant the client disconnects).
  • During reconnect the autossh forward refuses (connection refused, instant fail) until re-established.
  • Recovery: 14.3s total (9.3s after WiFi returned) for a 5s blip; full radio re-assoc ~31s. autossh self-heals with no manual restart.

Calibration for #1021 (resilient retry)

  • Catch both RemoteDisconnected (mid-call) and ConnectionRefused (forward down).
  • Retry budget must outlast ~15s recovery (backoff ~0/2/4/8/8/8 ≈ 30s); instant retries burn out in ~1s.
  • Mid-call abort is not resumable → re-issue the whole call.

Verdict

Hardwire deferred (tunnel adds ~1%, 0 drops when stable). #1021 retry covers transient blips. Residual mid-stream aborts → #1025 rog-fallback / hardwire.