Lab: Version control

Your feature gets real version management: a registry, a production pointer, an A/B-gated release, and a rehearsed rollback. This turns your prior hand-numbered versions into operable infrastructure.

Step 1 — The registry + pointer

1. 1Refactor your feature so its config (prompt, model, settings) is a versioned object loaded by id — prompts/<feature>/v1.py, v2.py, … each self-contained with a CHANGELOG.
2. 2Add a config.json with {"production": "v1", "last_known_good": "v1"}. Your feature reads the production id at call time and stamps every trace with it.
3. 3Confirm: changing the pointer changes which version serves, with no other code change.

Step 2 — Ship v2 through an offline A/B

1. 1Create v2 with one improvement (targeting your weakest eval check).
2. 2Run the harness on v1 and v2 over the same golden set. Do the paired comparison — list the flips, read them, check the holdout.
3. 3If v2 wins cleanly, update the pointer to v2 and set last_known_good to v2. Log the eval delta in v2's CHANGELOG. You've done a gated release.

Step 3 — The rollback drill

1. 1Create v3 that's deliberately worse (e.g., remove a key rule or example). Ship it (pointer → v3) without running the eval — simulating a careless deploy.
2. 2Run some traffic; watch a quality check drop. Now roll back: pointer → last_known_good (v2). Confirm production serves v2 again in one change.
3. 3Now do it right: reproduce v3's failure as a golden case, build v3-fixed against the full set, gate it, ship. Note the whole sequence in LAB-NOTES — the drill is the deliverable.