Hands-on labs · Turn mechanisms into verifiable artifacts
Three reproducible standard-library Python labs turn loop recovery, tool gates, and context pressure from reading conclusions into engineering acceptance tests.
Chapter brief
Question to answer
Which lab should I run first to turn my riskiest agent mechanism into an inspectable artifact?
By the end, you can
- Choose the loop, tool, or context lab by failure cost
- Record fault injection, expected invariants, and evidence in one format
- Map lab findings back to the related chapters and source paths
- Read this now if
- Engineers who have read one or two chapters and are ready to implement or review an agent runtime
- Prerequisites
- Run Python 3.11 or later; Recognize basic tool calls and JSON events
- Deliverable
- A lab route, an acceptance-report template, and three directly runnable scripts
- Evidence boundary
- The labs verify mechanisms and invariants; they are not model, provider, or production performance benchmarks
Start with the mechanism most likely to cause an incident
Section titled “Start with the mechanism most likely to cause an incident”Do not choose the most entertaining lab. Ask first: when this system fails once, which boundary creates the largest loss?
| Current risk | Run first | What you must prove |
|---|---|---|
| A tool committed, then the process died before the result became durable | Lab 01 · Loop recovery | One operation_id never repeats a side effect, and the verifier resumes at the right checkpoint |
| A user approved one action, but the execution arguments changed | Lab 02 · Tool gate | Schema, policy, and the approval token bind one exact action |
| Context fills, compaction drops a constraint or trusts a hostile file | Lab 03 · Context pressure | Protected segments survive, untrusted sources are scanned, and summaries retain provenance |
All three labs intentionally avoid live models and third-party packages. Stabilize the deterministic harness boundary before adding model variance.
How the three labs form learning routes
Section titled “How the three labs form learning routes”Runtime route: read Agent Loop and Session Lifecycle, then complete Lab 01. The artifact is an event timeline that can answer whether a tool actually committed.
Safety route: read Tool System, Permissions, and Sandbox, then complete Lab 02. The artifact is an execution gate that blocks argument drift and token replay.
Context route: read Context System and Memory, then complete Lab 03. The artifact is a record of retention, compaction, eviction, and blocking across context pressure levels.
One experiment record for every lab
Section titled “One experiment record for every lab”Each lab ends with the same report shape so “it ran” is never the conclusion:
{ "lab": "01-loop-recovery", "failure_injected": "after_effect_before_result", "expected_invariant": "duplicate_effects == 0", "observed": { "exit_reason": "injected_crash", "duplicate_effects": 0, "resumed_checkpoint": "effect_committed" }, "evidence": ["events.jsonl", "effect-ledger.json"], "remaining_risk": "external service must expose idempotency or commit lookup"}At minimum, record the failure point, invariant, observed value, evidence files, and unresolved boundary. A success without an evidence file does not pass.
Run and preserve the artifacts
Section titled “Run and preserve the artifacts”From the project root, run every self-test:
pnpm labs:checkOr download and run each asset:
Scripts place temporary results under .agent-mechanics-lab/. Do not commit those working directories. Commit a curated report, the relevant events, and a note explaining how your production system differs from the lab.
Move from a lab to a real product
Section titled “Move from a lab to a real product”The labs replace networks, models, and databases with deterministic simulations to isolate variables. Migrate in this order:
- Keep the event schema, operation IDs, and assertions.
- Replace the simulated effect with one real tool in a test environment.
- Replace fixed input with model-generated tool calls.
- Rerun the same failure matrix and attribute each new failure to model, protocol, or external system.
If the failure can no longer be reproduced after adding the model, restore the deterministic version. Do not tune prompts, policy, and persistence at the same time.
Completion means another engineer can reproduce the mechanism evidence
Section titled “Completion means another engineer can reproduce the mechanism evidence”A lab passes only when all of these are true:
- the self-test exits with code 0;
- at least one planned failure is triggered, not only the happy path;
- one file or event clearly proves the invariant;
- the report names one migration difference in your product;
- the report states what the lab did not cover.
The next step is not another lab. Put one of these assertions into your own CI or fault-injection suite.
Scripts and source paths
Section titled “Scripts and source paths”Appendix: pre-lab checks
Section titled “Appendix: pre-lab checks”Open six checks
- What is the worst side effect represented by this lab?
- Which state is the source of truth, and which is only an index or UI surface?
- Between which two durable actions should the fault land?
- Can code judge the passing condition without model self-evaluation?
- Which output lets another engineer verify the result independently?
- Which uncontrolled variables appear after a live model is connected?