Free sample lesson · Graph Engineering: The Karpathy Loop, Improved 1000x by Itself — The Anthropic Playbook
Externalize the Bottleneck
Where should memory and evaluation live when model calls keep failing?
Lesson 1 of the Wondering course Graph Engineering: The Karpathy Loop, Improved 1000x by Itself — The Anthropic Playbook
Externalize the Bottleneck Overview
Learn how to choose the smallest verifiable and provenance-aware architecture by matching structures directly to your actual system constraints.
The Architecture Ladder
Adding more layers to an agent system does not automatically improve results. Extra complexity often creates activity without better outcomes.
Instead, you should match your task to the smallest structure that solves your specific bottleneck.
Architecture is a tool to externalize constraints like memory or evaluation. Start simple and only grow when you have measured evidence of failure.
Matching Structure to Bottlenecks
Different structures solve different problems. A loop externalizes evaluation for tasks like checked extraction.
A chain externalizes task order for fixed pipelines. Use a swarm when you need parallel search or role specialization.
- Architecture is a bottleneck tool
Avoid using a swarm just because agents are available. Only add what the task requires.
Managing History and Facts
Complex tasks require advanced memory structures. A commit DAG externalizes experiment lineage, allowing you to revisit alternative hypotheses.
A knowledge graph externalizes shared facts and cross-session memory. This is useful when relationships must survive between different user sessions.
Do not use a graph for a single document answer with no persistent state. Match the storage to the data lifespan.
Measured Escalation Strategy
Build your architecture through measured escalation. Only move up the ladder when a simpler level fails to meet a specific metric.
For example, add planning only when task paths vary significantly. Add a graph only when facts must persist across sessions.
Every added layer carries overhead in latency, cost, and governance. The goal is a verifiable, reversible, and provenance-aware system.
- Start Simple: Begin with the smallest structure that meets current requirements.
- Measure Failure: Identify a specific metric failure before changing architecture.
- Escalate Layer: Move up the ladder only to address the measured bottleneck.
Putting It All Together
- Assess Task: Identify the primary constraint before adding layers.
- Add Loop: Incorporate evaluation checks when extraction quality needs verification.
- Scale Memory: Introduce advanced storage structures only when cross-session facts are required.
Check yourself
An architecture layer should be added to an agent system only after observing a _____.
- measured failure
- new available API
Answer: measured failure
Under measured escalation, you should only add architectural complexity after observing a measured failure in a simpler structure.
Different structures manage specific constraints: a _____ externalizes evaluation for checked extraction, while a _____ externalizes shared facts across user sessions.
Answer: Different structures manage specific constraints: a loop externalizes evaluation for checked extraction, while a knowledge graph externalizes shared facts across user sessions.
Loops provide feedback and evaluation for tasks requiring validation, whereas knowledge graphs store facts and relationships across separate sessions.
A team has a working single-agent pipeline, but needs to track experiment lineage and revisit alternative hypotheses during complex reasoning tasks. Which minimal structure should they adopt?
- Commit DAG
- Agent Swarm
- Knowledge Graph
- Evaluator Loop
Answer: Commit DAG
A commit DAG externalizes experiment lineage and allows revisiting alternative hypotheses. Adding a swarm or graph would introduce unnecessary architectural overhead for this specific bottleneck.
Keep going
This is lesson 1 of 12 in Graph Engineering: The Karpathy Loop, Improved 1000x by Itself — The Anthropic Playbook. Next up:
- Program the Program
- Build a Safe Ratchet
- Preserve Experiment Lineage
- Coordinate Without Transcripts
- Generate and Reduce Work
Wondering turns any topic into short lessons you can read, watch, or listen to, with practice questions and an AI tutor.