From 29f1cb3b4c3d4f95ce82692ad5347a12d10d7d8e Mon Sep 17 00:00:00 2001 From: Alex Newman Date: Wed, 15 Oct 2025 20:09:57 -0400 Subject: [PATCH] feat: Add comprehensive best practices guide for context engineering in AI agents --- context/context-engineering.md | 222 +++++++++++++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 context/context-engineering.md diff --git a/context/context-engineering.md b/context/context-engineering.md new file mode 100644 index 00000000..adb1dce3 --- /dev/null +++ b/context/context-engineering.md @@ -0,0 +1,222 @@ +# Context Engineering for AI Agents: Best Practices Cheat Sheet + +## Core Principle +**Find the smallest possible set of high-signal tokens that maximize the likelihood of your desired outcome.** + +--- + +## Context Engineering vs Prompt Engineering + +**Prompt Engineering**: Writing and organizing LLM instructions for optimal outcomes (one-time task) + +**Context Engineering**: Curating and maintaining the optimal set of tokens during inference across multiple turns (iterative process) + +Context engineering manages: +- System instructions +- Tools +- Model Context Protocol (MCP) +- External data +- Message history +- Runtime data retrieval + +--- + +## The Problem: Context Rot + +**Key Insight**: LLMs have an "attention budget" that gets depleted as context grows + +- Every token attends to every other token (n² relationships) +- As context length increases, model accuracy decreases +- Models have less training experience with longer sequences +- Context must be treated as a finite resource with diminishing marginal returns + +--- + +## System Prompts: Find the "Right Altitude" + +### The Goldilocks Zone + +**Too Prescriptive** ❌ +- Hardcoded if-else logic +- Brittle and fragile +- High maintenance complexity + +**Too Vague** ❌ +- High-level guidance without concrete signals +- Falsely assumes shared context +- Lacks actionable direction + +**Just Right** ✅ +- Specific enough to guide behavior effectively +- Flexible enough to provide strong heuristics +- Minimal set of information that fully outlines expected behavior + +### Best Practices +- Use simple, direct language +- Organize into distinct sections (``, ``, `## Tool guidance`, etc.) +- Use XML tags or Markdown headers for structure +- Start with minimal prompt, add based on failure modes +- Note: Minimal ≠ short (provide sufficient information upfront) + +--- + +## Tools: Minimal and Clear + +### Design Principles +- **Self-contained**: Each tool has a single, clear purpose +- **Robust to error**: Handle edge cases gracefully +- **Extremely clear**: Intended use is unambiguous +- **Token-efficient**: Returns relevant information without bloat +- **Descriptive parameters**: Unambiguous input names (e.g., `user_id` not `user`) + +### Critical Rule +**If a human engineer can't definitively say which tool to use in a given situation, an AI agent can't be expected to do better.** + +### Common Failure Modes to Avoid +- Bloated tool sets covering too much functionality +- Tools with overlapping purposes +- Ambiguous decision points about which tool to use + +--- + +## Examples: Diverse, Not Exhaustive + +**Do** ✅ +- Curate a set of diverse, canonical examples +- Show expected behavior effectively +- Think "pictures worth a thousand words" + +**Don't** ❌ +- Stuff in a laundry list of edge cases +- Try to articulate every possible rule +- Overwhelm with exhaustive scenarios + +--- + +## Context Retrieval Strategies + +### Just-In-Time Context (Recommended for Agents) +**Approach**: Maintain lightweight identifiers (file paths, queries, links) and dynamically load data at runtime + +**Benefits**: +- Avoids context pollution +- Enables progressive disclosure +- Mirrors human cognition (we don't memorize everything) +- Leverages metadata (file names, folder structure, timestamps) +- Agents discover context incrementally + +**Trade-offs**: +- Slower than pre-computed retrieval +- Requires proper tool guidance to avoid dead-ends + +### Pre-Inference Retrieval (Traditional RAG) +**Approach**: Use embedding-based retrieval to surface context before inference + +**When to Use**: Static content that won't change during interaction + +### Hybrid Strategy (Best of Both) +**Approach**: Retrieve some data upfront, enable autonomous exploration as needed + +**Example**: Claude Code loads CLAUDE.md files upfront, uses glob/grep for just-in-time retrieval + +**Rule of Thumb**: "Do the simplest thing that works" + +--- + +## Long-Horizon Tasks: Three Techniques + +### 1. Compaction +**What**: Summarize conversation nearing context limit, reinitiate with summary + +**Implementation**: +- Pass message history to model for compression +- Preserve critical details (architectural decisions, bugs, implementation) +- Discard redundant outputs +- Continue with compressed context + recently accessed files + +**Tuning Process**: +1. **First**: Maximize recall (capture all relevant information) +2. **Then**: Improve precision (eliminate superfluous content) + +**Low-Hanging Fruit**: Clear old tool calls and results + +**Best For**: Tasks requiring extensive back-and-forth + +### 2. Structured Note-Taking (Agentic Memory) +**What**: Agent writes notes persisted outside context window, retrieved later + +**Examples**: +- To-do lists +- NOTES.md files +- Game state tracking (Pokémon example: tracking 1,234 steps of training) +- Project progress logs + +**Benefits**: +- Persistent memory with minimal overhead +- Maintains critical context across tool calls +- Enables multi-hour coherent strategies + +**Best For**: Iterative development with clear milestones + +### 3. Sub-Agent Architectures +**What**: Specialized sub-agents handle focused tasks with clean context windows + +**How It Works**: +- Main agent coordinates high-level plan +- Sub-agents perform deep technical work +- Sub-agents explore extensively (tens of thousands of tokens) +- Return condensed summaries (1,000-2,000 tokens) + +**Benefits**: +- Clear separation of concerns +- Parallel exploration +- Detailed context remains isolated + +**Best For**: Complex research and analysis tasks + +--- + +## Quick Decision Framework + +| Scenario | Recommended Approach | +|----------|---------------------| +| Static content | Pre-inference retrieval or hybrid | +| Dynamic exploration needed | Just-in-time context | +| Extended back-and-forth | Compaction | +| Iterative development | Structured note-taking | +| Complex research | Sub-agent architectures | +| Rapid model improvement | "Do the simplest thing that works" | + +--- + +## Key Takeaways + +1. **Context is finite**: Treat it as a precious resource with an attention budget +2. **Think holistically**: Consider the entire state available to the LLM +3. **Stay minimal**: More context isn't always better +4. **Be iterative**: Context curation happens each time you pass to the model +5. **Design for autonomy**: As models improve, let them act intelligently +6. **Start simple**: Test with minimal setup, add based on failure modes + +--- + +## Anti-Patterns to Avoid + +- ❌ Cramming everything into prompts +- ❌ Creating brittle if-else logic +- ❌ Building bloated tool sets +- ❌ Stuffing exhaustive edge cases as examples +- ❌ Assuming larger context windows solve everything +- ❌ Ignoring context pollution over long interactions + +--- + +## Remember + +> "Even as models continue to improve, the challenge of maintaining coherence across extended interactions will remain central to building more effective agents." + +Context engineering will evolve, but the core principle stays the same: **optimize signal-to-noise ratio in your token budget**. + +--- + +*Based on Anthropic's "Effective context engineering for AI agents" (September 2025)* \ No newline at end of file