Agents SDK Architecture

This page is the deep reference for how the built-in agent works internally. It covers the dependency injection system, the provider-source composition pattern, the agent execution loop, and the cross-cutting concerns that tie everything together. For the conceptual overview, see How the Agent Harness Works. For the design rationale, see Design Principles.

The Dependency Injection System

The SDK uses a custom DI container built around three components: AgentDependencyFactory, AgentDependencyRegistry, and ChatAgentFactory.

AgentDependencyFactory

Every injectable component has a factory with a create() classmethod. The return type annotation tells the registry what type this factory produces:

from zav.agents_sdk import AgentDependencyFactory, AgentDependencyRegistry

class MyServiceFactory(AgentDependencyFactory):
    @classmethod
    def create(cls, some_config: SomeConfig) -> MyService:
        return MyService(some_config.value)

AgentDependencyRegistry.register(MyServiceFactory)

The create() method's parameters are resolved recursively by the same DI system — they can be configuration models (looked up by name in agent_configuration), other registered dependencies (looked up by type), or framework-provided objects like LLMClientConfiguration or ConversationContext.

AgentDependencyRegistry

The registry maps types to their factories:

class AgentDependencyRegistry:
    registry: Dict[type, AgentDependencyFactory] = {}

    @classmethod
    def register(cls, factory):
        # Uses the return annotation of factory.create() as the key
        created_cls = inspect.signature(factory.create).return_annotation
        cls.registry[created_cls] = factory

    @classmethod
    def get(cls, t: type) -> Optional[AgentDependencyFactory]:
        return cls.registry.get(t)

    @classmethod
    def get_subclasses_of(cls, base: type) -> List[AgentDependencyFactory]:
        return [f for return_type, f in cls.registry.items()
                if issubclass(return_type, base)]

The get_subclasses_of() method is what powers auto-collection — see DependencyGroup below.

DependencyGroup — Auto-Collection

A DependencyGroup subclass declares what base type it collects:

class ToolsSourceGroup(DependencyGroup[ToolsSource]):
    __collects__ = ToolsSource

When ChatAgentFactory encounters a constructor parameter typed as ToolsSourceGroup, it:

1. Reads __collects__ → ToolsSource
2. Calls registry.get_subclasses_of(ToolsSource) → all factories that produce ToolsSource subclasses
3. Resolves each factory via normal DI (recursively)
4. Wraps the results: ToolsSourceGroup(items=[index_tools, doc_tools, web_tools, ...])

This is how registration is all you need. Implement a source, register a factory, and it's automatically collected into the right group — no central list to maintain.

ChatAgentFactory — The Resolution Engine

ChatAgentFactory.create() is the entry point. When a request arrives:

1. Fetches AgentSetup via AgentSetupRetriever.get(agent_identifier)
2. Looks up the agent class via ChatAgentClassRegistry.get(agent_setup.agent_name)
3. Inspects the agent class constructor signature
4. For each parameter, resolves the value using these rules (in order):
   • AgentCreator → creates a closure for spawning sub-agents
   • Registered factory → resolves via DI (recursively parsing create() params)
   • DependencyGroup → auto-collects all matching subclass factories
   • ConversationContext → injects the current request context
   • AgentPosition → injects the agent being resolved (agent_identifier): the root agent's, or a sub-agent's when it is resolved for one
   • ChatAgent subclass → creates a sub-agent recursively
   • LLMClientConfiguration → injects from AgentSetup
   • Span → injects the tracing span
   • Config model → looks up param_name in agent_configuration, parses as Pydantic model

A resolution_cache ensures dependencies marked with __singleton__ = True are created once per request, not once per injection site.

The Provider-Source Pattern

The DI system above is a general-purpose container. What gives the SDK its additive composition property is the provider-source pattern built on top of it:

Source → DependencyGroup → Provider → Agent

A source is a single implementation of a capability (e.g., IndexToolsSource provides search tools). A group auto-collects all registered sources of a given type via DependencyGroup. A provider applies configuration — filtering, ordering, formatting — and exposes a uniform interface to the Agent.

Each source implementation registers a factory with AgentDependencyRegistry. When ChatAgentFactory builds the agent, DependencyGroup auto-collects all factories whose return type matches the source base class (e.g., ToolsSource), resolves each one, and hands the collection to the provider. The provider then applies source filtering (include_sources / exclude_sources from configuration) and exposes a uniform interface to the agent.

This pattern repeats for every capability: tools, skills, context, memory, instructions, delegation, MCP, and message processing — each with its own source base class, group, and provider.

Why This Pattern Matters

When a new capability is added — say, image search tools — the developer implements ToolsSource, registers a factory, and it automatically appears in ToolsSourceGroup. The ToolsProvider, the Agent class, and ChatAgentFactory don't change. The deployer enables it by adding the source name to include_sources.

This is the core architectural invariant: the Agent class does not change when capabilities are added or removed. Configuration controls composition.

For the complete table of providers, sources, groups, and what lifecycle phases each participates in, see the Configuring Capabilities overview.

The Agent Execution Loop

When agent.execute_streaming(conversation) is called, the agent goes through a fixed sequence of phases. Each phase invokes specific providers.

1. Dispatch Check

dispatch = await dispatch_provider.try_dispatch(conversation, conversation_context)

All DispatchRule implementations are checked. If any rule matches the ConversationContext, it short-circuits the entire flow — the rule handles the request directly, and the LLM is never called. This is used for special routing like scheduled task handling.

2. Process Conversation

completions = await context_provider.process_conversation(initial_context, conversation)

Context sources run first because they may derive per-turn state that other providers read. ContextProvider can interleave developer messages into the conversation history: document context, tag context and custom context go in as inline developer messages when injection_mode=conversation. The conversation context is seeded only on a conversation's first turn; a resumed stateful turn already holds it in its restored transcript.

3. Build System Prompt

The system prompt is built once per turn. Each provider appends its section in order:

base_system_prompt (from agent_configuration)
  + system_prompt_provider.to_prompt()      → the system prompt sources, in order
  + skills_provider.to_prompt()             → skill catalog with names and descriptions
  + agent_delegation_provider.to_prompt()   → available agents and delegation instructions
  + tools_provider.to_prompt()              → tool-specific guidance (per-source)
  + instructions_provider.to_prompt()       → cross-cutting instructions (date, citations, etc.)
  + memory_provider.to_prompt()             → memory instructions + recalled memories
  + context_provider.to_prompt()            → resolved context (when injection_mode=prompt)
  + interjections_provider.to_prompt()      → how to read interjected messages, when any source is on
  + repl_provider.to_prompt()               → the Python sandbox and stubs of the tools the turn's first round offers

Each source within a provider can contribute its own prompt section via a to_prompt() method. For example, IndexToolsSource adds guidance about how search works; DocumentToolsSource adds guidance about reading documents. The provider aggregates these.

4. LLM Loop

ZAVChatCompletionClient.complete() runs the turn through ChatTurnRunner. Before every model call it:

1. Adds the messages the user sent since the last call, read from the conversation's inbox and marked as sent mid-turn, then the interjection hooks' messages.
2. Reads the tools from the registry. Each provider's listing is cached, so this is a cheap read of the tools as they are now.
3. Runs the tool rule sources, from general to specific: each keeps or hides tools for this round and sets whether a kept tool asks first, whether it loads on demand, and who may call it (the model directly, Python in the REPL, or both).
4. Offers the model the kept tools it may call directly.

Before each call: inbox messages, tools → tool rules
Call the model with (system prompt, transcript, tools)
  └─ Tool calls:
       A call a rule's check_call declines does not run; the model gets the rule's message
       A call a rule's check_call asks about waits for the user, as below
       Tools that ask first wait for the user's approval (or are declined)
       Run the calls through this round's tools; a hidden tool gets "not found"
       The rules' check_result may replace or block each result before the model and the user see it
       Append the results and call the model again
  └─ No tool calls:
       Answer; a message that arrived meanwhile is answered in the same turn

A tool a rule hides is neither offered nor runnable: a call to it, from history or invented, gets "not found". An approval card shows for each call that asks first; the user can allow it once or for the session, edit its inputs, or deny it. A call nobody can be asked about, such as one from Python in the REPL, is declined unless the user already allowed the tool for the session.

The loop is capped by max_nesting_level, which defaults to 20 and is resolved by ZAVChatCompletionClientFactory when the completion client is injected. See SDK Runtime Configuration for the agent-level setting. Agent code can still pass max_nesting_level directly to client.complete() to override the configured default for one call. When the model keeps calling tools past that limit, the SDK makes one final call without tools, asking for its best answer from what it gathered, and emits {"max_nesting_level_reached": True} via log_fn. With no budget at all (max_nesting_level 0) it returns a static fallback message and emits {"max_nesting_level_exceeded": True}.

5. Post-Process

async for response, message in message_processing_provider.process_stream(raw_stream):
    yield message

The agent wraps the raw LLM output stream in a chain of MessageProcessor implementations. Each processor receives the full async generator of (ChatResponse, ChatMessage) pairs and yields transformed pairs — controlling its own buffering strategy. Processors compose via generator chaining: each wraps the previous one's output.

The primary processor is CitationProcessor, which passes through intermediate streaming chunks unchanged and resolves citation markers only on the final message.

6. Cleanup

After execution completes, the Agent iterates all instance attributes and calls aclose(), cleanup(), or async_cleanup() on anything that implements those methods — then deletes all attributes and triggers garbage collection. This ensures MCP connections, file handles, and other resources are released per-request.

Cross-Cutting Concerns — The agent_state Module

Some data needs to flow between components that don't have direct references to each other. The tool execution phase produces data that the post-processing phase consumes. These are handled by singleton request-scoped objects in the agent_state module.

CitationStore

The bridge between search tools and citation post-processing:

• During tool execution: whenever IndexToolsSource formats hits for the LLM — whether from search, browse, document listing, metadata retrieval, or document reading — it calls citation_store.register_hit() for each result. The store keeps the LLM-facing hit, the raw search hit, the evidence URL, optional short ID, index metadata, and a CitationContextContribution.
• During post-processing: CitationProcessor reads from the same CitationStore to resolve inline citation markers in the LLM's final response into structured evidence objects and next-turn conversation context. Internal search hits contribute DocumentContext document IDs. Federated and tool hits contribute CustomContextItem entries and keep their external evidence URLs.

ChatMessageEvidence.document_hit_url carries an evidence URL, despite its name: an internal /documents/... URL or an external URL. Consumers that fetch internal document data must first check that the URL starts with /documents/; an external URL is rendered as a link, not parsed as a document ID.

The earlier register_citation() API and the CitationFormat / RegisteredCitation models were replaced by register_hit(), CitationContextContribution, and RegisteredHit. Code that imported those internal SDK citation-state symbols should migrate to the new names and pass an explicit evidence_url plus either document_ids or custom_items for the context contribution.

The CitationStoreFactory is marked __singleton__ = True, so the same instance is shared across all components within a single request. Both the tool source and the message processor receive the same store through DI.

The citation strategy — how the LLM formats its references — is configured via CitationConfiguration:

Strategy	What the LLM outputs	What the processor does
inline_url	Full URLs in markdown links	Resolves to evidence with document metadata
short_id	Short hash IDs like [abc123]	Maps short ID → source URL → full evidence
deferred	Nothing — processor handles it	Matches response text to search results post-hoc
sup_numeric	Superscript numbers ¹²³	Maps numbers to search result order

ConversationImageStore

Handles images that appear in conversations:

• During context resolution: when context sources encounter images, they call image_store.register_image(base64_data) and get back an 8-character SHA256 hash.
• During tool execution: tools that need the original image data call image_store.get_image_data(short_id).

Like CitationStore, this is a singleton per request — the same instance bridges context resolution and tool execution.

The Pattern

Both stores follow the same pattern:

1. Registered via AgentDependencyFactory with __singleton__ = True
2. Auto-discovered through DI — any component that declares the store as a constructor parameter gets the same instance
3. No direct references between producer and consumer — they communicate through the shared store

This keeps the tool sources and processors decoupled. A ToolsSource doesn't need to know that a MessageProcessor exists; they share data through the store.

Source Filtering — The Valve Chain

Every provider uses the same filtering logic to decide which sources are active. The filter evaluates three inputs in order:

1. exclude_sources — hard veto. If a source name is in the exclude set, it's OFF regardless of anything else.
2. include_sources — allowlist. If defined and the source is in it, the source is ON (overrides the source's own enabled flag). If defined and the source is not in it, the source is OFF.
3. source_enabled — when no include list is defined, the source's own enabled flag decides.

def is_source_active(name, source_enabled, include, exclude) -> bool:
    if exclude is not None and name in exclude:
        return False          # exclude always wins
    if include is not None:
        return name in include  # allowlist overrides enabled flag
    return source_enabled       # fall back to source default

This creates a clean precedence chain. A source can ship as enabled=False by default (opt-in) or enabled=True (opt-out). The deployer's include_sources / exclude_sources in configuration always has the final say.

A second filter, passes_name_filter, applies the same logic at the individual tool level — a source might be active, but specific tools within it can be included or excluded.

Server Application and Request Lifecycle

The HTTP controller receives requests and routes them through the message bus to handlers. Handlers retrieve agent configuration (from tenant settings in managed deployments, from agent_setups.json locally), assemble the agent via ChatAgentFactory, and stream the response back.

The Complete Request Lifecycle

1. Request arrives at HTTP controller
2. Controller dispatches Command via message bus
3. Handler calls ChatAgentFactory.create(agent_identifier, bot_params)
      → Fetches AgentSetup from AgentSetupRetriever
      → Looks up agent class from ChatAgentClassRegistry
      → DI resolution: all constructor parameters resolved recursively
         - Providers via AgentDependencyFactory
         - Sources via DependencyGroup auto-collection
         - Singletons (CitationStore, ConversationImageStore) via resolution_cache
         - Config models from agent_configuration
4. Handler calls agent.execute_streaming(conversation)
      a. DispatchProvider.try_dispatch() — short-circuit if a rule matches
      b. ContextProvider.process_conversation() — interleave context into messages
      c. Assemble the system prompt from all providers, once per turn
      d. ZAVChatCompletionClient.complete() — the turn loop: before each model call,
         inbox messages, then tools from the providers' cached listings through the
         tool rules; tool calls run, asking for approval where set
      e. MessageProcessingProvider.process_stream() — post-process each response chunk
      f. Yield processed ChatMessage to caller
5. Cleanup: all provider/source attributes closed, GC triggered

The Resumable Turn Runtime

Everything above describes one request. Stateful conversations add a runtime around the turn so it survives the process that started it. Four pieces run alongside a stateful turn:

• The state checkpointer. A background worker calls dump() on every tracked resumable dependency every 5 seconds and once at the end of the turn, persisting the blobs to the configured ChatAgentStateStore. On the next turn (or a resume after a crash) the blobs are loaded back during DI resolution.
• The stream producer. A streaming turn is drained by a producer task that fans events out to connected clients through an in-memory StreamBuffer, flushes the latest cumulative message to the ChatStreamRecordingStore at most every 0.5 seconds, and stamps a heartbeat into the ChatStreamSupervisionStore every second. The turn runs to completion whether or not anyone is watching; disconnecting a client never cancels it.
• The supervision lease. The supervision row is a per-conversation mutex and ownership lease. Starting a turn claims the row with a monotonically increasing epoch; a fresh RUNNING row held by a different turn refuses the claim (the 409 the API returns). Every producer write is guarded by its epoch, so when a newer owner takes a presumed-dead turn over, the old producer's writes start failing and it stops: two replicas never produce one conversation.
• The follower. A client reattaching on a replica without the live buffer is served from the recording: the follower polls the single overwritten snapshot, paces new content out as a smooth stream, and classifies the supervision row into a verdict - done, errored, dead (stale heartbeat, resumable from the last checkpoint), or gone (the conversation moved on; refetch it). Liveness is always decided by the reader from the heartbeat's age, never stored, because a killed process writes nothing on the way down.

The durable transcript is separate from all of this: when a ChatConversationStore is configured, the handler appends the user turn before the agent runs and the final assistant message after it completes, which is what lets stateful clients send only the new turn.

The storage interfaces behind these pieces, their contracts, and the bundled file-backed implementations are covered in Persisting Conversations and State; the client-visible behavior is specified in Conversations and Streaming.

Agent Base Classes

Three base classes support different agent patterns:

• ChatAgent — core base with execute(), tracing instrumentation, ToolsRegistry, and cleanup lifecycle
• StreamableChatAgent — extends ChatAgent with execute_streaming() for streaming responses. The built-in Agent extends this.
• ProcessorAgent — non-chat agent for document processing pipelines (not covered here)

Register agent classes with the @ChatAgentClassRegistry.register() decorator. The agent_name class attribute maps to the agent_name field in AgentSetup.