48  Agent Architectures Overview

TipTL;DR

An agent perceives state, reasons about it, and selects actions to pursue a goal, iteratively, over multiple steps. The underlying model (Transformer, SSM, or otherwise) is one component; the architecture is how perception, memory, reasoning, and action are composed around it.

48.1 Why this matters

Every model covered so far in this book answers a single query: predict the next token, classify an image, denoise a patch. An agent is different in kind, not just in scale. It runs a loop: observe, decide, act, and observe the result of that action, repeating until a goal is reached or a budget is exhausted. This loop is what Embodied AI runs physically and what a coding assistant or research agent runs virtually; the pattern is the same regardless of whether the “action” is a motor command or a function call.

48.2 The mechanism

48.2.1 The perceive-reason-act loop

Every agent runs this cycle, model-agnostic: perceive the current state (text, images, sensor data), reason about what to do given the goal and history, act on the environment, then perceive the new state and repeat. This is the physical-world loop from Embodied AI, generalized: the environment can be a robot’s surroundings, a filesystem, a web browser, or another agent.

48.2.2 Memory

An agent’s memory of past steps can live in three places, each with a different cost and capacity tradeoff.

• In-context: prior steps stay in the model’s context window directly. Simple, but bounded by context length and expensive to attend over as it grows (see Efficient Attention and KV-Cache).
• Retrieval: prior steps or external knowledge are stored in an index and fetched on demand. Unbounded capacity, at the cost of retrieval latency and imperfect recall; the full mechanism is covered in RAG.
• External state: an explicit scratchpad, file, or database the agent reads and writes directly, outside the model’s context entirely.

48.2.3 Planning

Chain-of-thought (Scaling Laws) is the simplest planning mechanism: reasoning tokens generated before the final action. More elaborate agents add tree search (explore multiple candidate action sequences before committing) or reflection (critique and revise a plan after seeing its outcome), trading extra inference-time compute for a better chance of reaching the goal.

48.2.4 Backbone vs. orchestrator

A single model can play two different roles in an agent system. As a backbone, it is the perceive-reason-act loop’s reasoning engine directly. As an orchestrator, it coordinates other components, calling tools (Tool Use), delegating to other agents (Multi-Agent Systems), or querying a retrieval index (RAG), without doing all the reasoning itself in one forward pass.

48.3 Landscape of agent architectures

System	Backbone	Key capability
ReAct (Yao et al., 2022)	LLM	Interleaved reasoning and tool calls
Toolformer (Schick et al., 2023)	LLM	Self-taught tool use via fine-tuning
AutoGen (Wu et al., 2023)	LLM	Multi-agent conversation and orchestration
Generative Agents (Park et al., 2023)	LLM	Simulated agents with memory and social behavior
RT-2 / OpenVLA	VLA Transformer	Embodied physical action

NoteKey takeaway

An agent is a loop, not a model. The perceive-reason-act cycle, a memory strategy, and a planning mechanism are the architectural choices that turn a single-query model into a system that pursues a goal over many steps; the next three notes cover the three concrete patterns (retrieval, tool use, multi-agent coordination) that fill in this loop in practice.

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022). ReAct: Synergizing reasoning and acting in language models. arXiv Preprint arXiv:2210.03629.

Park, J. S., O’Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. arXiv Preprint arXiv:2304.03442.

Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. arXiv Preprint arXiv:2302.04761.

Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023). AutoGen: Enabling next-gen LLM applications via multi-agent conversation. arXiv Preprint arXiv:2308.08155.