AI LLM Orchestration Frameworks Cheat Sheet

Updated 2026-04-28

Next Topic: AI Reasoning Models Cheat Sheet

🧠Study flashcards on this topic104 cards · spaced repetition→

AI/LLM orchestration frameworks are the infrastructure layer that transforms isolated large language models into coordinated, production-ready agentic systems. These frameworks emerged to solve the fundamental challenge of building reliable multi-step workflows where AI agents must reason, plan, remember, delegate, recover from failures, and collaborate — capabilities that simple prompt-response patterns cannot provide. In 2026, the field consolidated around stateful graph-based architectures (LangGraph, Google ADK), multi-agent role systems (CrewAI, AG2), type-safe validation patterns (Pydantic AI), and lightweight code-first agents (smolagents), each optimized for distinct production use cases. The critical shift is from "prompting LLMs" to programming agent systems — treating orchestration as a software engineering discipline with observability, error handling, state management, and deterministic control flow.

What This Cheat Sheet Covers

This topic spans 12 focused tables and 110 indexed concepts, 104 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Major Orchestration FrameworksTable 2: Core Orchestration PatternsTable 3: State Management & MemoryTable 4: Tool Integration & Function CallingTable 5: Multi-Agent CoordinationTable 6: Error Handling & ResilienceTable 7: Observability & MonitoringTable 8: Workflow Orchestration FeaturesTable 9: Agent Planning & ReasoningTable 10: Advanced CapabilitiesTable 11: Production & DeploymentTable 12: Framework Selection Criteria

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Major Orchestration Frameworks

These are the libraries you'll actually choose between, and they cluster into a few camps — graph-based engines like LangGraph for explicit stateful control, role-based systems like CrewAI and AG2 for collaborating agents, type-safe single-agent tools like Pydantic AI, and the vendor SDKs from OpenAI, Microsoft, and Google. Read the rows less as a ranking and more as a map of trade-offs between control, ergonomics, and ecosystem.

Framework	Example	Description
LangGraph	`from langgraph.graph import StateGraph` `graph = StateGraph(State)` `graph.add_node("research", research_node)` `graph.add_edge("research", "write")`	• Graph-based workflow engine for building stateful, cyclic agent flows with explicit state management • supports branching, loops, human-in-the-loop, and checkpointing • most adopted multi-agent framework in 2026; optimized for complex multi-step reasoning.
CrewAI	`crew = Crew(` `agents=[researcher, writer],` `tasks=[research_task, write_task],` `process=Process.sequential` `)`	• Role-based multi-agent framework where agents collaborate through defined roles and responsibilities • supports sequential, hierarchical, and consensual process types • fastest path from idea to working multi-agent prototype.
OpenAI Agents SDK	`agent = Agent(` `model="gpt-4o",` `tools=[search_tool]` `)` `response = agent.run("query")`	• Production SDK from OpenAI replacing experimental Swarm (released March 2025) • built-in primitives: handoffs, guardrails, and end-to-end tracing • clean, opinionated handoff pattern; optimized for OpenAI models.
Microsoft Agent Framework	`agent = AssistantAgent(` `name="assistant",` `instructions="You are helpful"` `)` `session = AgentSession()`	• Unified successor to AutoGen and Semantic Kernel (GA target Q1 2026) • production-grade orchestration with session state, telemetry, agent skills, and enterprise features • optimized for .NET and Python; five built-in orchestration patterns.
Google ADK	`agent = Agent(` `name="MyAgent",` `model=gemini_model` `)` `agent.add_tool(SearchTool())`	• Google's modular agent framework (released April 2025) optimized for Gemini • native support for A2A cross-framework protocol and multimodal inputs • hierarchical agent tree; production integrations with GitHub, MongoDB, Jira.
Pydantic AI	`agent = Agent(` `model="openai:gpt-4o",` `result_type=MyModel` `)` `result = agent.run_sync(prompt)`	• Type-safe agent framework with Pydantic validation for structured outputs • FastAPI-style ergonomics; model-agnostic; best-in-class Python IDE support • minimal abstraction for developers prioritizing correctness and type safety.
AG2	`assistant = AssistantAgent(` `name="assistant", llm_config=cfg` `)` `user_proxy.run(assistant, message="...").process()`	• Community-maintained continuation of AutoGen v0.2 (renamed from AutoGen, fork managed by AG2AI organization) • supports group chat, swarm orchestration, CaptainAgent for auto team assembly, RAG, and code execution • Apache 2.0 license; 50k+ GitHub stars.
smolagents	`agent = CodeAgent(` `tools=[DuckDuckGoSearchTool()],` `model=InferenceClientModel()` `)` `result = agent.run("task")`	• Hugging Face's lightweight code-first agent framework — agent logic fits in ~1,000 lines • `CodeAgent` writes Python directly (loops, conditionals) enabling natural composability • model-agnostic; runs any HuggingFace Hub, OpenAI, or local model.
MetaGPT	`from metagpt.software_company import generate_repo` `repo = generate_repo(` `"Create a 2048 game"` `)`	• Software company simulation framework where agents hold roles (PM, Architect, Engineer, QA) • takes one-line requirement, outputs PRDs, designs, code, and tests • `Code = SOP(Team)` philosophy; 60k+ GitHub stars.
Mastra	`agent = new Agent({` `name: "assistant",` `tools: [searchTool]` `})` `await agent.generate(input)`	• TypeScript-first framework for web developers • unified workspace API for filesystem, sandbox execution, and search • built-in observability and streaming; by the Gatsby team.
LlamaIndex Workflows	`workflow = Workflow()` `@workflow.step` `async def retrieve(ctx, query):` `return index.query(query)`	• Event-driven agent workflows with async, multi-step orchestration • purpose-built for data-grounded RAG-first agents • tight integration with vector indexes and retrieval; best when agent's primary job is querying private knowledge.
Haystack	`pipeline = Pipeline()` `pipeline.add_node(retriever)` `pipeline.add_node(llm, inputs=["retriever"])` `result = pipeline.run(query)`	• Document-centric orchestration optimized for RAG, QA, and knowledge extraction • node-based pipelines with strong document processing and retrieval capabilities.
DSPy	`retriever = dspy.Retrieve(k=5)` `generator = dspy.ChainOfThought()` `compiled = dspy.Teleprompter()(pipeline)`	• Declarative prompt optimization framework that treats prompts as code • automatically optimizes instructions and few-shot examples via compilation • eliminates manual prompt engineering.
Semantic Kernel	`kernel = Kernel()` `kernel.add_plugin(MyPlugin())` `result = await kernel.invoke_prompt(` `"Generate summary"` `)`	• Plugin-based orchestration for integrating LLMs with enterprise services • function-calling abstraction with filters and telemetry • functionality merging into Microsoft Agent Framework, but maintained standalone.
AutoGen (legacy)	`user = UserProxyAgent(name="user")` `assistant = AssistantAgent(` `name="assistant"` `)` `user.initiate_chat(assistant)`	• Original conversational multi-agent framework by Microsoft Research • superseded: Microsoft's v0.4+ is evolving into Microsoft Agent Framework; the community v0.2 lineage continues as AG2 • legacy codebases still widespread.

Table 2: Core Orchestration Patterns

Underneath every framework sit a handful of control-flow shapes that decide how an agent moves through a task. ReAct interleaves thinking and acting step by step, Plan-and-Execute commits to a plan upfront, and the rest — sequential, hierarchical, parallel, router, loop, reflection, human-in-the-loop, swarm — are the structural choices you compose to balance flexibility, cost, and predictability.

Pattern	Example	Description
ReAct (Reason-Act)	`Thought: Need to search for X` `Action: search("X")` `Observation: Found Y` `Thought: Now I can answer`	• Interleaved reasoning and action where the agent alternates between thinking, taking actions, and observing results • every step re-plans based on new observations • high token usage but flexible; cornerstone pattern for most agent frameworks.

The rest of this section is part of the full cheat sheet

Table 3: State Management & Memory

An agent is only as capable as what it can remember, and memory spans a spectrum — from the conversation history sitting in the context window to external vector stores and dedicated memory layers like mem0 that persist facts across sessions. Checkpointing and session state keep a running workflow recoverable, while context engineering and summarization are the discipline of deciding what to keep, compress, or throw away as histories grow.

Technique	Example	Description
StateGraph (LangGraph)	`class State(TypedDict):` `messages: list[Message]` `data: dict` `graph = StateGraph(State)`	• Typed state container that flows through graph nodes • each node reads and updates state; supports reducers for merging parallel updates • persisted via checkpointers for crash recovery.

The rest of this section is part of the full cheat sheet

Table 4: Tool Integration & Function Calling

Tools are how an agent reaches outside its own text box to search, run code, or call an API, and it starts with declaring each tool as a JSON schema the model can decide to invoke. The newer entries reflect where the field is heading: MCP standardizing how agents discover external systems, AI gateways unifying hundreds of models behind one endpoint, and approval workflows that gate destructive actions behind a human.

Technique	Example	Description
Tool Definition (JSON Schema)	`tool = {` `"name": "search",` `"description": "Search web",` `"parameters": {...}` `}`	• Declare tools using JSON schema with name, description, and parameter definitions • LLM decides when and how to invoke tools • standard format across OpenAI, Anthropic, Google.

The rest of this section is part of the full cheat sheet

Table 1: Major Orchestration Frameworks

Framework	Example	Description
LangGraph	`from langgraph.graph import StateGraph` `graph = StateGraph(State)` `graph.add_node("research", research_node)` `graph.add_edge("research", "write")`	• Graph-based workflow engine for building stateful, cyclic agent flows with explicit state management • supports branching, loops, human-in-the-loop, and checkpointing • most adopted multi-agent framework in 2026; optimized for complex multi-step reasoning.
CrewAI	`crew = Crew(` `agents=[researcher, writer],` `tasks=[research_task, write_task],` `process=Process.sequential` `)`	• Role-based multi-agent framework where agents collaborate through defined roles and responsibilities • supports sequential, hierarchical, and consensual process types • fastest path from idea to working multi-agent prototype.
OpenAI Agents SDK	`agent = Agent(` `model="gpt-4o",` `tools=[search_tool]` `)` `response = agent.run("query")`	• Production SDK from OpenAI replacing experimental Swarm (released March 2025) • built-in primitives: handoffs, guardrails, and end-to-end tracing • clean, opinionated handoff pattern; optimized for OpenAI models.
Microsoft Agent Framework	`agent = AssistantAgent(` `name="assistant",` `instructions="You are helpful"` `)` `session = AgentSession()`	• Unified successor to AutoGen and Semantic Kernel (GA target Q1 2026) • production-grade orchestration with session state, telemetry, agent skills, and enterprise features • optimized for .NET and Python; five built-in orchestration patterns.
Google ADK	`agent = Agent(` `name="MyAgent",` `model=gemini_model` `)` `agent.add_tool(SearchTool())`	• Google's modular agent framework (released April 2025) optimized for Gemini • native support for A2A cross-framework protocol and multimodal inputs • hierarchical agent tree; production integrations with GitHub, MongoDB, Jira.
Pydantic AI	`agent = Agent(` `model="openai:gpt-4o",` `result_type=MyModel` `)` `result = agent.run_sync(prompt)`	• Type-safe agent framework with Pydantic validation for structured outputs • FastAPI-style ergonomics; model-agnostic; best-in-class Python IDE support • minimal abstraction for developers prioritizing correctness and type safety.
AG2	`assistant = AssistantAgent(` `name="assistant", llm_config=cfg` `)` `user_proxy.run(assistant, message="...").process()`	• Community-maintained continuation of AutoGen v0.2 (renamed from AutoGen, fork managed by AG2AI organization) • supports group chat, swarm orchestration, CaptainAgent for auto team assembly, RAG, and code execution • Apache 2.0 license; 50k+ GitHub stars.
smolagents	`agent = CodeAgent(` `tools=[DuckDuckGoSearchTool()],` `model=InferenceClientModel()` `)` `result = agent.run("task")`	• Hugging Face's lightweight code-first agent framework — agent logic fits in ~1,000 lines • `CodeAgent` writes Python directly (loops, conditionals) enabling natural composability • model-agnostic; runs any HuggingFace Hub, OpenAI, or local model.
MetaGPT	`from metagpt.software_company import generate_repo` `repo = generate_repo(` `"Create a 2048 game"` `)`	• Software company simulation framework where agents hold roles (PM, Architect, Engineer, QA) • takes one-line requirement, outputs PRDs, designs, code, and tests • `Code = SOP(Team)` philosophy; 60k+ GitHub stars.
Mastra	`agent = new Agent({` `name: "assistant",` `tools: [searchTool]` `})` `await agent.generate(input)`	• TypeScript-first framework for web developers • unified workspace API for filesystem, sandbox execution, and search • built-in observability and streaming; by the Gatsby team.
LlamaIndex Workflows	`workflow = Workflow()` `@workflow.step` `async def retrieve(ctx, query):` `return index.query(query)`	• Event-driven agent workflows with async, multi-step orchestration • purpose-built for data-grounded RAG-first agents • tight integration with vector indexes and retrieval; best when agent's primary job is querying private knowledge.
Haystack	`pipeline = Pipeline()` `pipeline.add_node(retriever)` `pipeline.add_node(llm, inputs=["retriever"])` `result = pipeline.run(query)`	• Document-centric orchestration optimized for RAG, QA, and knowledge extraction • node-based pipelines with strong document processing and retrieval capabilities.
DSPy	`retriever = dspy.Retrieve(k=5)` `generator = dspy.ChainOfThought()` `compiled = dspy.Teleprompter()(pipeline)`	• Declarative prompt optimization framework that treats prompts as code • automatically optimizes instructions and few-shot examples via compilation • eliminates manual prompt engineering.
Semantic Kernel	`kernel = Kernel()` `kernel.add_plugin(MyPlugin())` `result = await kernel.invoke_prompt(` `"Generate summary"` `)`	• Plugin-based orchestration for integrating LLMs with enterprise services • function-calling abstraction with filters and telemetry • functionality merging into Microsoft Agent Framework, but maintained standalone.
AutoGen (legacy)	`user = UserProxyAgent(name="user")` `assistant = AssistantAgent(` `name="assistant"` `)` `user.initiate_chat(assistant)`	• Original conversational multi-agent framework by Microsoft Research • superseded: Microsoft's v0.4+ is evolving into Microsoft Agent Framework; the community v0.2 lineage continues as AG2 • legacy codebases still widespread.

Table 2: Core Orchestration Patterns

Pattern	Example	Description
ReAct (Reason-Act)	`Thought: Need to search for X` `Action: search("X")` `Observation: Found Y` `Thought: Now I can answer`	• Interleaved reasoning and action where the agent alternates between thinking, taking actions, and observing results • every step re-plans based on new observations • high token usage but flexible; cornerstone pattern for most agent frameworks.

Table 3: State Management & Memory

Technique	Example	Description
StateGraph (LangGraph)	`class State(TypedDict):` `messages: list[Message]` `data: dict` `graph = StateGraph(State)`	• Typed state container that flows through graph nodes • each node reads and updates state; supports reducers for merging parallel updates • persisted via checkpointers for crash recovery.

Table 4: Tool Integration & Function Calling

Technique	Example	Description
Tool Definition (JSON Schema)	`tool = {` `"name": "search",` `"description": "Search web",` `"parameters": {...}` `}`	• Declare tools using JSON schema with name, description, and parameter definitions • LLM decides when and how to invoke tools • standard format across OpenAI, Anthropic, Google.

AI/LLM Orchestration Frameworks Cheat Sheet

Table 1: Major Orchestration Frameworks

Table 2: Core Orchestration Patterns

Table 3: State Management & Memory

Table 4: Tool Integration & Function Calling

AI/LLM Orchestration Frameworks Cheat Sheet

Table 1: Major Orchestration Frameworks

Table 2: Core Orchestration Patterns

Table 3: State Management & Memory

Table 4: Tool Integration & Function Calling