§ 04 · Orchestration
The orchestration layer is the agent runtime. It holds the planner, the memory, the tool-dispatch loop, and the re-planning logic. Every other layer exists to serve decisions made here.
§ Role in the stack
An orchestration framework takes a user objective, produces a plan, dispatches tool calls against the lower layers (semantic, governance, data), inspects the results, and decides whether to continue, branch, or finish. This is the loop that distinguishes agentic systems from single-shot text-to-SQL.
The tools available here are well populated: LangGraph, CrewAI, the Anthropic and OpenAI agent SDKs, LlamaIndex, AutoGen, Semantic Kernel. The dominant pattern today is a graph or finite-state machine over nodes that call LLMs, tools, and evaluators. The shift from 2024 to 2026 is that frameworks no longer need to invent their own tool protocol: MCP has become the portable interface between agents and everything below them.
Picking a framework matters less than picking the protocols it speaks. A framework-native tool definition will be locked to that framework. An MCP server or an OpenAPI spec will work across frameworks, and will survive the next framework rewrite. The same logic applies to traces: OpenTelemetry GenAI semantic conventions make your observability portable across vendors.
§ Open standards
Four specifications carry most of the portability at this layer. If your orchestration layer speaks these, you retain flexibility on framework, model, and observability vendor.
Open protocol for connecting agents to tools, resources, and prompts. Now the de facto standard for exposing data and actions to LLM runtimes. Analytics-specific MCP servers exist for Snowflake, BigQuery, dbt, and most catalog tools.
Specification →Standard trace attributes for LLM calls, tool invocations, and agent spans. Makes observability portable across vendors. If your agent framework emits these, your traces work in Langfuse, Grafana, Honeycomb, and Datadog without rewrites.
Specification →Protocol for discovery, messaging, and handoff between independent agents. Matters when analytics agents hand off to domain agents, or when organizations run agents built on different frameworks.
Specification →The substrate for defining tools the agent calls. Most agent frameworks accept OpenAPI specs as tool definitions directly. Keeps your analytics action surface declarative rather than wired into framework-specific code.
Specification →§ Failure modes
Planner thrashing. The agent re-plans repeatedly without converging on the investigation, usually when the planner prompt is under-constrained or the available tools overlap in unclear ways. Symptom: many LLM calls, few tool calls. Fix: cap re-plan iterations and write tool descriptions that clearly disambiguate.
Tool call loops. The agent calls the same tool with slight variations of the same arguments. Usually caused by missing error semantics: the tool returns empty results silently, and the planner interprets that as "try again with different phrasing." Fix: return structured errors with actionable messages.
Unobservable decisions. Without trace coverage on planner inputs and outputs, "why did the agent do that?" becomes unanswerable. In production this is disqualifying. Emit spans that cover every LLM call, every tool call, and every plan revision, and tag them with the OpenTelemetry GenAI conventions so that any backend can render them.
Context overflow. Long agent runs accumulate conversation state, tool results, and plan history. Past a threshold, performance degrades: later tool calls are worse than earlier ones. The fix is architectural (context compression, summary memory, sub-agent scoping), not a prompt tweak.