Orchestration
Most modern AI systems are not a single model call. They are complex pipelines of decisions, workflows, and retrieval.
The orchestration layer determines how prompts are structured, how agents iterate and loop through tasks, how requests are routed across models, and how external knowledge is applied.
Prompt & Workflow Frameworks
Prompt frameworks define structured sequences for tasks where the steps are already known
Example User Question
"What were Nvidia's earnings results and the analyst reaction?"
- 1) Retrieve earnings call transcript
- 2) Extract financial metrics
- 3) Summarize management commentary
- 4) Summarize analyst reaction
Prompt frameworks are best when the workflow is fixed. There is no recursion, planning, or iteration.
Agent Frameworks
Agent frameworks handle multistep tasks where the path is not predefined
Example User Question
"Is NVIDIA overvalued?"
- 1) Retrieve recent earnings reports
- 2) Review analyst ratings and targets
- 3) Evaluate whether evidence is sufficient
- 4) If not, gather more information, compare with competitors, and repeat analysis
Agent frameworks evaluate intermediate results and decide whether additional information or analysis is required.
Routing & Gateways
Model routing dynamically selects the appropriate model based on the complexity of the request
Example User Question
"What is NVIDIA trading at?"
- 1) Simple request, pull the current stock price via API
- 2) Moderate analysis, calculate valuation metrics like P/E or revenue multiple
- 3) Complex analysis, evaluate analyst sentiment and compare forward projections
The system routes each task to the appropriate model depending on the level of reasoning required.
Retrieval & Context
Retrieval systems provide models with proprietary or domain-specific data during inference
Example User Question
"Based on our analysis, should we buy NVIDIA stock?"
- 1) Retrieve internal research notes and analysis
- 2) Retrieve internal models and projections
- 3) Inject the context into the model prompt
- 4) Generate a response based on internal analysis
The model reasons over retrieved proprietary data rather than relying solely on training data.