Add support for agentic workflow optimization and cost-aware model routing

[aanshshah]

Feature Request: Agentic workflow optimization support

Problem

Current optimization targets single prompts, but modern LLM apps use multi-step agent workflows. These have different challenges:

• Agents make 10-100x more API calls than single prompts
• Need to optimize entire workflows, not just individual steps
• Different models work better for different workflow steps
• Hard to evaluate multi-step success vs. individual responses

Proposed Solution

Add workflow optimization that can optimize multi-step agent pipelines with cost-aware model routing.

Basic Usage

# Define multi-step workflow
workflow = AgentWorkflow([
    PlanningStep(models=["nova-pro", "claude-sonnet"]),
    ReasoningStep(models=["claude-sonnet", "gpt-4o"]),
    ToolCallingStep(models=["gpt-4o", "nova-lite"]),
    SynthesisStep(models=["nova-pro"])
])

# Optimize entire workflow
optimizer = WorkflowOptimizer(
    workflow=workflow,
    cost_budget=50.0,
    metric=workflow_success_metric
)

optimized_workflow = optimizer.optimize(dataset, metric)

Cost-Aware Model Routing

# Automatically route based on task complexity and cost
router = CostAwareRouter({
    "simple_tasks": "nova-lite",      # $0.0006/1k tokens
    "reasoning": "claude-sonnet",     # $0.003/1k tokens  
    "complex_coding": "gpt-4o"        # $0.005/1k tokens
})

workflow.add_router(router)

Key Features Needed

• Workflow adapter: Execute and track multi-step workflows
• Step-level optimization: Optimize prompts for each workflow step
• Model routing: Assign optimal models to different steps
• Workflow metrics: Evaluate entire pipeline success
• Cost tracking: Monitor costs across all workflow steps
• Context management: Optimize data passing between steps

Example Use Cases

Research Agent: Search (nova-lite) → Analysis (claude-sonnet) → Synthesis (nova-pro)
Coding Agent: Planning (claude-sonnet) → Implementation (gpt-4o) → Testing (nova-pro)
Support Agent: Classification (nova-lite) → Retrieval (nova-pro) → Response (claude-sonnet)

Why This Matters

1. Agent workflows are becoming standard - single prompts are less common/for prototypes
2. Cost explosion problem - agents can easily burn through budgets
3. No existing tools optimize multi-step workflows end-to-end
4. Model specialization - different models excel at different workflow steps

Consider

• Extend existing PromptAdapter to support workflow definitions
• Add WorkflowOptimizer that uses MIPROv2 across multiple steps
• Create workflow-specific metrics and evaluation methods
• Add cost tracking and budget allocation across steps

References

• LangGraph - Agent workflow patterns
• DSPy ReAct - Single-step agent optimization
• CrewAI - Multi-agent frameworks

[VikramShenoy97]

Thanks for the suggestion! Agent optimization is indeed on our roadmap for the Nova model family. We're currently evaluating the best approach for workflow optimization, and this feedback is helpful. We'll keep this issue open until we’ve internally aligned on a clear direction.