Observability for OpenAI Agents SDK with Agenta

Learn how to set up tracing for your OpenAI Agents SDK applications with Agenta for complete observability into your multi-agent application performance, debugging capabilities, and execution observability.

What is Agenta? Agenta is an open-source LLMOps platform for building reliable LLM applications. It offers observability, evaluation, and prompt management capabilities for AI applications.

What is OpenAI Agents? The OpenAI Agents SDK is a framework for building AI agents that can interact with tools, run asynchronous operations, and orchestrate complex multi-agent workflows. It provides a structured approach to creating intelligent agents with tool calling capabilities.

info

This tutorial is also available as a Jupyter notebook for interactive learning and experimentation.

Setup Guide

Follow these steps to integrate OpenAI Agents SDK with Agenta's observability platform for real-time tracing and performance analysis.

Install Required Packages

First, install the required dependencies:

pip install agenta openinference-instrumentation-openai-agents openai-agents

Package Overview:

agenta: The core Agenta SDK for prompt engineering and observability
openai-agents: The OpenAI Agents SDK framework for building multi-agent systems
openinference-instrumentation-openai-agents: Automatic instrumentation library for OpenAI Agents operations

Setup and Configuration

Initialize your environment and configure the Agenta SDK:

import os
import asyncio
import agenta as ag
from agents import Agent, Runner
from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor

# Load configuration from environment
os.environ["AGENTA_API_KEY"] = "your_agenta_api_key"
os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai"  # Optional, defaults to the Agenta cloud API
os.environ["OPENAI_API_KEY"] = "your_openai_api_key"  # Required for OpenAI Agents SDK

# Start Agenta SDK
ag.init()

What does ag.init() do? The ag.init() function initializes the Agenta SDK and sets up the configuration for observability. It establishes connection to the Agenta platform, configures tracing and logging settings, and prepares the instrumentation context for your application.

Enable OpenAI Agents Monitoring

Initialize the OpenInference OpenAI Agents instrumentation to automatically capture agent operations:

# Enable OpenAI Agents instrumentation
OpenAIAgentsInstrumentor().instrument()

Build Your Instrumented Multi-Agent Application

Here's a complete example showcasing a multi-agent translation system with Agenta instrumentation:

import os
import asyncio
import agenta as ag
from agents import Agent, Runner
from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor

# Configuration setup
os.environ["AGENTA_API_KEY"] = "your_agenta_api_key"
os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai"  # Optional, defaults to the Agenta cloud API
os.environ["OPENAI_API_KEY"] = "your_openai_api_key"  # Required for OpenAI Agents SDK

# Start Agenta observability
ag.init()

# Enable OpenAI Agents instrumentation
OpenAIAgentsInstrumentor().instrument()

# Define specialized translation agents
spanish_agent = Agent(
    name="Spanish agent",
    instructions="You translate the user's message to Spanish",
)

french_agent = Agent(
    name="French agent",
    instructions="You translate the user's message to French",
)

german_agent = Agent(
    name="German agent",
    instructions="You translate the user's message to German",
)

# Create orchestrator agent with tool integration
orchestrator_agent = Agent(
    name="orchestrator_agent",
    instructions=(
        "You are a translation agent. You use the tools given to you to translate."
        "If asked for multiple translations, you call the relevant tools."
    ),
    tools=[
        spanish_agent.as_tool(
            tool_name="translate_to_spanish",
            tool_description="Translate the user's message to Spanish",
        ),
        french_agent.as_tool(
            tool_name="translate_to_french",
            tool_description="Translate the user's message to French",
        ),
        german_agent.as_tool(
            tool_name="translate_to_german",
            tool_description="Translate the user's message to German",
        ),
    ],
)

@ag.instrument()
async def duolingo(query: str):
    result = await Runner.run(orchestrator_agent, input=query)
    return result.final_output


# Example usage
if __name__ == "__main__":
    async def main():
        response = await duolingo("What is 'What's your name' in French?")
        print("Response:", response)

    asyncio.run(main())

Understanding the @ag.instrument() Decorator

The @ag.instrument() decorator automatically captures all input and output data from your function, enabling observability without manual instrumentation.

View Traces in Agenta

Once your application runs, access detailed execution traces through Agenta's dashboard. The observability data includes:

End-to-end multi-agent workflow execution timeline
Each Agent's inputs and outputs
Tool calling and inter-agent communication
Cost metrics, usage and duration metrics for each step

The trace provides visibility into your application's execution, helping you:

Debug complex multi-agent interactions and tool calling
Monitor agent performance and decision-making patterns
Analyze orchestration effectiveness and workflow optimization, and identify bottlenecks

info

After setting up observability for your OpenAI Agents SDK application, you can use Agenta's evaluation features to evaluate the performance of your agents.

Real-world Example

Customer Support Agent System

support_agent = Agent(
    name="Support Agent",
    instructions="You help customers with their inquiries and route complex issues to specialists",
)

billing_agent = Agent(
    name="Billing Agent", 
    instructions="You handle billing-related questions and payment issues",
)

technical_agent = Agent(
    name="Technical Agent",
    instructions="You provide technical support and troubleshooting assistance",
)

@ag.instrument(spankind="workflow")
async def customer_support_system(customer_query: str):
    orchestrator = Agent(
        name="Support Orchestrator",
        instructions="Route customer queries to the appropriate specialist agent",
        tools=[
            support_agent.as_tool("general_support", "Handle general customer inquiries"),
            billing_agent.as_tool("billing_support", "Handle billing and payment issues"),
            technical_agent.as_tool("technical_support", "Provide technical assistance"),
        ]
    )
    
    result = await Runner.run(orchestrator, input=customer_query)
    return result.final_output

Research Analysis Team

@ag.instrument(spankind="chain")
async def research_analysis_pipeline(research_topic: str):
    data_collector = Agent(
        name="Data Collector",
        instructions="Gather relevant information and data on the given topic",
    )
    
    analyst = Agent(
        name="Research Analyst", 
        instructions="Analyze collected data and provide insights",
    )
    
    reporter = Agent(
        name="Report Writer",
        instructions="Create reports based on analysis",
        tools=[
            data_collector.as_tool("collect_data", "Gather relevant research data"),
            analyst.as_tool("analyze_data", "Perform data analysis and insights"),
        ]
    )
    
    result = await Runner.run(reporter, input=research_topic)
    return result.final_output

Content Creation Workflow

@ag.instrument(spankind="workflow")
async def content_creation_system(content_brief: str):
    writer = Agent(
        name="Content Writer",
        instructions="Create engaging content based on the brief",
    )
    
    editor = Agent(
        name="Content Editor",
        instructions="Review and improve content quality and style",
    )
    
    seo_specialist = Agent(
        name="SEO Specialist",
        instructions="Optimize content for search engines",
    )
    
    content_manager = Agent(
        name="Content Manager",
        instructions="Coordinate the content creation process and ensure quality",
        tools=[
            writer.as_tool("write_content", "Generate initial content draft"),
            editor.as_tool("edit_content", "Review and edit content"),
            seo_specialist.as_tool("optimize_seo", "Optimize content for SEO"),
        ]
    )
    
    result = await Runner.run(content_manager, input=content_brief)
    return result.final_output

Next Steps

For more detailed information about Agenta's observability features and advanced configuration options, visit the Agenta Observability SDK Documentation.

Setup Guide​

Install Required Packages​

Setup and Configuration​

Enable OpenAI Agents Monitoring​

Build Your Instrumented Multi-Agent Application​

View Traces in Agenta​

Real-world Example​

Customer Support Agent System​

Research Analysis Team​

Content Creation Workflow​

Next Steps​