1.2 AI Fundamentals

Here are 8 things you need to know about Azure AI Agent service before diving in.

1.1 Definitions

The Azure AI Agent Service is a fully managed service designed to empower developers to securely build, deploy, and scale high-quality, and extensible AI agents without needing to manage the underlying compute and storage resources.

An AI Agent acts as a "smart" microservice that can be used to answer questions (RAG), perform actions, or completely automate workflows - by combining the power of generative AI models with tools that allow it to access and interact with real-world data sources.

1.2 Agent Creation

The Azure AI Agent service uses the same wire protocol as Azure OpenAI Assistants allowing you to use either OpenAI SDKs or Azure AI Foundry SDK for building agents. Getting started with a simple agent is a 3-step process.

1.2.1 Create Agent

First, create the agent by specifying:

• the model - it uses to execute tasks
• the instructions - on how it should complete tasks
• the tools - it can acccess for support tasks
• the resources - it can use to support tools

Here is a simple agent constructor:

# Create am agent with a model, instructions & tools
agent = project_client.agents.create_agent(
    model="gpt-4o-mini",
    name="my-agent",
    instructions="You are helpful agent",
    tools=code_interpreter.definitions,
    tool_resources=code_interpreter.resources,
)

1.2.2 Create Activity Thread

Next, create an activity thread for the agent to run in. Then create the first message on that thread with the user request (role: "user") as the task for the agent to complete. This effectively adds the first message into the thread messages queue.

# Create a thread with messages
thread = project_client.agents.create_thread()
message = project_client.agents.create_message(
    thread_id=thread.id,
    role="user",
    content="Could you please create a bar chart for the operating profit using the following data and provide the file to me? Company A: $1.2 million, Company B: $2.5 million, Company C: $3.0 million, Company D: $1.8 million",
)

1.2.3 Run Agent in Thread

Finally, create an execution run instance with the thread and agent instances from above. Note that the agent simply retrieves the first message in the thread, to understand its task.

The agent will continue to update the thread message queue with new messages that reflect the ongoing conversation between the user, the agent, and other tools.

# Ask the agent to perform work on the thread
run = project_client.agents.create_and_process_run(thread_id=thread.id, agent_id=agent.id)

# Fetch and log all messages to see the agent's response
messages = project_client.agents.list_messages(thread_id=thread.id)
print(f"Messages: {messages}")

1.3 Agent Coordination

To build more complex workflows, you can have agents sending instructions to other agents (effectively representing the user in the context of delegating that task) and using frameworks like Semantic Kernel or AutoGen to orchestrate workflows and coordinate conversations in effective ways, to deliver the end response.

Because Azure AI Agent Service works seamlessly with these frameworks and is a managed service, you can get all the benefits of security, scale, and safety, without added effort.

1.4 Agent AI Service Benefits

Why use Azure AI Agent Service for orchestration instead of manually doing coordination using the Inference API for each model?

1. Automatic tool calling - now handled for you server-side, reducing client complexity
2. Secure data management - threads store conversation state for you
3. Rich tool ecosystem - includes file retrieval, code interpreter, Bing, Azure AI Search, Azure Functions

Why use Azure AI Agent Service for development instead of Azure OpenAI assistants?

1. Flexible model selection - pick models beyond Azure OpenAI e.g., Cohere, Llama 3
2. Extensive data connectors - integrate Bing, SharePoint, Fabric, AI Search, other API
3. Enterprise grade security - data privacy, compliance, keyless auth, secure data etc.
4. Flexible storage options - bring your own blog storage or use managed storage options

Plus, Azure AI Agent Service follows Responsible AI guidelines:

1. Responsible AI principles - for trustworthy AI
2. Content filters - for safety
3. Information & guidance - for customers

1.5 Agent AI Service: Limitations

1. Agent implementations have to work within the quota availability of underlying models
2. Agent AI Service has other limits for usage:
   • max files per agent/thread = 10K (API or Azure AI Foundry) or 20 (AOAI Studio)
   • max file size for agents = 512MB
   • max size for uploaded files = 100GB
   • agents token limit = 2M tokens
3. Agent AI Service supports additional models. Just deploy them in the same project before using in agents. Models currently limited to
   • Llama 3.1-70B-instruct
   • Mistral-large-2407
   • Cohere command R+

1.6 Agent AI Service: Costs

1. Understand Azure OpenAI deployment types to determine the differences in billing and performance for usage at scale.
2. Understand Azure AI Foundry pricing with respect to other AI services and tooling.

The FAQ breaks down other elements of cost including inference costs, tool usage costs, and storage costs.

1.7 About: Observability

Multi-agent workflows can be harder to debug since control flows through different threads and with different levels of parallelism and nesting.

Tracing support gives you built-in observability using OpenTelemetry standards for data. Trace data can then be exported to relevant OpenTelemetryProtocol (OLTP) compatible dashboards - including by:

• using console tracing (locally)
• using Azure monitor (in Azure AI Foundry portal)

1.8 About: Available Tools

When you create AI agents, you can specify different tools in order to help ground the model they use, or extend their capabilities. These are categorized as:

1. Knowledge Tools - give the agent access to data sources for grounding responses. Think of use cases for agentic RAG.
2. Action Tools - enhance the agent's capabilities by allowing it to run various tools at runtime. Think of search, code interpreter etc.

---