Skip to content

Commit ea3215e

Browse files
moonbox3moonbox3
authored
Python: Support Declarative Agent Spec for ChatCompletionAgent & AzureAIAgent (#11982)
### Motivation and Context This is the first installment of a series of PRs related to adding single agent declarative spec support. SK .NET has had support for single agent declarative specs for a bit now, and Python needs the functionality as well. This PR adds the core abstractions and functionality to be able to define agents via a declarative yaml spec. Declarative specs require the use of a kernel, and if in the case of function calling, require that any plugins defined in the spec already be present in the kernel. For naming, there are placeholder names that are camel case. This needs to be the case so it can match .NET. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Provide declarative spec support for ChatCompletionAgent and AzureAIAgent - Adds many samples to show the use of declarative specs to handle various AzureAIAgent + ChatCompletionAgent tools/plugins. - Closes #11978 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄
1 parent bb3c065 commit ea3215e

File tree

44 files changed

+3168
-91
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+3168
-91
lines changed

‎python/pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ autogen = [
6565
"autogen-agentchat >= 0.2, <0.4"
6666
]
6767
aws = [
68-
"boto3>=1.36.4,<1.38.0",
68+
"boto3>=1.36.4,<1.39.0",
6969
]
7070
azure = [
7171
"azure-ai-inference >= 1.0.0b6",

‎python/samples/concepts/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,13 @@
1313
- [Azure AI Agent with Bing Grounding Streaming with Message Callback](./agents/azure_ai_agent/azure_ai_agent_bing_grounding_streaming_with_message_callback.py)
1414
- [Azure AI Agent with Bing Grounding](./agents/azure_ai_agent/azure_ai_agent_bing_grounding.py)
1515
- [Azure AI Agent with Code Interpreter Streaming with Message Callback](./agents/azure_ai_agent/azure_ai_agent_code_interpreter_streaming_with_message_callback.py)
16+
- [Azure AI Agent Declarative with Azure AI Search](./agents/azure_ai_agent/azure_ai_agent_declarative_azure_ai_search.py)
17+
- [Azure AI Agent Declarative with Bing Grounding](./agents/azure_ai_agent/azure_ai_agent_declarative_bing_grounding.py)
18+
- [Azure AI Agent Declarative with Code Interpreter](./agents/azure_ai_agent/azure_ai_agent_declarative_code_interpreter.py)
19+
- [Azure AI Agent Declarative with File Search](./agents/azure_ai_agent/azure_ai_agent_declarative_file_search.py)
20+
- [Azure AI Agent Declarative with Function Calling From File](./agents/azure_ai_agent/azure_ai_agent_declarative_function_calling_from_file.py)
21+
- [Azure AI Agent Declarative with OpenAPI Interpreter](./agents/azure_ai_agent/azure_ai_agent_declarative_openapi.py)
22+
- [Azure AI Agent Declarative with Existing Agent ID](./agents/azure_ai_agent/azure_ai_agent_declarative_with_existing_agent_id.py)
1623
- [Azure AI Agent File Manipulation](./agents/azure_ai_agent/azure_ai_agent_file_manipulation.py)
1724
- [Azure AI Agent Prompt Templating](./agents/azure_ai_agent/azure_ai_agent_prompt_templating.py)
1825
- [Azure AI Agent Message Callback Streaming](./agents/azure_ai_agent/azure_ai_agent_message_callback_streaming.py)

‎python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_azure_ai_search.py

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
5+
from azure.identity.aio import DefaultAzureCredential
6+
7+
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
8+
from semantic_kernel.agents.agent import AgentRegistry
9+
from semantic_kernel.contents.chat_message_content import ChatMessageContent
10+
from semantic_kernel.contents.function_call_content import FunctionCallContent
11+
from semantic_kernel.contents.function_result_content import FunctionResultContent
12+
13+
"""
14+
The following sample demonstrates how to create an Azure AI agent that answers
15+
user questions using the Azure AI Search tool.
16+
17+
The agent is created using a YAML declarative spec that configures the
18+
Azure AI Search tool. The agent is then used to answer user questions
19+
that required grounding context from the Azure AI Search index.
20+
21+
Note: the `AzureAISearchConnectionId` is in the format of:
22+
/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.MachineLearningServices/workspaces/<workspace>/connections/AzureAISearch
23+
24+
It can either be configured as an env var `AZURE_AI_AGENT_BING_CONNECTION_ID` or passed in as an extra to
25+
`create_from_yaml`: extras={
26+
"AzureAISearchConnectionId": "<azure_ai_search_connection_id>",
27+
"AzureAISearchIndexName": "<azure_ai_search_index_name>"
28+
}
29+
"""
30+
31+
# Define the YAML string for the sample
32+
spec = """
33+
type: foundry_agent
34+
name: AzureAISearchAgent
35+
instructions: Answer questions using your index to provide grounding context.
36+
description: This agent answers questions using AI Search to provide grounding context.
37+
model:
38+
id: ${AzureAI:ChatModelId}
39+
options:
40+
temperature: 0.4
41+
tools:
42+
- type: azure_ai_search
43+
options:
44+
tool_connections:
45+
- ${AzureAI:AzureAISearchConnectionId}
46+
index_name: ${AzureAI:AzureAISearchIndexName}
47+
"""
48+
49+
settings = AzureAIAgentSettings() # ChatModelId comes from .env/env vars
50+
51+
52+
async def main():
53+
async with (
54+
DefaultAzureCredential() as creds,
55+
AzureAIAgent.create_client(credential=creds) as client,
56+
):
57+
try:
58+
# Create the AzureAI Agent from the YAML spec
59+
# Note: the extras can be provided in the short-format (shown below) or
60+
# in the long-format (as shown in the YAML spec, with the `AzureAI:` prefix).
61+
# The short-format is used here for brevity
62+
agent: AzureAIAgent = await AgentRegistry.create_from_yaml(
63+
yaml_str=spec,
64+
client=client,
65+
settings=settings,
66+
extras={
67+
"AzureAISearchConnectionId": "<azure-ai-search-connection-id>",
68+
"AzureAISearchIndexName": "<azure-ai-search-index-name>",
69+
},
70+
)
71+
72+
# Define the task for the agent
73+
TASK = "What is Semantic Kernel?"
74+
75+
print(f"# User: '{TASK}'")
76+
77+
# Define a callback function to handle intermediate messages
78+
async def on_intermediate_message(message: ChatMessageContent):
79+
if message.items:
80+
for item in message.items:
81+
if isinstance(item, FunctionCallContent):
82+
print(f"# FunctionCallContent: arguments={item.arguments}")
83+
elif isinstance(item, FunctionResultContent):
84+
print(f"# FunctionResultContent: result={item.result}")
85+
86+
# Invoke the agent for the specified task
87+
async for response in agent.invoke(messages=TASK, on_intermediate_message=on_intermediate_message):
88+
print(f"# {response.name}: {response}")
89+
finally:
90+
# Cleanup: Delete the agent
91+
await client.agents.delete_agent(agent.id)
92+
93+
94+
if __name__ == "__main__":
95+
asyncio.run(main())

‎python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_bing_grounding.py

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
5+
from azure.identity.aio import DefaultAzureCredential
6+
7+
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
8+
from semantic_kernel.agents.agent import AgentRegistry
9+
10+
"""
11+
The following sample demonstrates how to create an Azure AI agent that answers
12+
user questions using the Bing Grounding tool.
13+
14+
The agent is created using a YAML declarative spec that configures the
15+
Bing Grounding tool. The agent is then used to answer user questions
16+
that require web search to answer correctly.
17+
18+
Note: the `BingConnectionId` is in the format of:
19+
/subscriptions/<sub_id>/resourceGroups/<rg>/providers/Microsoft.MachineLearningServices/workspaces/<workspace>/connections/<bing_connection_id>
20+
21+
It can either be configured as an env var `AZURE_AI_AGENT_BING_CONNECTION_ID` or passed in as an extra to
22+
`create_from_yaml`: extras={"BingConnectionId": "<bing_connection_id>"}
23+
"""
24+
25+
# Define the YAML string for the sample
26+
spec = """
27+
type: foundry_agent
28+
name: BingAgent
29+
instructions: Answer questions using Bing to provide grounding context.
30+
description: This agent answers questions using Bing to provide grounding context.
31+
model:
32+
id: ${AzureAI:ChatModelId}
33+
options:
34+
temperature: 0.4
35+
tools:
36+
- type: bing_grounding
37+
options:
38+
tool_connections:
39+
- ${AzureAI:BingConnectionId}
40+
"""
41+
42+
settings = AzureAIAgentSettings() # ChatModelId & BingConnectionId come from .env/env vars
43+
44+
45+
async def main():
46+
async with (
47+
DefaultAzureCredential() as creds,
48+
AzureAIAgent.create_client(credential=creds) as client,
49+
):
50+
try:
51+
# Create the AzureAI Agent from the YAML spec
52+
agent: AzureAIAgent = await AgentRegistry.create_from_yaml(
53+
yaml_str=spec,
54+
client=client,
55+
settings=settings,
56+
)
57+
58+
# Define the task for the agent
59+
TASK = "Who won the 2025 NCAA basketball championship?"
60+
61+
print(f"# User: '{TASK}'")
62+
63+
# Invoke the agent for the specified task
64+
async for response in agent.invoke(
65+
messages=TASK,
66+
):
67+
print(f"# {response.name}: {response}")
68+
finally:
69+
# Cleanup: Delete the thread and agent
70+
await client.agents.delete_agent(agent.id)
71+
72+
"""
73+
Sample output:
74+
75+
# User: 'Who won the 2025 NCAA basketball championship?'
76+
# BingAgent: The Florida Gators won the 2025 NCAA men's basketball championship, narrowly defeating the Houston
77+
Cougars 65-63 in the final game. This marked Florida's first national title since
78+
2007【3:5†source】【3:9†source】.
79+
"""
80+
81+
82+
if __name__ == "__main__":
83+
asyncio.run(main())

‎python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_code_interpreter.py

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
import os
5+
6+
from azure.ai.projects.models import FilePurpose
7+
from azure.identity.aio import DefaultAzureCredential
8+
9+
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
10+
from semantic_kernel.agents.agent import AgentRegistry
11+
12+
"""
13+
The following sample demonstrates how to create an Azure AI agent that answers
14+
user questions using the code interpreter tool.
15+
16+
The agent is then used to answer user questions that require code to be generated and
17+
executed. The responses are handled in a streaming manner.
18+
"""
19+
20+
# Define the YAML string for the sample
21+
spec = """
22+
type: foundry_agent
23+
name: CodeInterpreterAgent
24+
description: Agent with code interpreter tool.
25+
instructions: >
26+
Use the code interpreter tool to answer questions that require code to be generated
27+
and executed.
28+
model:
29+
id: ${AzureAI:ChatModelId}
30+
connection:
31+
connection_string: ${AzureAI:ConnectionString}
32+
tools:
33+
- type: code_interpreter
34+
options:
35+
file_ids:
36+
- ${AzureAI:FileId1}
37+
"""
38+
39+
settings = AzureAIAgentSettings() # ChatModelId & ConnectionString come from env vars
40+
41+
42+
async def main():
43+
async with (
44+
DefaultAzureCredential() as creds,
45+
AzureAIAgent.create_client(credential=creds) as client,
46+
):
47+
# Create the CSV file path for the sample
48+
csv_file_path = os.path.join(
49+
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
50+
"resources",
51+
"agent_assistant_file_manipulation",
52+
"sales.csv",
53+
)
54+
55+
try:
56+
# Upload the CSV file to the agent service
57+
file = await client.agents.upload_file_and_poll(file_path=csv_file_path, purpose=FilePurpose.AGENTS)
58+
59+
# Create the AzureAI Agent from the YAML spec
60+
# Note: the extras can be provided in the short-format (shown below) or
61+
# in the long-format (as shown in the YAML spec, with the `AzureAI:` prefix).
62+
# The short-format is used here for brevity
63+
agent: AzureAIAgent = await AgentRegistry.create_from_yaml(
64+
yaml_str=spec,
65+
client=client,
66+
settings=settings,
67+
extras={"FileId1": file.id},
68+
)
69+
70+
# Define the task for the agent
71+
TASK = "Give me the code to calculate the total sales for all segments."
72+
73+
print(f"# User: '{TASK}'")
74+
75+
# Invoke the agent for the specified task
76+
is_code = False
77+
last_role = None
78+
async for response in agent.invoke_stream(
79+
messages=TASK,
80+
):
81+
current_is_code = response.metadata.get("code", False)
82+
83+
if current_is_code:
84+
if not is_code:
85+
print("\n\n```python")
86+
is_code = True
87+
print(response.content, end="", flush=True)
88+
else:
89+
if is_code:
90+
print("\n```")
91+
is_code = False
92+
last_role = None
93+
if hasattr(response, "role") and response.role is not None and last_role != response.role:
94+
print(f"\n# {response.role}: ", end="", flush=True)
95+
last_role = response.role
96+
print(response.content, end="", flush=True)
97+
if is_code:
98+
print("```\n")
99+
print()
100+
finally:
101+
# Cleanup: Delete the thread and agent
102+
await client.agents.delete_agent(agent.id)
103+
await client.agents.delete_file(file.id)
104+
105+
"""
106+
Sample output:
107+
108+
# User: 'Give me the code to calculate the total sales for all segments.'
109+
110+
# AuthorRole.ASSISTANT: Let me first examine the contents of the uploaded file to determine its structure. This
111+
will allow me to create the appropriate code for calculating the total sales for all segments. Hang tight!
112+
113+
```python
114+
import pandas as pd
115+
116+
# Load the uploaded file to examine its contents
117+
file_path = '/mnt/data/assistant-3nXizu2EX2EwXikUz71uNc'
118+
data = pd.read_csv(file_path)
119+
120+
# Display the first few rows and column names to understand the structure of the dataset
121+
data.head(), data.columns
122+
```
123+
124+
# AuthorRole.ASSISTANT: The dataset contains several columns, including `Segment`, `Sales`, and others such as
125+
`Country`, `Product`, and date-related information. To calculate the total sales for all segments, we will:
126+
127+
1. Group the data by the `Segment` column.
128+
2. Sum the `Sales` column for each segment.
129+
3. Calculate the grand total of all sales across all segments.
130+
131+
Here is the code snippet for this task:
132+
133+
```python
134+
# Group by 'Segment' and sum up 'Sales'
135+
segment_sales = data.groupby('Segment')['Sales'].sum()
136+
137+
# Calculate the total sales across all segments
138+
total_sales = segment_sales.sum()
139+
140+
print("Total Sales per Segment:")
141+
print(segment_sales)
142+
print(f"\nGrand Total Sales: {total_sales}")
143+
```
144+
145+
Would you like me to execute this directly for the uploaded data?
146+
"""
147+
148+
149+
if __name__ == "__main__":
150+
asyncio.run(main())

0 commit comments

Comments
 (0)