Integration Testing Framework

[rodrigobr-msft]

This pull request introduces a new testing framework for the Microsoft 365 Agents SDK for Python, adds a benchmarking improvement, and enhances developer tooling. The main highlights are the addition of a comprehensive agent testing package, improvements to benchmarking output and configuration, and cleanup of developer documentation.

Microsoft 365 Agents SDK Testing Framework

• Added the new microsoft-agents-testing package, providing utilities for agent integration testing, including authentication helpers, environment setup, and agent client classes. This includes modules for agent token generation, activity population, and running aiohttp-based agent applications. (dev/microsoft-agents-testing/microsoft_agents/testing/__init__.py, dev/microsoft-agents-testing/microsoft_agents/testing/auth/generate_token.py, dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/aiohttp/aiohttp_environment.py, dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/aiohttp/aiohttp_runner.py, dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/application_runner.py, dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/client/agent_client.py) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

• Added an installation step for the testing package to dev/install.sh to facilitate developer setup.

Benchmarking Improvements

• Enhanced the benchmark tool by adding a verbose mode (--verbose) to output detailed results for each execution, and updated the payload to include a sample message text. (dev/benchmark/src/main.py, dev/benchmark/payload.json, dev/benchmark/src/output.py) [1] [2] [3] [4]

Documentation Updates

• Updated and cleaned up documentation for the developer tools and the new testing framework, including removal of outdated information and addition of a README for the testing package. (dev/README.md, dev/microsoft-agents-testing/README.md) [1] [2]

[Copilot]

Pull Request Overview

This PR introduces a new testing framework package (microsoft-agents-testing) for the Microsoft 365 Agents SDK. The framework provides utilities for integration testing of agent applications with support for aiohttp-based hosting environments.

Key changes include:

• New testing framework with Sample, Environment, and Integration base classes for structured integration testing
• Client implementations (AgentClient, ResponseClient) for interacting with agent applications and capturing responses
• Aiohttp-specific implementations for running agent applications in test environments
• Enhanced benchmark tool with verbose output option

Reviewed Changes

Copilot reviewed 39 out of 45 changed files in this pull request and generated 24 comments.

Show a summary per file

File	Description
pyproject.toml	Package configuration and metadata
setup.py	Package dependencies specification
microsoft_agents/testing/*	Core testing framework implementation including SDK config, utilities, auth, and integration components
tests/*	Integration tests and sample implementations
dev/benchmark/src/*	Benchmark tool enhancements with verbose output
dev/README.md	Removed documentation (file deleted)

Comments suppressed due to low confidence (5)

dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/client/agent_client.py:1

• Import of 'default' is not used.

from email.policy import default

dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/aiohttp/aiohttp_environment.py:1

• Import of 'E' is not used.

from tkinter import E

dev/benchmark/src/main.py:1

• Import of 'sys' is not used.

import json, sys

dev/benchmark/src/main.py:2

• Import of 'StringIO' is not used.

from io import StringIO

dev/benchmark/src/main.py:5

• Import of 'contextmanager' is not used.

from contextlib import contextmanager

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull Request Overview

Copilot reviewed 39 out of 45 changed files in this pull request and generated 8 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull Request Overview

Copilot reviewed 39 out of 45 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull Request Overview

Copilot reviewed 39 out of 45 changed files in this pull request and generated 17 comments.

Comments suppressed due to low confidence (2)

dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/client/agent_client.py:137

• Variable timeout is not used.

        timeout = timeout or self._default_timeout

dev/microsoft-agents-testing/microsoft_agents/testing/integration/core/client/agent_client.py:148

• Variable timeout is not used.

        timeout = timeout or self._default_timeout

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.