Users/robrandao/data driven tests by rodrigobr-msft · Pull Request #245 · microsoft/Agents-for-python

rodrigobr-msft · 2025-11-14T19:40:55Z

No description provided.

Copilot

Pull Request Overview

This PR introduces a comprehensive data-driven testing infrastructure for the Microsoft Agents testing framework. The changes include assertion utilities, selector mechanisms, and a data-driven test runner to facilitate bot testing through configuration files.

Key changes:

Added assertion framework with field-level and activity-level assertion capabilities
Implemented selector mechanism for filtering activities based on criteria
Created data-driven test infrastructure supporting YAML/JSON configuration
Added comprehensive test coverage for all new functionality

Reviewed Changes

Copilot reviewed 50 out of 60 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
pyproject.toml	Fixed package name from "microsoft-agents-hosting-core" to "microsoft-agents-testing"
pytest.ini	Added pytest configuration with warnings, test discovery, and markers
microsoft_agents/testing/assertions/*	New assertion framework with type definitions, field/activity checkers, and selectors
microsoft_agents/testing/utils/populate.py	Utility functions for populating activities and dicts with defaults
microsoft_agents/testing/utils/misc.py	Refactored utility functions (URL parsing, activity normalization)
microsoft_agents/testing/integration/data_driven/*	Data-driven test runner implementation
tests/assertions/*	Comprehensive tests for assertion framework
tests/utils/test_populate.py	Tests for populate utility functions
tests/integration/data_driven/test_data_driven_test.py	Tests for data-driven test functionality

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-14T19:43:25Z

...icrosoft-agents-testing/microsoft_agents/testing/integration/data_driven/data_driven_test.py

@@ -0,0 +1,91 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.s


Corrected spelling: removed trailing 's' after 'License'.

Copilot · 2025-11-14T19:43:26Z

dev/microsoft-agents-testing/microsoft_agents/testing/assertions/type_defs.py

+        :return: The corresponding AssertionQuantifier.
+        """
+        value = value.upper()
+        if value not in AssertionQuantifier:


This test may raise an Exception as the target may be of non-container class type.

Suggested change

if value not in AssertionQuantifier:

if value not in (item.value for item in AssertionQuantifier):

Copilot · 2025-11-14T19:43:26Z

dev/integration/integration/integration/foundational/test_suite.py

@@ -0,0 +1,141 @@
+import json


Syntax Error (in Python 3).

Copilot · 2025-11-14T19:43:26Z

dev/microsoft-agents-testing/microsoft_agents/testing/assertions/assertions.py

+
+from microsoft_agents.activity import Activity
+
+from .type_defs import FieldAssertionType, AssertionQuantifier


Import of 'AssertionQuantifier' is not used.

Suggested change

from .type_defs import FieldAssertionType, AssertionQuantifier

from .type_defs import FieldAssertionType

Copilot · 2025-11-14T19:43:27Z

...icrosoft-agents-testing/microsoft_agents/testing/integration/data_driven/data_driven_test.py

+from microsoft_agents.testing.utils import (
+    populate_activity,
+    update_with_defaults,
+)


Import of 'populate_activity' is not used.

Copilot · 2025-11-14T19:43:27Z

dev/microsoft-agents-testing/microsoft_agents/testing/integration/ddt.py

+import asyncio
+
+from copy import deepcopy
+from typing import Awaitable, Callable


Import of 'Awaitable' is not used.
Import of 'Callable' is not used.

Copilot · 2025-11-14T19:43:27Z

dev/microsoft-agents-testing/tests/integration/data_driven/test_data_driven_test.py

+import os
+from unittest.mock import AsyncMock
+
+from microsoft_agents.activity import Activity, ActivityTypes


Import of 'ActivityTypes' is not used.

Copilot · 2025-11-14T19:43:27Z

dev/microsoft-agents-testing/tests/utils/test_populate.py

+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+import pytest


Import of 'pytest' is not used.

rodrigobr-msft added 30 commits October 21, 2025 14:16

basic integration utility

cfb8847

Integration test suite factory implementation

4d93a46

Implementing core models for integration testing

be2fb1c

AutoClient mockup

0d5539a

Adding runner starter code

bf4c006

Adding foundational classes

ebe622d

Drafting AgentClient and ResponseClient implementations

45678e9

Spec test

c2b84cb

Filling in more implementation details

2816e68

More files

2214827

Cleaning up implementations

c90c8e4

Adding expect replies sending method

36f5c3b

Beginning unit tests

dfdd959

Adding integration decor from sample test cases

d4828fb

Integration from service url tests

91b3c44

_handle_conversation implementation for response_client

1eefe44

AgentClient tests completed

a06da9a

Creating response client tests

d36ec7a

Hosting server for response client

c096048

Response client tests completed

5451ddf

Beginning refactor of aiohttp runner

c2cdd40

Unit test updates

7d91ef9

Fixing issues in refactor

87610a5

Fixed TestIntegrationFromSample unit test

ddfdd0b

Another commit

dd0a87d

Reorganizing files

6a4e8ef

Completed TestIntegrationFromServiceUrl unit tests

7793790

Reformatting with black

3ebbd44

Quickstart integration test beginning

a6681bf

Draft of quickstart sample setup

1a7264b

rodrigobr-msft added 25 commits November 6, 2025 15:22

Adjusting return values for check_activity with refactor

42b4ea1

Fixing package name in pyproject.toml

2f79824

Assertion test cases

cfded79

_parse_assertion unit tests

b6618b2

More assertion tests and fixes to data normalization logic

1e8f19e

Fixing more test cases with newest refactor

d9eb5fa

Fixing merge conflicts

f02b875

assert_activity tests

57b87eb

Reorganizing tests

6925093

Copyright comments

6d55b29

DDT implementation outline

6866f5d

DataDrivenTest class

1db2bf5

select_activity implementation for assertions

e44a2da

inverted assertions for quantifiers

1cf561b

ActivityAssertion class implementation

e651f92

DataDrivenTest implementation

ed7f2f1

Populate function for dictionaries

e88997e

Populate tests

0a48730

Selector class implementation

2f96d88

Assertion tests completed for now

3a04fb5

Adding ActivityAssertion tests

31b2b99

DataDrivenTest implementation connected with default config

4df4788

More test cases and formatting

0c195b1

Data driven testing single test setup support

7a79cc9

Formatting

f8c9166

Copilot AI review requested due to automatic review settings November 14, 2025 19:40

rodrigobr-msft closed this Nov 14, 2025

Copilot started reviewing on behalf of rodrigobr-msft November 14, 2025 19:41 View session

Copilot finished reviewing on behalf of rodrigobr-msft November 14, 2025 19:42

Copilot AI reviewed Nov 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Users/robrandao/data driven tests#245

Users/robrandao/data driven tests#245
rodrigobr-msft wants to merge 84 commits intomainfrom
users/robrandao/data-driven-tests

rodrigobr-msft commented Nov 14, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Copilot AI Nov 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

		@@ -0,0 +1,91 @@
		# Copyright (c) Microsoft Corporation. All rights reserved.
		# Licensed under the MIT License.s

	if value not in AssertionQuantifier:
	if value not in (item.value for item in AssertionQuantifier):


		from microsoft_agents.activity import Activity

		from .type_defs import FieldAssertionType, AssertionQuantifier

Conversation

rodrigobr-msft commented Nov 14, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments