FEAT: Refine Perf Benchmarking, test across different OSs and add it to CI

[bewithgaurav]

Work Item / Issue Reference

AB#40134

---

Summary

This pull request introduces a comprehensive performance benchmarking workflow for comparing mssql-python and pyodbc on real-world SQL Server workloads. The changes include a new benchmark script targeting the AdventureWorks2022 database, enhancements to the documentation, and updates to CI pipelines to automate database setup and benchmark execution on both Windows and Ubuntu environments.

Benchmarking infrastructure and documentation:

• Added a new script benchmarks/perf-benchmarking.py for real-world query performance comparisons between mssql-python and pyodbc, including statistical analysis, speedup calculations, and detailed reporting.
• Expanded the benchmarks/README.md with instructions and key features for running both the new and existing benchmark scripts, clarifying usage, output, and requirements.

CI/CD pipeline automation:

• Updated eng/pipelines/pr-validation-pipeline.yml to automatically download and restore the AdventureWorks2022 database for both Windows and Ubuntu environments, ensuring benchmarks run against a realistic dataset. [1] [2]
• Integrated steps in the pipeline to install required dependencies and execute the new benchmarking script, with conditional logic for platform and SQL Server availability. [1] [2]

These changes provide a robust framework for evaluating and monitoring database driver performance in CI, helping guide future optimizations and ensuring reliability across platforms.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 77%

📈 Total Lines Covered: 4569 out of 5899 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.ddbc_bindings.cpp: 70.7%
mssql_python.pybind.connection.connection_pool.cpp: 78.9%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection.cpp: 81.2%
mssql_python.connection.py: 82.9%
mssql_python.cursor.py: 83.4%
mssql_python.auth.py: 87.1%
mssql_python.pooling.py: 87.7%
mssql_python.helpers.py: 88.9%
mssql_python.row.py: 90.9%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull Request Overview

This PR adds comprehensive performance benchmarking capabilities comparing mssql-python against pyodbc using real-world queries on the AdventureWorks2022 database. The benchmarking is integrated into the CI/CD pipeline to run automatically on both Windows (SQL Server 2022) and Ubuntu (containerized SQL Server 2022).

Key Changes:

• New perf-benchmarking.py script with statistical analysis of query performance
• Pipeline integration to download, restore, and benchmark against AdventureWorks2022
• ODBC Driver 18 installation automation for both Windows and Linux environments

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File	Description
eng/pipelines/pr-validation-pipeline.yml	Adds three new pipeline steps: AdventureWorks2022 database restore (Windows), ODBC Driver 18 installation and benchmark execution (Windows), and Ubuntu container-based benchmarking with database restore and ODBC setup
benchmarks/perf-benchmarking.py	New standalone benchmarking script with statistical analysis comparing mssql-python vs pyodbc across complex queries, large datasets, and CTEs
benchmarks/README.md	Updated documentation describing both benchmark scripts, their usage, features, and requirements

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.