test(profiling): add missing prof correctness tests for I/O profiling

[realFlowControl]

Warning

this merges into #3648

Description

This PR adds missing I/O correctness tests

Reviewer checklist

• Test coverage seems ok.
• Appropriate labels assigned.

[pr-commenter]

Benchmarks [ profiler ]

Benchmark execution time: 2026-02-18 16:39:44

Comparing candidate commit 5c57467 in PR branch florian/macos-io-profiling-tests with baseline commit 6bac2fd in branch florian/macos-io-profiling.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 30 metrics, 6 unstable metrics.

[datadog-official]

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1062 Tests failed

    ❄️ Known flaky: testCommandWithArgument from cakephp-28-test.DDTrace\Tests\Integrations\CLI\CakePHP\V2_8\CommonScenariosTest (Fix with Cursor)

    ❄️ Known flaky: testCommandWithNoArguments from cakephp-28-test.DDTrace\Tests\Integrations\CLI\CakePHP\V2_8\CommonScenariosTest (Fix with Cursor)

❄️ Known flaky: testCommandWithArgument from cakephp-310-test.DDTrace\Tests\Integrations\CLI\CakePHP\V3_10\CommonScenariosTest (Datadog) (Fix with Cursor)

DDTrace\Tests\Integrations\CLI\CakePHP\V3_10\CommonScenariosTest::testCommandWithArgument
TypeError: Argument 1 passed to DDTrace\Tests\Common\SpanChecker::DDTrace\Tests\Common\{closure}() must be of the type array, string given

tests/Common/SpanChecker.php:599
tests/Common/SpanChecker.php:601
tests/Common/SpanChecker.php:346
tests/Common/SpanAssertionTrait.php:19
tests/Common/IntegrationTestCase.php:197
tests/Integrations/CLI/CakePHP/V3_10/CommonScenariosTest.php:51
tests/Common/RetryTraitVersionGeneric.php:28
...

View all

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 5c57467 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.21%. Comparing base (6bac2fd) to head (5c57467).

Additional details and impacted files

@@                      Coverage Diff                       @@
##           florian/macos-io-profiling    #3652      +/-   ##
==============================================================
+ Coverage                       62.11%   62.21%   +0.10%     
==============================================================
  Files                             141      141              
  Lines                           13387    13387              
  Branches                         1753     1753              
==============================================================
+ Hits                             8315     8329      +14     
+ Misses                           4273     4260      -13     
+ Partials                          799      798       -1     

see 3 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6bac2fd...5c57467. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[morrisonlevi]

An error_margin of 10 is pretty high for this deterministic benchmark, isn't it? Any particular reason it isn't tighter?

[realFlowControl]

I checked the runs and they all where at "0% error", but if we start adjusting the sampling distance, we might get some errors in the future.
I've tightened the error_margin to 2%. In case we see failures here when adjusting the sampling distance in the future, we can then make a decision of how to handle that.