Skip to content

Automatic triggering of CiVis test environment #9734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
daniel-mohedano merged 2 commits into from
Oct 20, 2025
Merged

Automatic triggering of CiVis test environment #9734

daniel-mohedano merged 2 commits into from
Oct 20, 2025

Conversation

@daniel-mohedano
Copy link
Contributor

@daniel-mohedano daniel-mohedano commented Oct 13, 2025 •
edited
Loading

What Does This Do

  • Trigger for Test Environment is now automatic instead of manual:
    • After publish-artifacts-to-s3 finishes, the new check-ci-visibility-label job queries GH API to read the PR labels.
    • If the PR contains the comp: ci visibility tag, the Test Environment job is triggered.
    • Test environment are now also responsible of posting PR comments with the performance overhead per project and tested scenario.

Motivation

We want to improve the Test Optimization developer workflow and more actively monitor performance regressions on our side.

Additional Notes

Current limitations:

  • The run-ci-visibility-test-environment job doesn't appear in the PR job list, likely because it is a trigger job. If it fails it will still fail the default-pipeline.

Contributor Checklist

Jira ticket: SDTEST-2768

@daniel-mohedano daniel-mohedano added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility comp: tooling Build & Tooling labels Oct 13, 2025
@datadog-official
Copy link

datadog-official bot commented Oct 13, 2025 •
edited by datadog-datadog-prod-us1 bot
Loading

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 59.88% (-0.02%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 529c394 | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Oct 13, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/test-environment-trigger
git_commit_date 1760520675 1760517543
git_commit_sha ff79486 529c394
release_version 1.55.0-SNAPSHOT~ff79486cac 1.55.0-SNAPSHOT~529c394bfd
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1760529724 1760529724
ci_job_id 1180074969 1180074969
ci_pipeline_id 79357466 79357466
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-qzvi8hzk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-qzvi8hzk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.02 s) : 0, 1019508
Total [baseline] (8.698 s) : 0, 8698359
Agent [candidate] (1.022 s) : 0, 1022372
Total [candidate] (8.674 s) : 0, 8674269
section iast
Agent [baseline] (1.153 s) : 0, 1153036
Total [baseline] (9.338 s) : 0, 9337956
Agent [candidate] (1.153 s) : 0, 1153090
Total [candidate] (9.287 s) : 0, 9286946
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.02 s -
Agent iast 1.153 s 133.528 ms (13.1%)
Total tracing 8.698 s -
Total iast 9.338 s 639.596 ms (7.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.022 s -
Agent iast 1.153 s 130.718 ms (12.8%)
Total tracing 8.674 s -
Total iast 9.287 s 612.678 ms (7.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.473 ms) : 0, 1473
crashtracking [candidate] (1.463 ms) : 0, 1463
BytebuddyAgent [baseline] (694.196 ms) : 0, 694196
BytebuddyAgent [candidate] (696.832 ms) : 0, 696832
GlobalTracer [baseline] (242.034 ms) : 0, 242034
GlobalTracer [candidate] (242.806 ms) : 0, 242806
AppSec [baseline] (32.394 ms) : 0, 32394
AppSec [candidate] (32.569 ms) : 0, 32569
Debugger [baseline] (6.534 ms) : 0, 6534
Debugger [candidate] (6.44 ms) : 0, 6440
Remote Config [baseline] (734.062 µs) : 0, 734
Remote Config [candidate] (702.307 µs) : 0, 702
Telemetry [baseline] (9.294 ms) : 0, 9294
Telemetry [candidate] (9.33 ms) : 0, 9330
Flare Poller [baseline] (11.671 ms) : 0, 11671
Flare Poller [candidate] (10.912 ms) : 0, 10912
section iast
crashtracking [baseline] (1.501 ms) : 0, 1501
crashtracking [candidate] (1.491 ms) : 0, 1491
BytebuddyAgent [baseline] (816.206 ms) : 0, 816206
BytebuddyAgent [candidate] (817.756 ms) : 0, 817756
GlobalTracer [baseline] (231.67 ms) : 0, 231670
GlobalTracer [candidate] (231.279 ms) : 0, 231279
AppSec [baseline] (35.298 ms) : 0, 35298
AppSec [candidate] (34.925 ms) : 0, 34925
Debugger [baseline] (6.214 ms) : 0, 6214
Debugger [candidate] (6.179 ms) : 0, 6179
Remote Config [baseline] (655.617 µs) : 0, 656
Remote Config [candidate] (608.003 µs) : 0, 608
Telemetry [baseline] (8.806 ms) : 0, 8806
Telemetry [candidate] (8.595 ms) : 0, 8595
Flare Poller [baseline] (4.245 ms) : 0, 4245
Flare Poller [candidate] (4.242 ms) : 0, 4242
IAST [baseline] (26.852 ms) : 0, 26852
IAST [candidate] (26.414 ms) : 0, 26414
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.02 s) : 0, 1020006
Total [baseline] (10.687 s) : 0, 10687178
Agent [candidate] (1.017 s) : 0, 1017221
Total [candidate] (10.742 s) : 0, 10742289
section appsec
Agent [baseline] (1.198 s) : 0, 1198309
Total [baseline] (11.031 s) : 0, 11031392
Agent [candidate] (1.195 s) : 0, 1194740
Total [candidate] (11.032 s) : 0, 11031831
section iast
Agent [baseline] (1.15 s) : 0, 1150053
Total [baseline] (11.031 s) : 0, 11031357
Agent [candidate] (1.161 s) : 0, 1161133
Total [candidate] (11.033 s) : 0, 11032563
section profiling
Agent [baseline] (1.167 s) : 0, 1167367
Total [baseline] (11.051 s) : 0, 11050834
Agent [candidate] (1.162 s) : 0, 1161585
Total [candidate] (11.072 s) : 0, 11071677
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.02 s -
Agent appsec 1.198 s 178.303 ms (17.5%)
Agent iast 1.15 s 130.047 ms (12.7%)
Agent profiling 1.167 s 147.362 ms (14.4%)
Total tracing 10.687 s -
Total appsec 11.031 s 344.214 ms (3.2%)
Total iast 11.031 s 344.179 ms (3.2%)
Total profiling 11.051 s 363.656 ms (3.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.017 s -
Agent appsec 1.195 s 177.518 ms (17.5%)
Agent iast 1.161 s 143.912 ms (14.1%)
Agent profiling 1.162 s 144.363 ms (14.2%)
Total tracing 10.742 s -
Total appsec 11.032 s 289.542 ms (2.7%)
Total iast 11.033 s 290.274 ms (2.7%)
Total profiling 11.072 s 329.389 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (693.513 ms) : 0, 693513
BytebuddyAgent [candidate] (693.213 ms) : 0, 693213
GlobalTracer [baseline] (243.238 ms) : 0, 243238
GlobalTracer [candidate] (241.624 ms) : 0, 241624
AppSec [baseline] (32.233 ms) : 0, 32233
AppSec [candidate] (32.292 ms) : 0, 32292
Debugger [baseline] (6.422 ms) : 0, 6422
Debugger [candidate] (6.415 ms) : 0, 6415
Remote Config [baseline] (707.861 µs) : 0, 708
Remote Config [candidate] (708.746 µs) : 0, 709
Telemetry [baseline] (9.344 ms) : 0, 9344
Telemetry [candidate] (9.387 ms) : 0, 9387
Flare Poller [baseline] (11.948 ms) : 0, 11948
Flare Poller [candidate] (10.924 ms) : 0, 10924
section appsec
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (720.008 ms) : 0, 720008
BytebuddyAgent [candidate] (717.673 ms) : 0, 717673
GlobalTracer [baseline] (235.749 ms) : 0, 235749
GlobalTracer [candidate] (234.497 ms) : 0, 234497
AppSec [baseline] (175.737 ms) : 0, 175737
AppSec [candidate] (176.071 ms) : 0, 176071
Debugger [baseline] (6.194 ms) : 0, 6194
Debugger [candidate] (6.175 ms) : 0, 6175
Remote Config [baseline] (631.566 µs) : 0, 632
Remote Config [candidate] (628.118 µs) : 0, 628
Telemetry [baseline] (8.525 ms) : 0, 8525
Telemetry [candidate] (8.414 ms) : 0, 8414
Flare Poller [baseline] (3.928 ms) : 0, 3928
Flare Poller [candidate] (3.886 ms) : 0, 3886
IAST [baseline] (24.933 ms) : 0, 24933
IAST [candidate] (24.849 ms) : 0, 24849
section iast
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (814.255 ms) : 0, 814255
BytebuddyAgent [candidate] (823.213 ms) : 0, 823213
GlobalTracer [baseline] (231.345 ms) : 0, 231345
GlobalTracer [candidate] (232.736 ms) : 0, 232736
AppSec [baseline] (35.339 ms) : 0, 35339
AppSec [candidate] (35.379 ms) : 0, 35379
Debugger [baseline] (6.134 ms) : 0, 6134
Debugger [candidate] (6.229 ms) : 0, 6229
Remote Config [baseline] (608.737 µs) : 0, 609
Remote Config [candidate] (632.846 µs) : 0, 633
Telemetry [baseline] (8.721 ms) : 0, 8721
Telemetry [candidate] (8.667 ms) : 0, 8667
Flare Poller [baseline] (4.182 ms) : 0, 4182
Flare Poller [candidate] (4.324 ms) : 0, 4324
IAST [baseline] (26.52 ms) : 0, 26520
IAST [candidate] (26.751 ms) : 0, 26751
section profiling
ProfilingAgent [baseline] (108.92 ms) : 0, 108920
ProfilingAgent [candidate] (108.441 ms) : 0, 108441
crashtracking [baseline] (1.452 ms) : 0, 1452
crashtracking [candidate] (1.433 ms) : 0, 1433
BytebuddyAgent [baseline] (724.779 ms) : 0, 724779
BytebuddyAgent [candidate] (720.67 ms) : 0, 720670
GlobalTracer [baseline] (218.559 ms) : 0, 218559
GlobalTracer [candidate] (217.786 ms) : 0, 217786
AppSec [baseline] (32.731 ms) : 0, 32731
AppSec [candidate] (32.506 ms) : 0, 32506
Debugger [baseline] (7.37 ms) : 0, 7370
Debugger [candidate] (6.456 ms) : 0, 6456
Remote Config [baseline] (792.613 µs) : 0, 793
Remote Config [candidate] (822.635 µs) : 0, 823
Telemetry [baseline] (15.202 ms) : 0, 15202
Telemetry [candidate] (16.147 ms) : 0, 16147
Flare Poller [baseline] (4.159 ms) : 0, 4159
Flare Poller [candidate] (4.208 ms) : 0, 4208
Profiling [baseline] (110.072 ms) : 0, 110072
Profiling [candidate] (109.582 ms) : 0, 109582
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/test-environment-trigger
git_commit_date 1760520675 1760517543
git_commit_sha ff79486 529c394
release_version 1.55.0-SNAPSHOT~ff79486cac 1.55.0-SNAPSHOT~529c394bfd
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1760529449 1760529449
ci_job_id 1180074970 1180074970
ci_pipeline_id 79357466 79357466
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-q6em4tzf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-q6em4tzf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 3 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load worse
[+140.433µs; +246.011µs] or [+3.285%; +5.754%]		 unstable
[-161.648op/s; +70.898op/s] or [-15.064%; +6.607%]		 4.468ms 1027.688op/s 4.275ms 1073.062op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-947.230µs; -538.138µs] or [-8.554%; -4.860%]		 unstable
[-21.841op/s; +81.841op/s] or [-5.200%; +19.486%]		 10.331ms 450.000op/s 11.074ms 420.000op/s
scenario:load:petclinic:iast:high_load better
[-2.977ms; -2.135ms] or [-6.547%; -4.697%]		 unstable
[-1.342op/s; +13.567op/s] or [-1.303%; +13.179%]		 42.910ms 109.050op/s 45.466ms 102.938op/s
scenario:load:petclinic:profiling:high_load worse
[+1.775ms; +2.723ms] or [+3.741%; +5.739%]		 unstable
[-11.110op/s; +2.160op/s] or [-11.259%; +2.189%]		 49.701ms 94.200op/s 47.452ms 98.675op/s
scenario:load:petclinic:tracing:high_load worse
[+2.096ms; +2.914ms] or [+4.863%; +6.761%]		 unstable
[-13.276op/s; +1.326op/s] or [-12.227%; +1.221%]		 45.602ms 102.600op/s 43.097ms 108.575op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.275 ms) : 4227, 4323
.   : milestone, 4275,
iast (9.569 ms) : 9412, 9726
.   : milestone, 9569,
iast_FULL (14.175 ms) : 13896, 14454
.   : milestone, 14175,
iast_GLOBAL (11.074 ms) : 10875, 11273
.   : milestone, 11074,
profiling (9.05 ms) : 8899, 9201
.   : milestone, 9050,
tracing (7.81 ms) : 7697, 7923
.   : milestone, 7810,
section candidate
no_agent (4.468 ms) : 4418, 4519
.   : milestone, 4468,
iast (9.38 ms) : 9226, 9533
.   : milestone, 9380,
iast_FULL (14.166 ms) : 13887, 14446
.   : milestone, 14166,
iast_GLOBAL (10.331 ms) : 10151, 10512
.   : milestone, 10331,
profiling (8.978 ms) : 8838, 9118
.   : milestone, 8978,
tracing (7.846 ms) : 7734, 7958
.   : milestone, 7846,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.275 ms [4.227 ms, 4.323 ms] -
iast 9.569 ms [9.412 ms, 9.726 ms] 5.294 ms (123.8%)
iast_FULL 14.175 ms [13.896 ms, 14.454 ms] 9.9 ms (231.6%)
iast_GLOBAL 11.074 ms [10.875 ms, 11.273 ms] 6.799 ms (159.0%)
profiling 9.05 ms [8.899 ms, 9.201 ms] 4.775 ms (111.7%)
tracing 7.81 ms [7.697 ms, 7.923 ms] 3.535 ms (82.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.468 ms [4.418 ms, 4.519 ms] -
iast 9.38 ms [9.226 ms, 9.533 ms] 4.911 ms (109.9%)
iast_FULL 14.166 ms [13.887 ms, 14.446 ms] 9.698 ms (217.0%)
iast_GLOBAL 10.331 ms [10.151 ms, 10.512 ms] 5.863 ms (131.2%)
profiling 8.978 ms [8.838 ms, 9.118 ms] 4.51 ms (100.9%)
tracing 7.846 ms [7.734 ms, 7.958 ms] 3.378 ms (75.6%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.183 ms) : 35879, 36486
.   : milestone, 36183,
appsec (49.078 ms) : 48632, 49524
.   : milestone, 49078,
code_origins (43.671 ms) : 43295, 44047
.   : milestone, 43671,
iast (45.466 ms) : 45068, 45864
.   : milestone, 45466,
profiling (47.452 ms) : 47025, 47878
.   : milestone, 47452,
tracing (43.097 ms) : 42733, 43462
.   : milestone, 43097,
section candidate
no_agent (36.672 ms) : 36382, 36962
.   : milestone, 36672,
appsec (48.823 ms) : 48380, 49266
.   : milestone, 48823,
code_origins (44.313 ms) : 43927, 44698
.   : milestone, 44313,
iast (42.91 ms) : 42527, 43293
.   : milestone, 42910,
profiling (49.701 ms) : 49247, 50155
.   : milestone, 49701,
tracing (45.602 ms) : 45208, 45997
.   : milestone, 45602,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.183 ms [35.879 ms, 36.486 ms] -
appsec 49.078 ms [48.632 ms, 49.524 ms] 12.895 ms (35.6%)
code_origins 43.671 ms [43.295 ms, 44.047 ms] 7.489 ms (20.7%)
iast 45.466 ms [45.068 ms, 45.864 ms] 9.283 ms (25.7%)
profiling 47.452 ms [47.025 ms, 47.878 ms] 11.269 ms (31.1%)
tracing 43.097 ms [42.733 ms, 43.462 ms] 6.915 ms (19.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.672 ms [36.382 ms, 36.962 ms] -
appsec 48.823 ms [48.38 ms, 49.266 ms] 12.151 ms (33.1%)
code_origins 44.313 ms [43.927 ms, 44.698 ms] 7.641 ms (20.8%)
iast 42.91 ms [42.527 ms, 43.293 ms] 6.238 ms (17.0%)
profiling 49.701 ms [49.247 ms, 50.155 ms] 13.029 ms (35.5%)
tracing 45.602 ms [45.208 ms, 45.997 ms] 8.93 ms (24.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/test-environment-trigger
git_commit_date 1760520675 1760517543
git_commit_sha ff79486 529c394
release_version 1.55.0-SNAPSHOT~ff79486cac 1.55.0-SNAPSHOT~529c394bfd
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1760529949 1760529949
ci_job_id 1180074971 1180074971
ci_pipeline_id 79357466 79357466
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-pt84quid 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-pt84quid 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.909 s) : 14909000, 14909000
.   : milestone, 14909000,
appsec (15.035 s) : 15035000, 15035000
.   : milestone, 15035000,
iast (18.28 s) : 18280000, 18280000
.   : milestone, 18280000,
iast_GLOBAL (18.035 s) : 18035000, 18035000
.   : milestone, 18035000,
profiling (15.42 s) : 15420000, 15420000
.   : milestone, 15420000,
tracing (15.104 s) : 15104000, 15104000
.   : milestone, 15104000,
section candidate
no_agent (14.706 s) : 14706000, 14706000
.   : milestone, 14706000,
appsec (14.918 s) : 14918000, 14918000
.   : milestone, 14918000,
iast (18.791 s) : 18791000, 18791000
.   : milestone, 18791000,
iast_GLOBAL (18.204 s) : 18204000, 18204000
.   : milestone, 18204000,
profiling (14.942 s) : 14942000, 14942000
.   : milestone, 14942000,
tracing (14.961 s) : 14961000, 14961000
.   : milestone, 14961000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.909 s [14.909 s, 14.909 s] -
appsec 15.035 s [15.035 s, 15.035 s] 126.0 ms (0.8%)
iast 18.28 s [18.28 s, 18.28 s] 3.371 s (22.6%)
iast_GLOBAL 18.035 s [18.035 s, 18.035 s] 3.126 s (21.0%)
profiling 15.42 s [15.42 s, 15.42 s] 511.0 ms (3.4%)
tracing 15.104 s [15.104 s, 15.104 s] 195.0 ms (1.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.706 s [14.706 s, 14.706 s] -
appsec 14.918 s [14.918 s, 14.918 s] 212.0 ms (1.4%)
iast 18.791 s [18.791 s, 18.791 s] 4.085 s (27.8%)
iast_GLOBAL 18.204 s [18.204 s, 18.204 s] 3.498 s (23.8%)
profiling 14.942 s [14.942 s, 14.942 s] 236.0 ms (1.6%)
tracing 14.961 s [14.961 s, 14.961 s] 255.0 ms (1.7%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~529c394bfd, baseline=1.55.0-SNAPSHOT~ff79486cac
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1470, 1493
.   : milestone, 1482,
appsec (3.75 ms) : 3530, 3970
.   : milestone, 3750,
iast (2.225 ms) : 2161, 2289
.   : milestone, 2225,
iast_GLOBAL (2.278 ms) : 2213, 2343
.   : milestone, 2278,
profiling (2.479 ms) : 2325, 2634
.   : milestone, 2479,
tracing (2.043 ms) : 1992, 2093
.   : milestone, 2043,
section candidate
no_agent (1.48 ms) : 1468, 1491
.   : milestone, 1480,
appsec (3.724 ms) : 3503, 3945
.   : milestone, 3724,
iast (2.237 ms) : 2173, 2302
.   : milestone, 2237,
iast_GLOBAL (2.264 ms) : 2199, 2329
.   : milestone, 2264,
profiling (2.064 ms) : 2011, 2116
.   : milestone, 2064,
tracing (2.042 ms) : 1992, 2093
.   : milestone, 2042,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.482 ms [1.47 ms, 1.493 ms] -
appsec 3.75 ms [3.53 ms, 3.97 ms] 2.268 ms (153.1%)
iast 2.225 ms [2.161 ms, 2.289 ms] 743.551 µs (50.2%)
iast_GLOBAL 2.278 ms [2.213 ms, 2.343 ms] 796.208 µs (53.7%)
profiling 2.479 ms [2.325 ms, 2.634 ms] 997.623 µs (67.3%)
tracing 2.043 ms [1.992 ms, 2.093 ms] 561.023 µs (37.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.468 ms, 1.491 ms] -
appsec 3.724 ms [3.503 ms, 3.945 ms] 2.245 ms (151.7%)
iast 2.237 ms [2.173 ms, 2.302 ms] 757.7 µs (51.2%)
iast_GLOBAL 2.264 ms [2.199 ms, 2.329 ms] 784.535 µs (53.0%)
profiling 2.064 ms [2.011 ms, 2.116 ms] 584.138 µs (39.5%)
tracing 2.042 ms [1.992 ms, 2.093 ms] 562.282 µs (38.0%)

@daniel-mohedano daniel-mohedano added the tag: no release notes Changes to exclude from release notes label Oct 13, 2025
@daniel-mohedano daniel-mohedano marked this pull request as ready for review October 13, 2025 10:29
@daniel-mohedano daniel-mohedano requested a review from a team as a code owner October 13, 2025 10:29
@daniel-mohedano daniel-mohedano requested review from erikayasuda and removed request for a team October 13, 2025 10:29
PerfectSlayer
PerfectSlayer reviewed Oct 13, 2025
View reviewed changes
.gitlab/ci-visibility-tests.yml
echo "Found PR #${PR_NUMBER}"
# Check if PR has the CI visibility label
if pr_has_label "$PR_NUMBER" "comp: ci visibility"; then
Copy link
Contributor

@PerfectSlayer PerfectSlayer Oct 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ question: ‏What about using run-tests: label category?
As I will soon enforce to only have one comp: or inst: label to prevent duplicate changelog, I would recommend using the labels dedicated to CI task run.

Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog Oct 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then we'll have to add run-tests:... to every one of our PRs just for the sake of it. Adding comp:ci-visibility is something we already do and won't forget to do.

I will soon enforce to only have one comp: or inst:

That's a pity, is there any other way we could avoid duplicates? E.g. if there's an instrumentation label, use it to determine the section, and then fallback to the comp one?

Copy link
Contributor Author

@daniel-mohedano daniel-mohedano Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coming back to this, I could modify the trigger to activate when either one of the two labels is set, so that our use case is still supported. Although after checking, in our last over 100 PRs we haven't used an inst: label, so I don't think the conflict will be common at all (in cases where there were multiple comp: labels it was mainly related to either testing or tooling)

@DataDog DataDog deleted a comment from cit-pr-commenter bot Oct 15, 2025
@cit-pr-commenter
Copy link

cit-pr-commenter bot commented Oct 15, 2025

Test Environment - sbt-scalatest

Job Status: success

Scenario Overhead (%)
agent 55.28
agentEvpProxy 55.72

@cit-pr-commenter
Copy link

cit-pr-commenter bot commented Oct 15, 2025

Test Environment - nebula-release-plugin

Job Status: success

Scenario Overhead (%)
agent 37.87
agentless 36.72
agentlessCodeCoverage 42.89
agentlessLineCoverage 73.32

@cit-pr-commenter
Copy link

cit-pr-commenter bot commented Oct 15, 2025

Test Environment - pass4s

Job Status: success

Scenario Overhead (%)
agent 14.70
agentless 8.24
agentlessCodeCoverage 13.14

@cit-pr-commenter
Copy link

cit-pr-commenter bot commented Oct 15, 2025

Test Environment - jolokia

Job Status: success

Scenario Overhead (%)
agent 90.54
agentless 85.18
agentlessCodeCoverage 95.70
agentlessLineCoverage 96.51

@cit-pr-commenter
Copy link

cit-pr-commenter bot commented Oct 15, 2025

Test Environment - okhttp

Job Status: success

Scenario Overhead (%)
agent 15.69
agentless 16.22
agentlessCodeCoverage 19.99
agentlessLineCoverage 35.23

@daniel-mohedano daniel-mohedano merged commit e0bca5e into master Oct 20, 2025
534 checks passed
@daniel-mohedano daniel-mohedano deleted the daniel.mohedano/test-environment-trigger branch October 20, 2025 08:03
@github-actions github-actions bot added this to the 1.55.0 milestone Oct 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer left review comments

@nikita-tkachenko-datadog nikita-tkachenko-datadog nikita-tkachenko-datadog approved these changes

@erikayasuda erikayasuda Awaiting requested review from erikayasuda erikayasuda is a code owner automatically assigned from DataDog/apm-release-platform

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

1.55.0

Development

Successfully merging this pull request may close these issues.

4 participants

@daniel-mohedano @PerfectSlayer @nikita-tkachenko-datadog