Cover muzzle plugin by tests#10643
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 22 commits intomasterfrom Feb 23, 2026
Merged
Cover muzzle plugin by tests#10643gh-worker-dd-mergequeue-cf854d[bot] merged 22 commits intomasterfrom
gh-worker-dd-mergequeue-cf854d[bot] merged 22 commits intomasterfrom
Conversation
bric3
added
tag: no release notes
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1065059
Total [baseline] (8.735 s) : 0, 8734518
Agent [candidate] (1.064 s) : 0, 1064407
Total [candidate] (8.75 s) : 0, 8749753
section iast
Agent [baseline] (1.242 s) : 0, 1242173
Total [baseline] (9.424 s) : 0, 9423938
Agent [candidate] (1.236 s) : 0, 1236149
Total [candidate] (9.411 s) : 0, 9411260
gantt
title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.21 ms) : 0, 1210
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (628.439 ms) : 0, 628439
BytebuddyAgent [candidate] (627.788 ms) : 0, 627788
AgentMeter [baseline] (29.037 ms) : 0, 29037
AgentMeter [candidate] (29.071 ms) : 0, 29071
GlobalTracer [baseline] (257.564 ms) : 0, 257564
GlobalTracer [candidate] (258.237 ms) : 0, 258237
AppSec [baseline] (33.042 ms) : 0, 33042
AppSec [candidate] (33.137 ms) : 0, 33137
Debugger [baseline] (62.985 ms) : 0, 62985
Debugger [candidate] (64.424 ms) : 0, 64424
Remote Config [baseline] (643.929 µs) : 0, 644
Remote Config [candidate] (628.09 µs) : 0, 628
Telemetry [baseline] (10.677 ms) : 0, 10677
Telemetry [candidate] (9.849 ms) : 0, 9849
Flare Poller [baseline] (5.232 ms) : 0, 5232
Flare Poller [candidate] (3.832 ms) : 0, 3832
section iast
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (804.058 ms) : 0, 804058
BytebuddyAgent [candidate] (798.443 ms) : 0, 798443
AgentMeter [baseline] (11.59 ms) : 0, 11590
AgentMeter [candidate] (11.4 ms) : 0, 11400
GlobalTracer [baseline] (249.598 ms) : 0, 249598
GlobalTracer [candidate] (249.014 ms) : 0, 249014
IAST [baseline] (27.46 ms) : 0, 27460
IAST [candidate] (27.33 ms) : 0, 27330
AppSec [baseline] (35.073 ms) : 0, 35073
AppSec [candidate] (33.339 ms) : 0, 33339
Debugger [baseline] (64.389 ms) : 0, 64389
Debugger [candidate] (66.623 ms) : 0, 66623
Remote Config [baseline] (532.957 µs) : 0, 533
Remote Config [candidate] (535.762 µs) : 0, 536
Telemetry [baseline] (8.571 ms) : 0, 8571
Telemetry [candidate] (8.603 ms) : 0, 8603
Flare Poller [baseline] (3.428 ms) : 0, 3428
Flare Poller [candidate] (3.442 ms) : 0, 3442
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064703
Total [baseline] (10.861 s) : 0, 10860530
Agent [candidate] (1.067 s) : 0, 1067427
Total [candidate] (10.925 s) : 0, 10925249
section appsec
Agent [baseline] (1.241 s) : 0, 1241349
Total [baseline] (11.083 s) : 0, 11082672
Agent [candidate] (1.241 s) : 0, 1240927
Total [candidate] (11.03 s) : 0, 11030082
section iast
Agent [baseline] (1.233 s) : 0, 1232571
Total [baseline] (11.248 s) : 0, 11248247
Agent [candidate] (1.231 s) : 0, 1231476
Total [candidate] (11.218 s) : 0, 11217805
section profiling
Agent [baseline] (1.191 s) : 0, 1190784
Total [baseline] (10.963 s) : 0, 10962909
Agent [candidate] (1.199 s) : 0, 1198690
Total [candidate] (10.943 s) : 0, 10943055
gantt
title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (628.148 ms) : 0, 628148
BytebuddyAgent [candidate] (629.058 ms) : 0, 629058
AgentMeter [baseline] (29.118 ms) : 0, 29118
AgentMeter [candidate] (29.131 ms) : 0, 29131
GlobalTracer [baseline] (258.069 ms) : 0, 258069
GlobalTracer [candidate] (258.415 ms) : 0, 258415
AppSec [baseline] (33.017 ms) : 0, 33017
AppSec [candidate] (33.018 ms) : 0, 33018
Debugger [baseline] (64.646 ms) : 0, 64646
Debugger [candidate] (65.163 ms) : 0, 65163
Remote Config [baseline] (707.828 µs) : 0, 708
Remote Config [candidate] (613.981 µs) : 0, 614
Telemetry [baseline] (9.215 ms) : 0, 9215
Telemetry [candidate] (9.922 ms) : 0, 9922
Flare Poller [baseline] (4.471 ms) : 0, 4471
Flare Poller [candidate] (4.648 ms) : 0, 4648
section appsec
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (659.037 ms) : 0, 659037
BytebuddyAgent [candidate] (658.451 ms) : 0, 658451
AgentMeter [baseline] (11.99 ms) : 0, 11990
AgentMeter [candidate] (11.985 ms) : 0, 11985
GlobalTracer [baseline] (258.637 ms) : 0, 258637
GlobalTracer [candidate] (259.053 ms) : 0, 259053
IAST [baseline] (25.374 ms) : 0, 25374
IAST [candidate] (25.522 ms) : 0, 25522
AppSec [baseline] (167.877 ms) : 0, 167877
AppSec [candidate] (168.942 ms) : 0, 168942
Debugger [baseline] (67.066 ms) : 0, 67066
Debugger [candidate] (66.19 ms) : 0, 66190
Remote Config [baseline] (671.604 µs) : 0, 672
Remote Config [candidate] (651.23 µs) : 0, 651
Telemetry [baseline] (9.581 ms) : 0, 9581
Telemetry [candidate] (9.269 ms) : 0, 9269
Flare Poller [baseline] (3.691 ms) : 0, 3691
Flare Poller [candidate] (3.537 ms) : 0, 3537
section iast
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (796.327 ms) : 0, 796327
BytebuddyAgent [candidate] (796.033 ms) : 0, 796033
AgentMeter [baseline] (11.294 ms) : 0, 11294
AgentMeter [candidate] (11.336 ms) : 0, 11336
GlobalTracer [baseline] (247.14 ms) : 0, 247140
GlobalTracer [candidate] (247.155 ms) : 0, 247155
IAST [baseline] (26.996 ms) : 0, 26996
IAST [candidate] (26.931 ms) : 0, 26931
AppSec [baseline] (34.15 ms) : 0, 34150
AppSec [candidate] (31.152 ms) : 0, 31152
Debugger [baseline] (66.533 ms) : 0, 66533
Debugger [candidate] (68.858 ms) : 0, 68858
Remote Config [baseline] (549.053 µs) : 0, 549
Remote Config [candidate] (539.109 µs) : 0, 539
Telemetry [baseline] (8.761 ms) : 0, 8761
Telemetry [candidate] (8.629 ms) : 0, 8629
Flare Poller [baseline] (3.512 ms) : 0, 3512
Flare Poller [candidate] (3.438 ms) : 0, 3438
section profiling
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.21 ms) : 0, 1210
BytebuddyAgent [baseline] (681.71 ms) : 0, 681710
BytebuddyAgent [candidate] (686.246 ms) : 0, 686246
AgentMeter [baseline] (8.517 ms) : 0, 8517
AgentMeter [candidate] (8.614 ms) : 0, 8614
GlobalTracer [baseline] (215.964 ms) : 0, 215964
GlobalTracer [candidate] (217.443 ms) : 0, 217443
AppSec [baseline] (32.481 ms) : 0, 32481
AppSec [candidate] (33.013 ms) : 0, 33013
Debugger [baseline] (67.71 ms) : 0, 67710
Debugger [candidate] (68.222 ms) : 0, 68222
Remote Config [baseline] (638.189 µs) : 0, 638
Remote Config [candidate] (627.543 µs) : 0, 628
Telemetry [baseline] (9.08 ms) : 0, 9080
Telemetry [candidate] (8.98 ms) : 0, 8980
Flare Poller [baseline] (3.671 ms) : 0, 3671
Flare Poller [candidate] (3.768 ms) : 0, 3768
ProfilingAgent [baseline] (99.102 ms) : 0, 99102
ProfilingAgent [candidate] (99.514 ms) : 0, 99514
Profiling [baseline] (99.675 ms) : 0, 99675
Profiling [candidate] (100.088 ms) : 0, 100088
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 2 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section baseline
no_agent (1.248 ms) : 1236, 1260
. : milestone, 1248,
iast (3.048 ms) : 3012, 3084
. : milestone, 3048,
iast_FULL (5.818 ms) : 5761, 5875
. : milestone, 5818,
iast_GLOBAL (3.313 ms) : 3264, 3363
. : milestone, 3313,
profiling (2.015 ms) : 1993, 2036
. : milestone, 2015,
tracing (1.772 ms) : 1757, 1786
. : milestone, 1772,
section candidate
no_agent (1.18 ms) : 1168, 1191
. : milestone, 1180,
iast (3.166 ms) : 3121, 3210
. : milestone, 3166,
iast_FULL (5.807 ms) : 5749, 5866
. : milestone, 5807,
iast_GLOBAL (3.517 ms) : 3463, 3571
. : milestone, 3517,
profiling (2.018 ms) : 2000, 2036
. : milestone, 2018,
tracing (1.783 ms) : 1768, 1798
. : milestone, 1783,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section baseline
no_agent (17.144 ms) : 16973, 17315
. : milestone, 17144,
appsec (18.623 ms) : 18434, 18811
. : milestone, 18623,
code_origins (18.124 ms) : 17939, 18310
. : milestone, 18124,
iast (17.833 ms) : 17654, 18012
. : milestone, 17833,
profiling (18.684 ms) : 18499, 18868
. : milestone, 18684,
tracing (18.683 ms) : 18494, 18872
. : milestone, 18683,
section candidate
no_agent (17.886 ms) : 17701, 18071
. : milestone, 17886,
appsec (18.612 ms) : 18426, 18798
. : milestone, 18612,
code_origins (18.016 ms) : 17836, 18197
. : milestone, 18016,
iast (17.584 ms) : 17410, 17758
. : milestone, 17584,
profiling (19.075 ms) : 18887, 19263
. : milestone, 19075,
tracing (17.474 ms) : 17298, 17651
. : milestone, 17474,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section baseline
no_agent (15.562 s) : 15562000, 15562000
. : milestone, 15562000,
appsec (14.803 s) : 14803000, 14803000
. : milestone, 14803000,
iast (18.13 s) : 18130000, 18130000
. : milestone, 18130000,
iast_GLOBAL (18.118 s) : 18118000, 18118000
. : milestone, 18118000,
profiling (15.078 s) : 15078000, 15078000
. : milestone, 15078000,
tracing (14.52 s) : 14520000, 14520000
. : milestone, 14520000,
section candidate
no_agent (14.94 s) : 14940000, 14940000
. : milestone, 14940000,
appsec (14.986 s) : 14986000, 14986000
. : milestone, 14986000,
iast (18.049 s) : 18049000, 18049000
. : milestone, 18049000,
iast_GLOBAL (17.742 s) : 17742000, 17742000
. : milestone, 17742000,
profiling (15.465 s) : 15465000, 15465000
. : milestone, 15465000,
tracing (14.9 s) : 14900000, 14900000
. : milestone, 14900000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~4e9a5952a7, baseline=1.60.0-SNAPSHOT~af8b84438c
dateFormat X
axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
. : milestone, 1473,
appsec (2.511 ms) : 2457, 2566
. : milestone, 2511,
iast (2.25 ms) : 2182, 2319
. : milestone, 2250,
iast_GLOBAL (2.292 ms) : 2223, 2361
. : milestone, 2292,
profiling (2.084 ms) : 2029, 2138
. : milestone, 2084,
tracing (2.071 ms) : 2018, 2125
. : milestone, 2071,
section candidate
no_agent (1.474 ms) : 1462, 1485
. : milestone, 1474,
appsec (3.793 ms) : 3573, 4014
. : milestone, 3793,
iast (2.247 ms) : 2178, 2316
. : milestone, 2247,
iast_GLOBAL (2.289 ms) : 2220, 2358
. : milestone, 2289,
profiling (2.074 ms) : 2019, 2128
. : milestone, 2074,
tracing (2.055 ms) : 2002, 2108
. : milestone, 2055,
|
bric3
commented
Feb 20, 2026
Comment on lines
-125
to
+128
| project.extensions.getByType<MuzzleExtension>().directives.forEach { directive -> | ||
| project.logger.debug("configuring {}", directive) | ||
|
|
||
| if (directive.isCoreJdk) { | ||
| runAfter = addMuzzleTask(directive, null, project, runAfter, muzzleBootstrap, muzzleTooling) | ||
| muzzleReportTasks.add(runAfter) | ||
| } else { | ||
| val range = resolveVersionRange(directive, system, session) | ||
|
|
||
| muzzleDirectiveToArtifacts(directive, range).forEach { | ||
| runAfter = addMuzzleTask(directive, it, project, runAfter, muzzleBootstrap, muzzleTooling) | ||
| muzzleReportTasks.add(runAfter) | ||
| } | ||
|
|
||
| if (directive.assertInverse) { | ||
| inverseOf(directive, system, session).forEach { inverseDirective -> | ||
| val inverseRange = resolveVersionRange(inverseDirective, system, session) | ||
|
|
||
| muzzleDirectiveToArtifacts(inverseDirective, inverseRange).forEach { | ||
| runAfter = addMuzzleTask(inverseDirective, it, project, runAfter, muzzleBootstrap, muzzleTooling) | ||
| muzzleReportTasks.add(runAfter) | ||
| } | ||
| } | ||
| } | ||
| } | ||
| project.logger.info("configured $directive") | ||
| val directives = project.extensions.getByType<MuzzleExtension>().directives | ||
| taskPlanner.plan(directives).forEach { plan -> | ||
| runAfter = registerMuzzleTask(plan.directive, plan.artifact, project, runAfter, muzzleBootstrap, muzzleTooling) | ||
| muzzleReportTasks.add(runAfter) | ||
| project.logger.info("configured ${plan.directive}") |
Contributor
Author
There was a problem hiding this comment.
note: This is extracted in the new MuzzleTaskPlanner for easier testing. Leaving only the task registration in the MuzzlePlugin
Comment on lines
+77
to
-83
| // Version-specific task: created by MuzzlePlugin for each resolved artifact. | ||
| muzzleDirective.isPresent -> { | ||
| assertMuzzle(muzzleDirective.get()) | ||
| } | ||
| // Fallback for the root "muzzle" lifecycle task when no pass{} directives are | ||
| // declared. In that case there are no version-specific pass tasks, so we assert | ||
| // the instrumentation against its own compile-time classpath as a basic sanity check. | ||
| !project.extensions.getByType<MuzzleExtension>().directives.any { it.assertPass } -> { | ||
| project.logger.info("No muzzle pass directives configured. Asserting pass against instrumentation compile-time dependencies") | ||
| assertMuzzle() | ||
| } | ||
| muzzleDirective.isPresent -> { | ||
| assertMuzzle(muzzleDirective.get()) | ||
| } |
Contributor
Author
There was a problem hiding this comment.
note: This was actually a bug, detected by a unit test. If there were no pass directive at all, the wrong branch was used.
| val locator = MavenRepositorySystemUtils.newServiceLocator().apply { | ||
| addService(RepositoryConnectorFactory::class.java, BasicRepositoryConnectorFactory::class.java) | ||
| addService(TransporterFactory::class.java, HttpTransporterFactory::class.java) | ||
| addService(TransporterFactory::class.java, FileTransporterFactory::class.java) |
Contributor
Author
There was a problem hiding this comment.
note: Necessary to support a fake maven repository (file based)
Comment on lines
-104
to
+106
| // removing leading ':' if present | ||
| val muzzleTaskName = taskName.removePrefix(":") | ||
| val projectPath = project.path.removePrefix(":") | ||
| muzzleTaskName == "muzzle" || "$projectPath:muzzle" == muzzleTaskName || | ||
| muzzleTaskName == "runMuzzle" | ||
| val taskProjectPath = taskName.substringBeforeLast(":", "") | ||
| val taskNameOnly = taskName.substringAfterLast(":") | ||
| val isRelevantForProject = taskProjectPath.isEmpty() || taskProjectPath == project.path | ||
|
|
||
| isRelevantForProject && taskNameOnly.endsWith("muzzle", ignoreCase = true) |
Contributor
Author
There was a problem hiding this comment.
note: Small change for better readability and avoiding useless and confusing prefix removal. Change performed after unit tests.
|
|
||
| check_build_src: | ||
| extends: .check_job | ||
| needs: [] |
Contributor
Author
There was a problem hiding this comment.
note: check_build_src do|should not need to wait on the build job.
| targets.configureEach { | ||
| testTask.configure { | ||
| enabled = project.hasProperty("runBuildSrcTests") | ||
| enabled = providers.systemProperty("runBuildSrcTests").isPresent or providers.systemProperty("idea.active").isPresent |
Contributor
Author
There was a problem hiding this comment.
note: Allow running tests from IntelliJ in ./buildSrc without the need to set the system property.
amarziali
approved these changes
Feb 23, 2026
Contributor
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
github-merge-queue bot
pushed a commit
that referenced
this pull request
Feb 23, 2026
* chore: New MuzzlePlugin tests * chore: Factors out the task planification out of muzzle plugin for easier testing * feat: Easier testing for buildSrc tests * chore: New plan tests * chore: First full integration test * chore: New muzzle integration tests * chore: New muzzle integration tests about reacting to environment * chore: Add language for easier code review * test: Rename * test: Uses task outcome * test: Simplify some test * test: Add new avoidance tests, simplify hasRelevantTask check * test: Adds basic up-to-date testcase * test: Refactors maven fixture to support more usecases * test: Ensures case is not up-to-date when classpath changes * test: Adds some muzzle plugin utils * test: adds muzzle directive tests * test: adds muzzle repo tests * test: adds assertInverse functional test * test: Use AssertJ * test: check_build_src do not the build job to run before * fix: Properly run fail directives
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Contributor
|
/merge |
|
View all feedbacks in Devflow UI.
PR already in the queue with status in_progress |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
gh-worker-dd-mergequeue-cf854d
bot
deleted the
bdu/muzzle-plugin-tests
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Cover the muzzle plugin by unit test, and functional tests.
Motivation
The muzzle plugin is a very important part of our test suite. However, without test it is easy to introduce silent breakage, the tests here help prevent this.
Additional notes
While AI was used, the results needed a lot of human intervention.
The
MuzzlePlugincode changes was only to help testing. One notable change though is a detected bug on theMuzzleTaskwhile writing unit tests (fortunately nothing was broken in our instrumentation muzzle checks).