Skip to content

Conversation

@bric3
Copy link
Contributor

@bric3 bric3 commented Sep 5, 2025

What Does This Do

Disclaimer: This only touches the build part (i.e. Gradle).

The current InstrumentPlugin uses eager gradle API and inject a task to run after the compile task, and in doing so, exchanging their output. This approach is uncommon, create as many instrument task as they are compile tasks, and make it harder to predict compilation output.

This PR rethink the approach to a compilation post-processor. So instead of injecting a task in the gradle task graph, each compile task have a post-processor action.

Also, the plugin is renamed to dd-trace-java.build-time-instrumentation for better expressing intent. (Related configuration points are also renamed),

In doing so, it makes the intent of this plug-in easier to grasp.

On another note, the single Groovy file has been spread to distinct types.

Motivation

  • Good usage of Gradle API.
  • Make it lazy.
  • Rename better conveys what this plugins do: instrument at build time. The name dd-trace-java.build-time-instrumentation was chosen to align dd-trace-java.call-site-instrumentation. Also, searching these string will bring more focused results.

Avoid Gradle eager API, avoid messing with compile tasks avoid,

Helps going toward convention plug-ins.

Additional Notes

Related PRs

Related to

Blocked by

What does this plugin

ℹ️ italic word indicate Gradle linguo.

  • Registers an extension under the buildTimeInstrumentation name (previously instrument)
  • Registers a configuration (to declare dependencies) under the buildTimeInstrumentationPlugin name (previously instrumentPluginClasspath)
  • Applies to Java, Kotlin, Scala, and Groovy main sources
  • Appends a post-processing doLast action to each matching compile task

Currently, this build time instrumentation is applied on instrumentation modules :

  • datadog.trace.agent.tooling.muzzle.MuzzleGradlePlugin : Creates muzzle-references at compile time for classes extending InstrumenterModule. This generates metadata about instrumentation compatibility.
  • datadog.trace.agent.tooling.bytebuddy.NewTaskForGradlePlugin : Used for datadog.trace.instrumentation.java.concurrent.WrapRunnableAsNewTaskInstrumentation instrumentation, to replace datadog.trace.bootstrap.instrumentation.java.concurrent.NewTaskForPlaceholder#newTaskFor by java.util.concurrent.AbstractExecutorService#newTaskFor(java.lang.Runnable, T) (which is protected and not accessible during compilation).
  • datadog.trace.agent.tooling.bytebuddy.reqctx.RewriteRequestContextAdvicePlugin : Transforms classes annotated with @RequiresRequestContext to inject request context handling.

...and in the otel agent

  • datadog.opentelemetry.tooling.shim.OtelShimGradlePlugin: Injects Datadog's OpenTelemetry shim into specific OpenTelemetry API classes like DefaultOpenTelemetry, GlobalOpenTelemetry$ObfuscatedOpenTelemetry, ThreadLocalContextStorage, etc. This allows intercepting OpenTelemetry API calls at build-time.

Additional checks

As in #9514 I also checked the produced jars between master and this branch using my jardiff tool I already mentioned there (I crafted it for validating #9514, other tools didn't work or didn't perform what was needed).

Note

🔗 https://github.com/bric3/jardiff

Shortcomings:

  • not released anywhere, one has to build it - a possible idea would be to integrate this in a gradle plugin to make the diff easier to read.
  • needs JDK 25

Ran the following command on this branch (folder dd-trace-java-copy-2), and on master (folder dd-trace-java-copy-1).

$ ./gradlew :dd-java-agent:shadowJar

The following assumes that jardiff is actually this command java -jar jardiff/build/libs/jardiff-0.1.0-SNAPSHOT.jar, and assuming this is a Java 25. Also, while not strictly necessary the following snippets use delta to render the diff.

The following difference have been checked:

  • dd-java-agent

    jardiff ../dd-trace-java-copy-{1,2}/dd-java-agent/build/libs/dd-java-agent-1.58.0-SNAPSHOT.jar \
      -c classdata \
      --exclude "**/*.yaml,**/*.MF,**/*.version,*.version,*.txt" \
      | delta --syntax-theme=GitHub --light 

    No differences:

    image

@bric3 bric3 requested review from a team as code owners September 5, 2025 12:50
@bric3 bric3 requested a review from amarziali September 5, 2025 12:50
@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2025

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@bric3 bric3 marked this pull request as draft September 5, 2025 12:50
@bric3 bric3 added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Sep 5, 2025
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, what would be the benefit? Faster build?

@bric3 bric3 force-pushed the bdu/lazy-instrument-plugin branch from 248b4eb to 3094d50 Compare September 5, 2025 17:02
@bric3
Copy link
Contributor Author

bric3 commented Sep 10, 2025

Status report

The new approach of the instrument post-processing works in almost all simple projects, e.g.

Important

The diff was made by this custom tool https://github.com/bric3/jardiff

:dd-java-agent:instrumentation:iast-instrumenter
$ mise exec java@corretto-24 -- jardiff /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java{-copy-1,}/dd-java-agent/instrumentation/iast-instrumenter/build/classes/java/main                      17:01:29
Comparing:
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java-copy-1/dd-java-agent/instrumentation/iast-instrumenter/build/classes/java/main
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java/dd-java-agent/instrumentation/iast-instrumenter/build/classes/java/main

✔️ META-INF/services/datadog.trace.agent.tooling.InstrumenterModule
✔️ datadog/trace/instrumentation/iastinstrumenter/IastExclusionTrie.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$IastMatchers.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastHardcodedSecretListener$ReportSecretConsumer.class
✔️ datadog/trace/instrumentation/iastinstrumenter/SourceMapperImpl.class
✔️ datadog/trace/instrumentation/iastinstrumenter/StratumListener.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastHardcodedSecretListener.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$IastCallSiteSupplier.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$IastMatchers$1.class
✔️ datadog/trace/instrumentation/iastinstrumenter/telemetry/TelemetryCallSiteSupplier$IteratorAdapter.class
✔️ datadog/trace/instrumentation/iastinstrumenter/telemetry/TelemetryCallSiteSupplier$1.class
✔️ datadog/trace/instrumentation/iastinstrumenter/telemetry/TelemetryCallSiteSupplier.class
✔️ datadog/trace/instrumentation/iastinstrumenter/service/CallSitesLoader.class
:dd-java-agent:instrumentation:mule-4
$  mise exec java@corretto-24 -- jardiff /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java{-copy-1,}/dd-java-agent/instrumentation/mule-4/build/classes/java/main                                 17:26:08
Comparing:
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java-copy-1/dd-java-agent/instrumentation/mule-4/build/classes/java/main
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java/dd-java-agent/instrumentation/mule-4/build/classes/java/main

✔️ META-INF/services/datadog.trace.agent.tooling.InstrumenterModule
✔️ datadog/trace/instrumentation/mule4/EventTracerInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/ComponentMessageProcessorInstrumentation$ProcessAdvice.class
✔️ datadog/trace/instrumentation/mule4/AbstractMuleInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/EventTracerInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/SpanState.class
✔️ datadog/trace/instrumentation/mule4/ComponentMessageProcessorInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/JpmsMuleInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/ExecutionInitialSpanInfoInstrumentation$StoreComponentAdvice.class
✔️ datadog/trace/instrumentation/mule4/EventContextInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/ComponentMessageProcessorInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/EventContextCreationAdvice.class
✔️ datadog/trace/instrumentation/mule4/NoopMuleSpan.class
✔️ datadog/trace/instrumentation/mule4/DDEventTracer.class
✔️ datadog/trace/instrumentation/mule4/ExecutionInitialSpanInfoInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/MuleDecorator.class
✔️ datadog/trace/instrumentation/mule4/EventTracerInstrumentation$SwapCoreTracerAdvice.class
✔️ datadog/trace/instrumentation/mule4/JpmsMuleInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/ExecutionInitialSpanInfoInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/EventContextInstrumentation.class
:dd-java-agent:instrumentation:play.2.4
:dd-java-agent:agent-otel:otel-bootstrap

jardiff on otel-bootstrap

Etc.

However, :dd-java-agent:instrumentation:jetty-9 is oddly configured and must be

mise exec java@corretto-24 -- jardiff /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java{-copy-1,}/dd-java-agent/instrumentation/jetty-9/build/classes/java/main

And as such not all classes are available when post processing kicks in. Possibly this project needs a heavier setup refactoring.

Few other projects with non-standard configurations to fix

  • :dd-java-agent:instrumentation:play.2.4
  • :dd-java-agent:instrumentation:play.2.6

Executing a simple ./gradlew help :

~5K less created tasks during configuration
image

https://scans.gradle.com/s/jqasddzbs75ve/performance/configuration#summary-tasks-created-during-configuration

Note

~40k tasks created is still way too much and impacts the build performance considerably

versus master, with ~45k task created during configuration
image

https://scans.gradle.com/s/vg7nn7oitpx5k/performance/configuration#summary-tasks-created-during-configuration

@AlexeyKuznetsov-DD
Copy link
Contributor

AlexeyKuznetsov-DD commented Dec 17, 2025

Just a minor comment. Probably it make sense to refactor this plugin to Kotlin.
I did some naive 1-1 migration here
It is not working properly, so I will close my PR as you are working on it and have more context.

The current implementation was using bad practices in various ways,
`project.afterEvaluate`, task creation, explicit `dependsOn`

# Conflicts:
#	buildSrc/src/main/groovy/InstrumentPlugin.groovy
#	buildSrc/src/main/groovy/MuzzlePlugin.groovy
#	dd-java-agent/instrumentation/jetty-9/build.gradle

# Conflicts:
#	buildSrc/src/main/groovy/InstrumentPlugin.groovy
#	buildSrc/src/test/groovy/InstrumentPluginTest.groovy
#	dd-java-agent/instrumentation/play/play-2.4/build.gradle
#	dd-java-agent/instrumentation/play/play-2.6/build.gradle
@bric3 bric3 force-pushed the bdu/lazy-instrument-plugin branch from 899f804 to 9b5556c Compare December 31, 2025 09:06
@bric3
Copy link
Contributor Author

bric3 commented Dec 31, 2025

Just a minor comment. Probably it make sense to refactor this plugin to Kotlin.

Yes, but I'd rather do it, in a second pass. Once this settles down.

@pr-commenter
Copy link

pr-commenter bot commented Jan 5, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/lazy-instrument-plugin
git_commit_date 1767637205 1767703998
git_commit_sha bd3f6f5 5311fd3
release_version 1.58.0~bd3f6f5c89 1.58.0-SNAPSHOT~5311fd3ea8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1767706097 1767706097
ci_job_id 1329399495 1329399495
ci_pipeline_id 89187572 89187572
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-wwhu1gye 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-wwhu1gye 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 10 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:iast:Debugger better
[-7.157ms; -1.605ms] or [-10.282%; -2.305%]
65.227ms 69.608ms
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1085410
Total [baseline] (8.777 s) : 0, 8776766
Agent [candidate] (1.095 s) : 0, 1095327
Total [candidate] (8.812 s) : 0, 8812288
section iast
Agent [baseline] (1.227 s) : 0, 1226795
Total [baseline] (9.373 s) : 0, 9373371
Agent [candidate] (1.233 s) : 0, 1232838
Total [candidate] (9.339 s) : 0, 9339380
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent iast 1.227 s 141.385 ms (13.0%)
Total tracing 8.777 s -
Total iast 9.373 s 596.606 ms (6.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.095 s -
Agent iast 1.233 s 137.511 ms (12.6%)
Total tracing 8.812 s -
Total iast 9.339 s 527.091 ms (6.0%)
gantt
    title insecure-bank - break down per module: candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (651.831 ms) : 0, 651831
BytebuddyAgent [candidate] (657.349 ms) : 0, 657349
GlobalTracer [baseline] (282.883 ms) : 0, 282883
GlobalTracer [candidate] (285.87 ms) : 0, 285870
AppSec [baseline] (32.731 ms) : 0, 32731
AppSec [candidate] (33.079 ms) : 0, 33079
Debugger [baseline] (67.728 ms) : 0, 67728
Debugger [candidate] (68.475 ms) : 0, 68475
Remote Config [baseline] (642.001 µs) : 0, 642
Remote Config [candidate] (629.818 µs) : 0, 630
Telemetry [baseline] (9.009 ms) : 0, 9009
Telemetry [candidate] (9.093 ms) : 0, 9093
Flare Poller [baseline] (3.762 ms) : 0, 3762
Flare Poller [candidate] (3.759 ms) : 0, 3759
section iast
crashtracking [baseline] (1.194 ms) : 0, 1194
crashtracking [candidate] (1.188 ms) : 0, 1188
BytebuddyAgent [baseline] (793.543 ms) : 0, 793543
BytebuddyAgent [candidate] (796.776 ms) : 0, 796776
GlobalTracer [baseline] (256.574 ms) : 0, 256574
GlobalTracer [candidate] (258.959 ms) : 0, 258959
AppSec [baseline] (34.553 ms) : 0, 34553
AppSec [candidate] (33.897 ms) : 0, 33897
Debugger [baseline] (65.38 ms) : 0, 65380
Debugger [candidate] (66.295 ms) : 0, 66295
Remote Config [baseline] (601.877 µs) : 0, 602
Remote Config [candidate] (593.286 µs) : 0, 593
Telemetry [baseline] (8.591 ms) : 0, 8591
Telemetry [candidate] (8.548 ms) : 0, 8548
Flare Poller [baseline] (3.669 ms) : 0, 3669
Flare Poller [candidate] (3.622 ms) : 0, 3622
IAST [baseline] (27.139 ms) : 0, 27139
IAST [candidate] (27.342 ms) : 0, 27342
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.103 s) : 0, 1102909
Total [baseline] (10.822 s) : 0, 10821666
Agent [candidate] (1.086 s) : 0, 1086349
Total [candidate] (10.835 s) : 0, 10834565
section appsec
Agent [baseline] (1.266 s) : 0, 1265820
Total [baseline] (10.977 s) : 0, 10976557
Agent [candidate] (1.291 s) : 0, 1290887
Total [candidate] (11.023 s) : 0, 11023422
section iast
Agent [baseline] (1.235 s) : 0, 1235492
Total [baseline] (11.3 s) : 0, 11300205
Agent [candidate] (1.243 s) : 0, 1242797
Total [candidate] (11.243 s) : 0, 11242785
section profiling
Agent [baseline] (1.207 s) : 0, 1207159
Total [baseline] (10.953 s) : 0, 10953459
Agent [candidate] (1.216 s) : 0, 1216212
Total [candidate] (11.004 s) : 0, 11003573
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.103 s -
Agent appsec 1.266 s 162.911 ms (14.8%)
Agent iast 1.235 s 132.583 ms (12.0%)
Agent profiling 1.207 s 104.25 ms (9.5%)
Total tracing 10.822 s -
Total appsec 10.977 s 154.89 ms (1.4%)
Total iast 11.3 s 478.539 ms (4.4%)
Total profiling 10.953 s 131.793 ms (1.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent appsec 1.291 s 204.538 ms (18.8%)
Agent iast 1.243 s 156.448 ms (14.4%)
Agent profiling 1.216 s 129.863 ms (12.0%)
Total tracing 10.835 s -
Total appsec 11.023 s 188.857 ms (1.7%)
Total iast 11.243 s 408.221 ms (3.8%)
Total profiling 11.004 s 169.008 ms (1.6%)
gantt
    title petclinic - break down per module: candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.209 ms) : 0, 1209
crashtracking [candidate] (1.18 ms) : 0, 1180
BytebuddyAgent [baseline] (662.931 ms) : 0, 662931
BytebuddyAgent [candidate] (651.819 ms) : 0, 651819
GlobalTracer [baseline] (286.712 ms) : 0, 286712
GlobalTracer [candidate] (283.198 ms) : 0, 283198
AppSec [baseline] (33.222 ms) : 0, 33222
AppSec [candidate] (32.666 ms) : 0, 32666
Debugger [baseline] (69.168 ms) : 0, 69168
Debugger [candidate] (68.51 ms) : 0, 68510
Remote Config [baseline] (645.39 µs) : 0, 645
Remote Config [candidate] (614.18 µs) : 0, 614
Telemetry [baseline] (9.166 ms) : 0, 9166
Telemetry [candidate] (9.031 ms) : 0, 9031
Flare Poller [baseline] (3.77 ms) : 0, 3770
Flare Poller [candidate] (3.74 ms) : 0, 3740
section appsec
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.2 ms) : 0, 1200
BytebuddyAgent [baseline] (690.618 ms) : 0, 690618
BytebuddyAgent [candidate] (706.716 ms) : 0, 706716
GlobalTracer [baseline] (258.163 ms) : 0, 258163
GlobalTracer [candidate] (264.087 ms) : 0, 264087
AppSec [baseline] (174.866 ms) : 0, 174866
AppSec [candidate] (176.774 ms) : 0, 176774
Debugger [baseline] (66.937 ms) : 0, 66937
Debugger [candidate] (66.845 ms) : 0, 66845
Remote Config [baseline] (770.227 µs) : 0, 770
Remote Config [candidate] (751.551 µs) : 0, 752
Telemetry [baseline] (9.486 ms) : 0, 9486
Telemetry [candidate] (9.497 ms) : 0, 9497
Flare Poller [baseline] (3.792 ms) : 0, 3792
Flare Poller [candidate] (3.76 ms) : 0, 3760
IAST [baseline] (24.544 ms) : 0, 24544
IAST [candidate] (25.358 ms) : 0, 25358
section iast
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.201 ms) : 0, 1201
BytebuddyAgent [baseline] (798.331 ms) : 0, 798331
BytebuddyAgent [candidate] (804.609 ms) : 0, 804609
GlobalTracer [baseline] (258.354 ms) : 0, 258354
GlobalTracer [candidate] (259.866 ms) : 0, 259866
AppSec [baseline] (32.08 ms) : 0, 32080
AppSec [candidate] (35.63 ms) : 0, 35630
Debugger [baseline] (69.608 ms) : 0, 69608
Debugger [candidate] (65.227 ms) : 0, 65227
Remote Config [baseline] (555.089 µs) : 0, 555
Remote Config [candidate] (569.906 µs) : 0, 570
Telemetry [baseline] (8.618 ms) : 0, 8618
Telemetry [candidate] (8.627 ms) : 0, 8627
Flare Poller [baseline] (3.716 ms) : 0, 3716
Flare Poller [candidate] (3.643 ms) : 0, 3643
IAST [baseline] (27.372 ms) : 0, 27372
IAST [candidate] (27.538 ms) : 0, 27538
section profiling
crashtracking [baseline] (1.23 ms) : 0, 1230
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (704.48 ms) : 0, 704480
BytebuddyAgent [candidate] (710.637 ms) : 0, 710637
GlobalTracer [baseline] (220.713 ms) : 0, 220713
GlobalTracer [candidate] (222.6 ms) : 0, 222600
AppSec [baseline] (32.135 ms) : 0, 32135
AppSec [candidate] (32.193 ms) : 0, 32193
Debugger [baseline] (68.249 ms) : 0, 68249
Debugger [candidate] (68.034 ms) : 0, 68034
Remote Config [baseline] (664.737 µs) : 0, 665
Remote Config [candidate] (639.971 µs) : 0, 640
Telemetry [baseline] (8.734 ms) : 0, 8734
Telemetry [candidate] (8.913 ms) : 0, 8913
Flare Poller [baseline] (3.641 ms) : 0, 3641
Flare Poller [candidate] (3.673 ms) : 0, 3673
ProfilingAgent [baseline] (97.346 ms) : 0, 97346
ProfilingAgent [candidate] (97.817 ms) : 0, 97817
Profiling [baseline] (97.915 ms) : 0, 97915
Profiling [candidate] (98.388 ms) : 0, 98388
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/lazy-instrument-plugin
git_commit_date 1767637205 1767703998
git_commit_sha bd3f6f5 5311fd3
release_version 1.58.0~bd3f6f5c89 1.58.0-SNAPSHOT~5311fd3ea8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1767706740 1767706740
ci_job_id 1329399497 1329399497
ci_pipeline_id 89187572 89187572
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-ky6dp7bw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-ky6dp7bw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 19 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:petclinic:tracing:high_load better
[-759.728µs; -366.567µs] or [-4.254%; -2.052%]
same
[-1124.189µs; +256.212µs] or [-3.860%; +0.880%]
unstable
[-23.629op/s; +33.316op/s] or [-9.187%; +12.954%]
17.297ms 28.691ms 262.031op/s 17.861ms 29.125ms 257.188op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.143 ms) : 17956, 18330
.   : milestone, 18143,
appsec (18.676 ms) : 18485, 18868
.   : milestone, 18676,
code_origins (17.591 ms) : 17419, 17763
.   : milestone, 17591,
iast (17.861 ms) : 17682, 18041
.   : milestone, 17861,
profiling (18.854 ms) : 18664, 19043
.   : milestone, 18854,
tracing (18.143 ms) : 17964, 18323
.   : milestone, 18143,
section candidate
no_agent (18.355 ms) : 18170, 18541
.   : milestone, 18355,
appsec (19.15 ms) : 18958, 19341
.   : milestone, 19150,
code_origins (17.603 ms) : 17429, 17778
.   : milestone, 17603,
iast (17.802 ms) : 17625, 17978
.   : milestone, 17802,
profiling (18.553 ms) : 18370, 18735
.   : milestone, 18553,
tracing (17.808 ms) : 17630, 17986
.   : milestone, 17808,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.143 ms [17.956 ms, 18.33 ms] -
appsec 18.676 ms [18.485 ms, 18.868 ms] 533.417 µs (2.9%)
code_origins 17.591 ms [17.419 ms, 17.763 ms] -552.01 µs (-3.0%)
iast 17.861 ms [17.682 ms, 18.041 ms] -281.688 µs (-1.6%)
profiling 18.854 ms [18.664 ms, 19.043 ms] 710.554 µs (3.9%)
tracing 18.143 ms [17.964 ms, 18.323 ms] 0.492 µs (0.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.355 ms [18.17 ms, 18.541 ms] -
appsec 19.15 ms [18.958 ms, 19.341 ms] 794.076 µs (4.3%)
code_origins 17.603 ms [17.429 ms, 17.778 ms] -751.968 µs (-4.1%)
iast 17.802 ms [17.625 ms, 17.978 ms] -553.876 µs (-3.0%)
profiling 18.553 ms [18.37 ms, 18.735 ms] 197.107 µs (1.1%)
tracing 17.808 ms [17.63 ms, 17.986 ms] -547.633 µs (-3.0%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.197 ms) : 1185, 1208
.   : milestone, 1197,
iast (3.166 ms) : 3123, 3209
.   : milestone, 3166,
iast_FULL (5.587 ms) : 5533, 5641
.   : milestone, 5587,
iast_GLOBAL (3.65 ms) : 3582, 3719
.   : milestone, 3650,
profiling (2.078 ms) : 2060, 2096
.   : milestone, 2078,
tracing (1.916 ms) : 1897, 1935
.   : milestone, 1916,
section candidate
no_agent (1.189 ms) : 1177, 1201
.   : milestone, 1189,
iast (3.154 ms) : 3115, 3193
.   : milestone, 3154,
iast_FULL (5.623 ms) : 5567, 5679
.   : milestone, 5623,
iast_GLOBAL (3.654 ms) : 3591, 3716
.   : milestone, 3654,
profiling (2.076 ms) : 2057, 2095
.   : milestone, 2076,
tracing (1.775 ms) : 1760, 1790
.   : milestone, 1775,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.197 ms [1.185 ms, 1.208 ms] -
iast 3.166 ms [3.123 ms, 3.209 ms] 1.969 ms (164.5%)
iast_FULL 5.587 ms [5.533 ms, 5.641 ms] 4.39 ms (366.8%)
iast_GLOBAL 3.65 ms [3.582 ms, 3.719 ms] 2.454 ms (205.0%)
profiling 2.078 ms [2.06 ms, 2.096 ms] 881.029 µs (73.6%)
tracing 1.916 ms [1.897 ms, 1.935 ms] 719.554 µs (60.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.189 ms [1.177 ms, 1.201 ms] -
iast 3.154 ms [3.115 ms, 3.193 ms] 1.965 ms (165.3%)
iast_FULL 5.623 ms [5.567 ms, 5.679 ms] 4.434 ms (372.9%)
iast_GLOBAL 3.654 ms [3.591 ms, 3.716 ms] 2.465 ms (207.3%)
profiling 2.076 ms [2.057 ms, 2.095 ms] 887.298 µs (74.6%)
tracing 1.775 ms [1.76 ms, 1.79 ms] 586.201 µs (49.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/lazy-instrument-plugin
git_commit_date 1767637205 1767703998
git_commit_sha bd3f6f5 5311fd3
release_version 1.58.0~bd3f6f5c89 1.58.0-SNAPSHOT~5311fd3ea8
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1767706325 1767706325
ci_job_id 1329399499 1329399499
ci_pipeline_id 89187572 89187572
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-78mhrdon 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-78mhrdon 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (3.744 ms) : 3523, 3964
.   : milestone, 3744,
iast (2.22 ms) : 2155, 2285
.   : milestone, 2220,
iast_GLOBAL (2.264 ms) : 2199, 2330
.   : milestone, 2264,
profiling (2.098 ms) : 2043, 2153
.   : milestone, 2098,
tracing (2.046 ms) : 1995, 2097
.   : milestone, 2046,
section candidate
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (3.748 ms) : 3526, 3970
.   : milestone, 3748,
iast (2.224 ms) : 2158, 2289
.   : milestone, 2224,
iast_GLOBAL (2.253 ms) : 2188, 2318
.   : milestone, 2253,
profiling (2.072 ms) : 2019, 2125
.   : milestone, 2072,
tracing (2.053 ms) : 2002, 2104
.   : milestone, 2053,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 3.744 ms [3.523 ms, 3.964 ms] 2.268 ms (153.7%)
iast 2.22 ms [2.155 ms, 2.285 ms] 743.989 µs (50.4%)
iast_GLOBAL 2.264 ms [2.199 ms, 2.33 ms] 788.617 µs (53.4%)
profiling 2.098 ms [2.043 ms, 2.153 ms] 622.232 µs (42.2%)
tracing 2.046 ms [1.995 ms, 2.097 ms] 570.146 µs (38.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.468 ms, 1.491 ms] -
appsec 3.748 ms [3.526 ms, 3.97 ms] 2.268 ms (153.3%)
iast 2.224 ms [2.158 ms, 2.289 ms] 744.182 µs (50.3%)
iast_GLOBAL 2.253 ms [2.188 ms, 2.318 ms] 774.1 µs (52.3%)
profiling 2.072 ms [2.019 ms, 2.125 ms] 592.578 µs (40.1%)
tracing 2.053 ms [2.002 ms, 2.104 ms] 573.61 µs (38.8%)
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.58.0-SNAPSHOT~5311fd3ea8, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.441 s) : 15441000, 15441000
.   : milestone, 15441000,
appsec (14.603 s) : 14603000, 14603000
.   : milestone, 14603000,
iast (17.736 s) : 17736000, 17736000
.   : milestone, 17736000,
iast_GLOBAL (17.733 s) : 17733000, 17733000
.   : milestone, 17733000,
profiling (15.03 s) : 15030000, 15030000
.   : milestone, 15030000,
tracing (14.508 s) : 14508000, 14508000
.   : milestone, 14508000,
section candidate
no_agent (15.538 s) : 15538000, 15538000
.   : milestone, 15538000,
appsec (14.76 s) : 14760000, 14760000
.   : milestone, 14760000,
iast (18.134 s) : 18134000, 18134000
.   : milestone, 18134000,
iast_GLOBAL (17.696 s) : 17696000, 17696000
.   : milestone, 17696000,
profiling (14.758 s) : 14758000, 14758000
.   : milestone, 14758000,
tracing (14.582 s) : 14582000, 14582000
.   : milestone, 14582000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.441 s [15.441 s, 15.441 s] -
appsec 14.603 s [14.603 s, 14.603 s] -838.0 ms (-5.4%)
iast 17.736 s [17.736 s, 17.736 s] 2.295 s (14.9%)
iast_GLOBAL 17.733 s [17.733 s, 17.733 s] 2.292 s (14.8%)
profiling 15.03 s [15.03 s, 15.03 s] -411.0 ms (-2.7%)
tracing 14.508 s [14.508 s, 14.508 s] -933.0 ms (-6.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.538 s [15.538 s, 15.538 s] -
appsec 14.76 s [14.76 s, 14.76 s] -778.0 ms (-5.0%)
iast 18.134 s [18.134 s, 18.134 s] 2.596 s (16.7%)
iast_GLOBAL 17.696 s [17.696 s, 17.696 s] 2.158 s (13.9%)
profiling 14.758 s [14.758 s, 14.758 s] -780.0 ms (-5.0%)
tracing 14.582 s [14.582 s, 14.582 s] -956.0 ms (-6.2%)

@bric3 bric3 changed the title Rewrite InstrumentPlugin to be lazy Rewrite InstrumentPlugin as compilation post-processor and lazy Jan 5, 2026
@bric3 bric3 force-pushed the bdu/lazy-instrument-plugin branch from a485d08 to 83aa2a4 Compare January 6, 2026 12:41
@bric3 bric3 changed the title Rewrite InstrumentPlugin as compilation post-processor and lazy Rewrite build-time instrumentation as compilation post-processor (using Gradle lazy APIs) Jan 6, 2026
@bric3 bric3 force-pushed the bdu/lazy-instrument-plugin branch from 83aa2a4 to 5311fd3 Compare January 6, 2026 12:53
@bric3 bric3 marked this pull request as ready for review January 6, 2026 15:43
@bric3 bric3 requested review from a team as code owners January 6, 2026 15:43
@bric3 bric3 requested review from AlexeyKuznetsov-DD, claponcet, manuel-alvarez-alvarez and mcculls and removed request for a team January 6, 2026 15:43
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left minor notes and questions.

logger.info('[BuildTimeInstrumentationPlugin] Applying buildTimeInstrumentationPlugin configuration as compile task input')
it.inputs.files(project.configurations.named(BUILD_TIME_INSTRUMENTATION_PLUGIN_CONFIGURATION))

if (it.source.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this check? if it was already checked on line 103?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, maybe this can be removed.

Comment on lines +124 to +128
it.inputs.property("javaVersion", javaVersion)

it.inputs.property("plugins", extension.plugins)

it.inputs.files(extension.additionalClasspath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor - remove empty lines to keep lines logically together.

Suggested change
it.inputs.property("javaVersion", javaVersion)
it.inputs.property("plugins", extension.plugins)
it.inputs.files(extension.additionalClasspath)
it.inputs.property("javaVersion", javaVersion)
it.inputs.property("plugins", extension.plugins)
it.inputs.files(extension.additionalClasspath)

tmpUninstrumentedClasses
)

// This were the post processing happens, i.e. were the instrumentation is applied
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: I would re-phrase this comment a bit, probably make it short: // Apply instrumentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// This were the post processing happens, i.e. were the instrumentation is applied
// Post-process classes with instrumentation

Comment on lines +56 to +62
logger.info(
"[InstrumentPostProcessingAction] About to instrument classes \n" +
" javaVersion=${javaVersion}, \n" +
" plugins=${plugins.get()}, \n" +
" instrumentingClassPath=${instrumentingClassPath.files}, \n" +
" rawClassesDirectory=${compilerOutputDirectory.get().asFile}"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: can we use multiline string here?

logger.info('[BuildTimeInstrumentationPlugin] Applying buildTimeInstrumentationPlugin configuration as compile task input')
it.inputs.files(project.configurations.named(BUILD_TIME_INSTRUMENTATION_PLUGIN_CONFIGURATION))

if (it.source.isEmpty()) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, maybe this can be removed.

tmpUninstrumentedClasses
)

// This were the post processing happens, i.e. were the instrumentation is applied
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// This were the post processing happens, i.e. were the instrumentation is applied
// Post-process classes with instrumentation

it.dependencies.add(subProj.dependencies.project(path: ':dd-java-agent:agent-tooling', configuration: 'instrumentPluginClasspath'))
it.dependencies.add(subProj.dependencies.project(
path: ':dd-java-agent:agent-tooling',
configuration: 'buildTimeInstrumentationPlugin'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: Maybe I'll rename this configuration to avoid confusion with the instrumentation plugin.

Something like

        configuration: 'buildTimeInstrumentationPlugin'
        configuration: 'buildTimeInstrumentationToolingPlugins'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: needs investigation Issues needing investigations tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants