-
Notifications
You must be signed in to change notification settings - Fork 323
Fix DSM queue names with Kafka Connect IBM MQ connectors #10318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Fix DSM queue names with Kafka Connect IBM MQ connectors #10318
Conversation
When using Kafka Connect with IBM MQ connectors, DSM was reporting incorrect queue names with schema-derived suffixes like _messagebody_0. This occurred because Kafka Connect schema converters add index suffixes to field names for union/optional types. This fix sanitizes queue/topic names to remove these suffixes: - _messagebody_N - _text_N - _bytes_N - _map_N - _value_N Fixes Zendesk ticket #2429181 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 6 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083227
Total [baseline] (8.745 s) : 0, 8744817
Agent [candidate] (1.083 s) : 0, 1082741
Total [candidate] (8.737 s) : 0, 8736927
section iast
Agent [baseline] (1.221 s) : 0, 1220526
Total [baseline] (9.254 s) : 0, 9254333
Agent [candidate] (1.226 s) : 0, 1225755
Total [candidate] (9.312 s) : 0, 9311950
gantt
title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.188 ms) : 0, 1188
BytebuddyAgent [baseline] (651.157 ms) : 0, 651157
BytebuddyAgent [candidate] (651.288 ms) : 0, 651288
GlobalTracer [baseline] (281.936 ms) : 0, 281936
GlobalTracer [candidate] (281.962 ms) : 0, 281962
AppSec [baseline] (32.703 ms) : 0, 32703
AppSec [candidate] (32.538 ms) : 0, 32538
Debugger [baseline] (67.441 ms) : 0, 67441
Debugger [candidate] (67.028 ms) : 0, 67028
Remote Config [baseline] (619.197 µs) : 0, 619
Remote Config [candidate] (614.599 µs) : 0, 615
Telemetry [baseline] (8.954 ms) : 0, 8954
Telemetry [candidate] (8.878 ms) : 0, 8878
Flare Poller [baseline] (3.721 ms) : 0, 3721
Flare Poller [candidate] (3.683 ms) : 0, 3683
section iast
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (789.751 ms) : 0, 789751
BytebuddyAgent [candidate] (793.032 ms) : 0, 793032
GlobalTracer [baseline] (255.449 ms) : 0, 255449
GlobalTracer [candidate] (256.529 ms) : 0, 256529
IAST [baseline] (27.066 ms) : 0, 27066
IAST [candidate] (27.033 ms) : 0, 27033
AppSec [baseline] (34.219 ms) : 0, 34219
AppSec [candidate] (35.462 ms) : 0, 35462
Debugger [baseline] (64.624 ms) : 0, 64624
Debugger [candidate] (64.302 ms) : 0, 64302
Remote Config [baseline] (617.818 µs) : 0, 618
Remote Config [candidate] (603.348 µs) : 0, 603
Telemetry [baseline] (8.484 ms) : 0, 8484
Telemetry [candidate] (8.551 ms) : 0, 8551
Flare Poller [baseline] (3.584 ms) : 0, 3584
Flare Poller [candidate] (3.514 ms) : 0, 3514
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.087 s) : 0, 1087135
Total [baseline] (10.733 s) : 0, 10733337
Agent [candidate] (1.083 s) : 0, 1082948
Total [candidate] (10.774 s) : 0, 10774473
section appsec
Agent [baseline] (1.276 s) : 0, 1275764
Total [baseline] (10.932 s) : 0, 10931832
Agent [candidate] (1.267 s) : 0, 1267297
Total [candidate] (10.951 s) : 0, 10950506
section iast
Agent [baseline] (1.241 s) : 0, 1240815
Total [baseline] (11.215 s) : 0, 11215337
Agent [candidate] (1.225 s) : 0, 1225372
Total [candidate] (11.221 s) : 0, 11221325
section profiling
Agent [baseline] (1.206 s) : 0, 1206282
Total [baseline] (10.998 s) : 0, 10998104
Agent [candidate] (1.209 s) : 0, 1208722
Total [candidate] (10.979 s) : 0, 10979176
gantt
title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.179 ms) : 0, 1179
BytebuddyAgent [baseline] (654.028 ms) : 0, 654028
BytebuddyAgent [candidate] (650.188 ms) : 0, 650188
GlobalTracer [baseline] (282.148 ms) : 0, 282148
GlobalTracer [candidate] (282.101 ms) : 0, 282101
AppSec [baseline] (32.56 ms) : 0, 32560
AppSec [candidate] (32.625 ms) : 0, 32625
Debugger [baseline] (68.123 ms) : 0, 68123
Debugger [candidate] (68.048 ms) : 0, 68048
Remote Config [baseline] (623.22 µs) : 0, 623
Remote Config [candidate] (625.398 µs) : 0, 625
Telemetry [baseline] (8.98 ms) : 0, 8980
Telemetry [candidate] (9.012 ms) : 0, 9012
Flare Poller [baseline] (3.699 ms) : 0, 3699
Flare Poller [candidate] (3.719 ms) : 0, 3719
section appsec
crashtracking [baseline] (1.196 ms) : 0, 1196
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (697.927 ms) : 0, 697927
BytebuddyAgent [candidate] (692.594 ms) : 0, 692594
GlobalTracer [baseline] (260.577 ms) : 0, 260577
GlobalTracer [candidate] (258.293 ms) : 0, 258293
IAST [baseline] (24.935 ms) : 0, 24935
IAST [candidate] (24.678 ms) : 0, 24678
AppSec [baseline] (174.07 ms) : 0, 174070
AppSec [candidate] (174.76 ms) : 0, 174760
Debugger [baseline] (67.469 ms) : 0, 67469
Debugger [candidate] (66.411 ms) : 0, 66411
Remote Config [baseline] (753.565 µs) : 0, 754
Remote Config [candidate] (799.702 µs) : 0, 800
Telemetry [baseline] (9.389 ms) : 0, 9389
Telemetry [candidate] (9.429 ms) : 0, 9429
Flare Poller [baseline] (3.762 ms) : 0, 3762
Flare Poller [candidate] (3.723 ms) : 0, 3723
section iast
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (803.212 ms) : 0, 803212
BytebuddyAgent [candidate] (791.875 ms) : 0, 791875
GlobalTracer [baseline] (258.876 ms) : 0, 258876
GlobalTracer [candidate] (256.326 ms) : 0, 256326
IAST [baseline] (27.425 ms) : 0, 27425
IAST [candidate] (27.122 ms) : 0, 27122
AppSec [baseline] (34.926 ms) : 0, 34926
AppSec [candidate] (34.556 ms) : 0, 34556
Debugger [baseline] (66.488 ms) : 0, 66488
Debugger [candidate] (66.123 ms) : 0, 66123
Remote Config [baseline] (579.998 µs) : 0, 580
Remote Config [candidate] (610.949 µs) : 0, 611
Telemetry [baseline] (8.629 ms) : 0, 8629
Telemetry [candidate] (8.491 ms) : 0, 8491
Flare Poller [baseline] (3.606 ms) : 0, 3606
Flare Poller [candidate] (3.585 ms) : 0, 3585
section profiling
crashtracking [baseline] (1.22 ms) : 0, 1220
crashtracking [candidate] (1.226 ms) : 0, 1226
BytebuddyAgent [baseline] (703.014 ms) : 0, 703014
BytebuddyAgent [candidate] (706.021 ms) : 0, 706021
GlobalTracer [baseline] (220.357 ms) : 0, 220357
GlobalTracer [candidate] (220.957 ms) : 0, 220957
AppSec [baseline] (31.987 ms) : 0, 31987
AppSec [candidate] (32.513 ms) : 0, 32513
Debugger [baseline] (68.497 ms) : 0, 68497
Debugger [candidate] (67.794 ms) : 0, 67794
Remote Config [baseline] (631.005 µs) : 0, 631
Remote Config [candidate] (630.836 µs) : 0, 631
Telemetry [baseline] (8.751 ms) : 0, 8751
Telemetry [candidate] (8.677 ms) : 0, 8677
Flare Poller [baseline] (3.617 ms) : 0, 3617
Flare Poller [candidate] (3.592 ms) : 0, 3592
ProfilingAgent [baseline] (98.367 ms) : 0, 98367
ProfilingAgent [candidate] (97.074 ms) : 0, 97074
Profiling [baseline] (98.961 ms) : 0, 98961
Profiling [candidate] (97.64 ms) : 0, 97640
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section baseline
no_agent (18.163 ms) : 17976, 18351
. : milestone, 18163,
appsec (18.242 ms) : 18059, 18425
. : milestone, 18242,
code_origins (17.631 ms) : 17457, 17805
. : milestone, 17631,
iast (17.565 ms) : 17392, 17739
. : milestone, 17565,
profiling (18.705 ms) : 18517, 18893
. : milestone, 18705,
tracing (17.637 ms) : 17463, 17810
. : milestone, 17637,
section candidate
no_agent (17.96 ms) : 17778, 18142
. : milestone, 17960,
appsec (18.583 ms) : 18395, 18770
. : milestone, 18583,
code_origins (18.596 ms) : 18410, 18782
. : milestone, 18596,
iast (17.578 ms) : 17403, 17753
. : milestone, 17578,
profiling (18.513 ms) : 18328, 18698
. : milestone, 18513,
tracing (17.603 ms) : 17427, 17779
. : milestone, 17603,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section baseline
no_agent (1.199 ms) : 1187, 1211
. : milestone, 1199,
iast (3.126 ms) : 3089, 3163
. : milestone, 3126,
iast_FULL (5.668 ms) : 5612, 5724
. : milestone, 5668,
iast_GLOBAL (3.563 ms) : 3506, 3621
. : milestone, 3563,
profiling (1.986 ms) : 1968, 2004
. : milestone, 1986,
tracing (1.784 ms) : 1769, 1798
. : milestone, 1784,
section candidate
no_agent (1.176 ms) : 1164, 1187
. : milestone, 1176,
iast (3.155 ms) : 3114, 3197
. : milestone, 3155,
iast_FULL (5.69 ms) : 5633, 5746
. : milestone, 5690,
iast_GLOBAL (3.472 ms) : 3422, 3523
. : milestone, 3472,
profiling (1.906 ms) : 1890, 1921
. : milestone, 1906,
tracing (1.794 ms) : 1780, 1809
. : milestone, 1794,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section baseline
no_agent (1.486 ms) : 1474, 1498
. : milestone, 1486,
appsec (4.214 ms) : 3959, 4469
. : milestone, 4214,
iast (2.576 ms) : 2473, 2678
. : milestone, 2576,
iast_GLOBAL (2.64 ms) : 2537, 2743
. : milestone, 2640,
profiling (2.442 ms) : 2354, 2531
. : milestone, 2442,
tracing (2.367 ms) : 2284, 2450
. : milestone, 2367,
section candidate
no_agent (1.486 ms) : 1474, 1498
. : milestone, 1486,
appsec (4.206 ms) : 3951, 4461
. : milestone, 4206,
iast (2.588 ms) : 2486, 2690
. : milestone, 2588,
iast_GLOBAL (2.634 ms) : 2531, 2736
. : milestone, 2634,
profiling (2.448 ms) : 2359, 2536
. : milestone, 2448,
tracing (2.382 ms) : 2299, 2466
. : milestone, 2382,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~5d4bd10805, baseline=1.59.0-SNAPSHOT~372ceb0ed6
dateFormat X
axisFormat %s
section baseline
no_agent (15.232 s) : 15232000, 15232000
. : milestone, 15232000,
appsec (14.523 s) : 14523000, 14523000
. : milestone, 14523000,
iast (17.391 s) : 17391000, 17391000
. : milestone, 17391000,
iast_GLOBAL (17.047 s) : 17047000, 17047000
. : milestone, 17047000,
profiling (14.542 s) : 14542000, 14542000
. : milestone, 14542000,
tracing (14.127 s) : 14127000, 14127000
. : milestone, 14127000,
section candidate
no_agent (14.576 s) : 14576000, 14576000
. : milestone, 14576000,
appsec (14.21 s) : 14210000, 14210000
. : milestone, 14210000,
iast (17.753 s) : 17753000, 17753000
. : milestone, 17753000,
iast_GLOBAL (16.683 s) : 16683000, 16683000
. : milestone, 16683000,
profiling (14.106 s) : 14106000, 14106000
. : milestone, 14106000,
tracing (14.087 s) : 14087000, 14087000
. : milestone, 14087000,
|
| // Pattern to match Kafka Connect schema-derived suffixes like _messagebody_0, _text_0, _bytes_0 | ||
| // These suffixes are added by Kafka Connect converters when handling union/optional fields | ||
| private static final java.util.regex.Pattern KAFKA_CONNECT_SCHEMA_SUFFIX_PATTERN = | ||
| java.util.regex.Pattern.compile("_(?:messagebody|text|bytes|map|value)_\\d+$", java.util.regex.Pattern.CASE_INSENSITIVE); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 suggestion: Pattern usage is not recommended for performance reason.
It seems the pattern you're looking for would be easy to find as the last characters would be a number.
I would implement the clean up using String.lastIndexOf() and String.substring() instead of Pattern, Matcher and String.replaceFirst() which also use a pattern internally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, updated it to no longer use a regex pattern.
…mance Address review feedback to avoid Pattern/Matcher usage for performance reasons. Use lastIndexOf() and regionMatches() instead of regex to strip Kafka Connect schema-derived suffixes from queue names. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
Summary
When using Kafka Connect with IBM MQ connectors, DSM was reporting incorrect queue names with schema-derived suffixes like
_messagebody_0. For example:trainmgt.dispatch.trnsheet.p30.v1.pub_messagebody_0trainmgt.dispatch.trnsheet.p30.v1.pubHow IBM MQ and Kafka Connect are linked
Kafka Connect IBM MQ connectors (from Confluent or IBM) bridge IBM MQ and Kafka:
The connector internally uses JMS to communicate with IBM MQ. When dd-trace-java is attached to the Kafka Connect worker JVM, it instruments these JMS calls and creates DSM checkpoints using
Queue.getQueueName().Hypothesis
When the connector creates JMS Queue objects, somehow the queue name returned by
getQueueName()includes schema-derived suffixes. This could happen if:_0suffixes to union/optional fields likemessageBody, and this leaks into the destination nameWhy pure Java apps work
Pure Java apps create queues directly:
With Kafka Connect, the connector creates the Queue internally, and something in that process adds the suffix.
Changes
Added sanitization in
JMSDecorator.getDestinationName()to strip these suffixes:_messagebody_N_text_N_bytes_N_map_N_value_NUses a string-based approach with
lastIndexOf()andregionMatches()for performance (avoids Pattern/Matcher overhead).Test plan
JMSDecoratorTest.groovycovering:References
🤖 Generated with Claude Code