Skip to content

feat(prof): use a trampoline for FLF functions to intercept timings#3595

Open
bwoebi wants to merge 16 commits intomasterfrom
bob/prof-flf-test
Open

feat(prof): use a trampoline for FLF functions to intercept timings#3595
bwoebi wants to merge 16 commits intomasterfrom
bob/prof-flf-test

Conversation

@bwoebi
Copy link
Collaborator

@bwoebi bwoebi commented Jan 21, 2026

Frameless functions (FLF) do not check the EG(vm_interrupt) flag like it would happen usually on internal function returns. This means that from the point of view of the profiler, those functions do not exist, as the engine never checks the interrupt flag and as such never calls the profilers interrupt handler.

This PR adds a trampoline for aarch64 and x84_64 to all known frameless functions and a tail call to check the EG(vm_interrupt) flag and in case it is raised, call the interrupt handler.

One (acceptable) trade-off is that using these trampolines, libunwind in the crash tracker is not able to unwind passed the trampoline. Anyway, the impact is minimal:

  • Stack frames from the crash site up to the trampoline are still captured
  • Only frames above the trampoline (from caller to main) are lost
  • For most crashes, the relevant context is near the crash site, not at the program entry point

The profiler itself is unaffected of this, as we are unwinding the VM stack, following the execute_data linked list.

https://datadoghq.atlassian.net/browse/PROF-12085

@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 1976c3c to 222e3a4 Compare January 21, 2026 17:19
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Jan 21, 2026

⚠️ Tests

Fix all issues with Cursor

⚠️ Warnings

🧪 1026 Tests failed

    testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Fix with Cursor)

testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Datadog) (Fix with Cursor)
Risky Test
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:60
testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 699350ef00000000f0161d2ef392ff64
tid: 699350ef00000000
hexProcessTraceId: f0161d2ef392ff64
hexProcessSpanId: 8ac716aa24c57361
processTraceId: 17300047106082537316
processSpanId: 9999986417616647009
View all

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0dd28b4 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@codecov-commenter
Copy link

codecov-commenter commented Jan 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.11%. Comparing base (72a021a) to head (0dd28b4).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3595      +/-   ##
==========================================
- Coverage   62.21%   62.11%   -0.11%     
==========================================
  Files         141      141              
  Lines       13387    13387              
  Branches     1753     1753              
==========================================
- Hits         8329     8315      -14     
- Misses       4260     4273      +13     
- Partials      798      799       +1     

see 3 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 72a021a...0dd28b4. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Collaborator

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this can run afoul of permissions somehow, since we're making executable code at runtime? I'm not well versed here, and yes, if the JIT is enabled you'd have to have that capability anyway, but I'm trying to understand implications of this WIP.

@pr-commenter
Copy link

pr-commenter bot commented Jan 21, 2026

Benchmarks [ profiler ]

Benchmark execution time: 2026-02-16 17:25:03

Comparing candidate commit 0dd28b4 in PR branch bob/prof-flf-test with baseline commit 72a021a in branch master.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 26 metrics, 7 unstable metrics.

scenario:php-profiler-timeline-memory-control

  • 🟥 cpu_user_time [+31.545ms; +40.104ms] or [+5.189%; +6.597%]
  • 🟥 execution_time [+30.889ms; +34.650ms] or [+4.847%; +5.437%]

scenario:php-profiler-timeline-memory-with-profiler

  • 🟥 execution_time [+27.285ms; +40.639ms] or [+2.712%; +4.040%]

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 222e3a4 to 0eb96ee Compare January 21, 2026 17:52
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from dc825a7 to c6bdefc Compare January 21, 2026 18:06
@bwoebi
Copy link
Collaborator Author

bwoebi commented Jan 21, 2026

@morrisonlevi dynasmrt takes care of setting RX permissions after compiling the code. As long as you're not running this under a hardened runtime (like app store apps or android (?)), there's no fundamental problem. Also I currently call assembler.finalize().unwrap() (which takes care of settign RX) - proper usage should just handle the potential error here and abort.

@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 95f422a to 7471599 Compare January 21, 2026 18:17
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 7471599 to 4358ab8 Compare January 21, 2026 18:24
dynasm!(assembler
; mov rax, QWORD original as i64
; call rax
; mov rax, QWORD interrupt_addr as i64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume original is not returning anything, because you're writing over RAX.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct

#[cfg(target_arch = "aarch64")]
dynasm!(assembler
; mov x16, original as u64
; blr x16
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this overwrites x30/lr, aren't you gonna lose the original return location of the handler? so when br x16 returns, it goes back to calling interrupt_addr

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right. call on x86_64 pushes %eip to the stack, but blr doesn't.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed up, is it correct now?

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from a3c79dc to 078acef Compare January 21, 2026 19:34
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch 2 times, most recently from e4930fe to 2604b22 Compare January 21, 2026 19:35
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 2604b22 to 6307860 Compare January 21, 2026 19:38
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from a246940 to bcebfb0 Compare January 21, 2026 20:37
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from b0995bc to d9d3f43 Compare January 22, 2026 13:45
@realFlowControl realFlowControl changed the title WIP: use a trampoline for FLF functions to intercept timings feat(prof): use a trampoline for FLF functions to intercept timings Feb 16, 2026
@realFlowControl realFlowControl marked this pull request as ready for review February 16, 2026 17:17
@realFlowControl realFlowControl requested review from a team as code owners February 16, 2026 17:17
Copy link
Member

@realFlowControl realFlowControl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bwoebi this is awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Comments