Skip to content

Conversation

@rahulgurnani
Copy link
Contributor

@rahulgurnani rahulgurnani commented Dec 4, 2025

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR implements PrepareDataPlugin hook for prefix cache match plugin. The goal is to split the prefix cache match plugin into match plugin and scorer. So, the PR adds a new scorer plugin to be used alongside the prepare data prefix cache match plugin for scoring.
This is an intermediate step towards migration of the plugin.

Did some initial scale testing using vllm:

@kaushikmitr / @BenjaminBraunDev FYI

Which issue(s) this PR fixes:

Fixes #1924

Does this PR introduce a user-facing change?:

yes. this PR adds a new feature flag "prepareDataPlugins"

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. labels Dec 4, 2025
@netlify
Copy link

netlify bot commented Dec 4, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 5752e58
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6933bc0ce5d58100089c557a
😎 Deploy Preview https://deploy-preview-1942--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rahulgurnani
Once this PR has been reviewed and has the lgtm label, please assign nirrozenbaum for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 4, 2025
@rahulgurnani rahulgurnani changed the title Implement PreapreDataPlugin for prefix cache match plugin Implement PrepareDataPlugin for prefix cache match plugin Dec 4, 2025
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 4, 2025
@rahulgurnani rahulgurnani requested a review from liu-cong December 6, 2025 05:16
@rahulgurnani rahulgurnani marked this pull request as ready for review December 6, 2025 05:16
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

migrate prefix cache scoring to prepare data plugin

3 participants