test/integration: introduce robust harness and migrate BBR suite

[LukeAVanDrie]

What type of PR is this?
/kind cleanup

What this PR does / why we need it:

This PR overhauls the integration testing infrastructure to introduce a type-safe DSL, eliminate concurrency race conditions, and enable full test parallelism.

As a reference implementation for this new architecture, the Body-Based Routing (BBR) test suite has been fully migrated to the new harness. This lays the groundwork for an upcoming, larger migration of the Endpoint Picker (EPP) test suite.

Key Improvements

1. Testing DSL: Introduces high-level builders (e.g., ReqLLM, ReqRaw) and assertions (ExpectBBRHeader) to decouple test intent from gRPC protocol boilerplate. This significantly reduces code noise in table-driven tests.
2. Network Isolation & Parallelism: Introduces BBRHarness, which spins up a dedicated server instance on a random loopback port for every single test case.
   • Replaces brittle localhost bindings with strict 127.0.0.1 to prevent IPv4/IPv6 race conditions in CI.
   • Enables t.Parallel() for all BBR tests, reducing execution time.
3. Fixes gRPC Race Condition: Refactors the shared StreamedRequest utility to use a serial read loop. The previous implementation spawned concurrent goroutines to read from the same gRPC stream, leading to undefined behavior that masked server response errors.

Impact on Existing Tests

• BBR: Fully migrated to the new harness. Legacy setup code has been removed.
• EPP: The fix to StreamedRequest (removing the race condition) stabilized the test output, revealing an incorrect assertion in test/integration/epp/hermetic_test.go. The server correctly returns a 400 Bad Request without echoing the invalid body (a security best practice), but the legacy test expected the echo. The assertion has been updated to match the correct, stable server behavior.

Which issue(s) this PR fixes:
N/A -- groundwork for test/integration/epp/hermetic_test.go refactoring which is necessary to support Flow Control tests which require mutating pod state (for changing pool saturation).

Does this PR introduce a user-facing change?:

NONE

[k8s-ci-robot]

@LukeAVanDrie: The label(s) kind/test cannot be applied, because the repository doesn't have them.

In response to this:

What type of PR is this?
/kind test
/kind cleanup

What this PR does / why we need it:

This PR overhauls the integration testing infrastructure to introduce a type-safe DSL, eliminate concurrency race conditions, and enable full test parallelism.

As a reference implementation for this new architecture, the Body-Based Routing (BBR) test suite has been fully migrated to the new harness. This lays the groundwork for an upcoming, larger migration of the Endpoint Picker (EPP) test suite.

Key Improvements

1. Testing DSL: Introduces high-level builders (e.g., ReqLLM, ReqRaw) and assertions (ExpectBBRHeader) to decouple test intent from gRPC protocol boilerplate. This significantly reduces code noise in table-driven tests.
2. Network Isolation & Parallelism: Introduces BBRHarness, which spins up a dedicated server instance on a random loopback port for every single test case.

• Replaces brittle localhost bindings with strict 127.0.0.1 to prevent IPv4/IPv6 race conditions in CI.
• Enables t.Parallel() for all BBR tests, reducing execution time.

3. Fixes gRPC Race Condition: Refactors the shared StreamedRequest utility to use a serial read loop. The previous implementation spawned concurrent goroutines to read from the same gRPC stream, leading to undefined behavior that masked server response errors.

Impact on Existing Tests

• BBR: Fully migrated to the new harness. Legacy setup code has been removed.
• EPP: The fix to StreamedRequest (removing the race condition) stabilized the test output, revealing an incorrect assertion in test/integration/epp/hermetic_test.go. The server correctly returns a 400 Bad Request without echoing the invalid body (a security best practice), but the legacy test expected the echo. The assertion has been updated to match the correct, stable server behavior.

Which issue(s) this PR fixes:
N/A -- groundwork for test/integration/epp/hermetic_test.go refactoring which is necessary to support Flow Control tests which require mutating pod state (for changing pool saturation).

Does this PR introduce a user-facing change?:

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

Hi @LukeAVanDrie. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	69160f8
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6933942e65c08d00088f33f6
😎 Deploy Preview	https://deploy-preview-1959--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[ahg-g]

/ok-to-test

[ahg-g]

This is great.

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, LukeAVanDrie

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ahg-g]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[LukeAVanDrie]

@ahg-g

This is the EPP test case I changed. It looks like it is still non-deterministic. Locally, it never plumbs ": no healthy upstream" with the error, but in CI/CD it does. I need to take a deeper look into the new utils to see why this isn't fully resolved. Else, BBR tests are working as expected in CI.

Should hopefully be a small change I will patch Monday morning.

   --- FAIL: TestFullDuplexStreamed_KubeInferenceObjectiveRequest/invalid_json;_return_body (5.09s)
       hermetic_test.go:1028: Sending request: request_headers:{headers:{headers:{key:"hi" value:"mom"}}}
       hermetic_test.go:1028: Sending request: request_body:{body:"no healthy upstream" end_of_stream:true}
       hermetic_test.go:1028: Received response: immediate_response:{status:{code:BadRequest} body:"inference gateway: BadRequest - Error unmarshaling request body: no healthy upstream"}
       hermetic_test.go:1039: In test invalid json; return body, unexpected response, (-want +got):   []*ext_procv3.ProcessingResponse{
             	Inverse(protocmp.Transform, protocmp.Message{
             		"@type": s"envoy.service.ext_proc.v3.ProcessingResponse",
             		"immediate_response": protocmp.Message{
             			"@type": s"envoy.service.ext_proc.v3.ImmediateResponse",
             			"body": bytes.Join({
             				"inference gateway: BadRequest - Error unmarshaling request body",
           + 				": no healthy upstream",
             			}, ""),
             			"status": protocmp.Message{"@type": s"envoy.type.v3.HttpStatus", "code": s"BadRequest"},
             		},
             	}),
             }
FAIL

[LukeAVanDrie]

/retest

This is likely to fail again, but I am trying to isolate whether this is general non-determinism or an environment issue. It's also possible that it's the EPP code itself that's nondeterministic, not the test suite.

[k8s-ci-robot]

@LukeAVanDrie: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gateway-api-inference-extension-test-unit-main	69160f8	link	true	/test pull-gateway-api-inference-extension-test-unit-main

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[LukeAVanDrie]

I am trying to isolate whether this is general non-determinism or an environment issue. It's also possible that it's the EPP code itself that's nondeterministic, not the test suite.

Found it! It is non-determinism in the server code.

In server.go:

case *extProcPb.ProcessingRequest_RequestBody:
  loggerTrace.Info("Incoming body chunk", "EoS", v.RequestBody.EndOfStream)
  // In the stream case, we can receive multiple request bodies.
  body = append(body, v.RequestBody.Body...)
  
  // Message is buffered, we can read and decode.
  if v.RequestBody.EndOfStream {
  loggerTrace.Info("decoding")
  err = json.Unmarshal(body, &reqCtx.Request.Body)
  if err != nil {
	  if logger.V(logutil.DEBUG).Enabled() {
		  err = errutil.Error{Code: errutil.BadRequest, Msg: "Error unmarshaling request body: " + string(body)}
	  } else {
		  err = errutil.Error{Code: errutil.BadRequest, Msg: "Error unmarshaling request body"}
	  }
	  break
  }

...

The error message returned to the client depends on the logger verbosity level (logger.V(logutil.DEBUG).Enabled()).

My local run uses likely doesn't run with DEBUG level, so it returns the generic message. The CI environment likely runs tests with higher verbosity, enabling DEBUG. This causes the server to branch into the more detailed error path, echoing the body.

IMO, public API responses should not change based on internal logging flags. It could cause clients to break unpredictably and leaks implementation details.

Thoughts on making this change?

if v.RequestBody.EndOfStream {
  loggerTrace.Info("decoding")
  if errUnmarshal := json.Unmarshal(body, &reqCtx.Request.Body); errUnmarshal != nil {
	  // Log the details internally for the operator (if debug is enabled).
	  if logger.V(logutil.DEBUG).Enabled() {
		  logger.Info("Error unmarshaling request body", "body", string(body), "err", errUnmarshal)
	  }
  
	  // ALWAYS return the safe, standard error to the client.
	  err = errutil.Error{
                      Code: errutil.BadRequest, 
                      Msg: "Error unmarshaling request body", // No dynamic suffix
                  }
	  break
  }
...