test: Mock Claude models to improve LLM test reliability

[shuoweil]

Mock Claude models to improve LLM test reliability
b/436340035

[GarrettWu]

what was the root cause of the failure?

[shuoweil]

what was the root cause of the failure?

Hi @GarrettWu , the Claude model endpoint is only available in the us-east5 region, not in us-central1.
https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/claude/sonnet-3-5
https://screenshot.googleplex.com/34kotytZWcQhgmd

[GarrettWu]

what was the root cause of the failure?

Hi @GarrettWu , the Claude model endpoint is only available in the us-east5 region, not in us-central1. https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/claude/sonnet-3-5 https://screenshot.googleplex.com/34kotytZWcQhgmd

Does it mean BQML US region can't use the model? Could you check with jasperxu@? If so, we'd deprecate the model and remove the tests. Mocking it doesn't really testing anything.

[shuoweil]

us-east5 region,

You are right, mock does not anything. I can use a session in us-east5 region, it also limits our testing. I have talked to Jasper before I work on this PR, we can use claude in us-east5. Let's chat offline tomorrow.