fix(serving): add async client support for OpenAI integrations #1175
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add http_async_client to get_langchain_chat_open_ai_client() to fix 401 errors on async operations (ainvoke, astream) - fixes databricks#1173 - Add get_async_open_ai_client() method for native AsyncOpenAI - fixes databricks#847 - Extract BearerAuth to module-level _get_bearer_auth() for reuse - Refactor reserved param validation into shared helper methods 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Josh Wiley <josh@cloudbend.dev>
Signed-off-by: Josh Wiley <josh@cloudbend.dev>
|
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
mr-brobot
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
WHAT:
http_async_clientparameter toget_langchain_chat_open_ai_client()so async operations use authenticated Databricks credentialsget_async_open_ai_client()method that returns a properly configuredAsyncOpenAIclientBearerAuthclass to module-level_get_bearer_auth()function for reuse across sync and async clients_check_reserved_openai_params()and_build_openai_client_params()helper methodsWHY:
The
get_langchain_chat_open_ai_client()method was only settinghttp_clientbut nothttp_async_client. This caused async operations likeainvoke()andastream()to use LangChain's default unauthenticated HTTP client, resulting in 401 errors against Databricks endpoints.Additionally, users wanting to use OpenAI's native
AsyncOpenAIclient had no equivalent toget_open_ai_client()for async workflows.The httpx
Authclass with the generator-basedauth_flow()pattern works identically for both sync and async operations, so the existingBearerAuthimplementation was extracted and reused for both client types.Fixes #1173
Fixes #847
How is this tested?
Unit tests added/updated in
tests/test_open_ai_mixin.py:test_langchain_open_ai_client- extended to verify bothhttp_client.authandhttp_async_client.authare configured