Skip to content

Conversation

@hassiebp
Copy link
Contributor

@hassiebp hassiebp commented May 6, 2025

Important

Adds parsing for Vertex AI token details in _parse_usage_model() to extract modality-specific token counts in langchain.py.

  • Behavior:
    • Adds parsing for prompt_tokens_details, candidates_tokens_details, and cache_tokens_details in _parse_usage_model() in langchain.py.
    • Extracts token_count for each modality and updates usage_model with keys input_modality_{modality}, output_modality_{modality}, and cached_modality_{modality}.

This description was created by Ellipsis for 1065560. You can customize this summary. It will automatically update as commits are pushed.

Greptile Summary

Disclaimer: Experimental PR review

Enhanced token usage parsing for Vertex AI responses in the Langfuse Python SDK, focusing on extracting and storing token counts for different modalities.

  • Added support for parsing input, output, and cached token details from Vertex AI usage models in langfuse/callback/langchain.py
  • Implemented modality-specific token count storage with descriptive keys (e.g., 'input_modality_X')
  • Ensured compatibility with existing OpenAI usage schema in langfuse/api/resources/ingestion/types/open_ai_usage.py
  • Cleaned up usage model by removing raw token details lists after processing

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

@hassiebp hassiebp enabled auto-merge (squash) May 6, 2025 12:49
@hassiebp hassiebp disabled auto-merge May 6, 2025 13:00
@hassiebp hassiebp merged commit c1b023e into main May 6, 2025
8 of 10 checks passed
@hassiebp hassiebp deleted the fix-vertexai-token-usage branch May 6, 2025 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: trace google vertex via langchain chain throws some errors bug(sdk-python): pydantic model fails validation on vertexai usage details

2 participants