feat(cost-tracking): trace all usage keys #1021
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Disclaimer: Experimental PR review
PR Summary
This PR enhances cost tracking capabilities across the Langfuse Python SDK by introducing more granular usage and cost monitoring features.
- Introduces new
usage_detailsandcost_detailsfields in key classes while deprecating olderusagefield for more detailed token and cost tracking - Adds
OpenAiUsageSchemafor structured token usage tracking with prompt/completion token details - Increases comment content character limit from 500 to 3000 in
CreateCommentRequest - Updates LangChain and LlamaIndex handlers to support granular token usage tracking from various LLM providers
- Maintains backward compatibility for V2 self-hosters while introducing new cost tracking features
20 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile
| APPLICATION_XML = "application/xml" | ||
| APPLICATION_OCTET_STREAM = "application/octet-stream" | ||
|
|
||
| def visit( |
There was a problem hiding this comment.
logic: visit() method lacks exhaustive check or default case. Consider adding a default handler or raising NotImplementedError for unhandled cases to prevent silent failures if new enum values are added.
There was a problem hiding this comment.
Disclaimer: Experimental PR review
PR Summary
(updates since last review)
This PR continues to enhance the cost tracking implementation with additional changes focused on the OpenAI integration and usage tracking. Here's a summary of the latest changes:
- Added new
OpenAiUsageSchemaclass in/ingestion/types/open_ai_usage_schema.pywith detailed token tracking fields for prompt and completion tokens - Updated OpenAI integration in
openai.pyto handle both old and new usage formats for backward compatibility - Enhanced error handling in OpenAI integration to include zero-value cost details for error cases
- Added proper type safety with enum usage for
MediaContentTypeinstead of string literals - Improved token parsing in LangChain handler to support various LLM provider formats (Anthropic, Bedrock, Vertex AI, IBM)
20 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile
| class OpenAiUsageSchema(pydantic_v1.BaseModel): | ||
| prompt_tokens: int | ||
| completion_tokens: int | ||
| total_tokens: int |
There was a problem hiding this comment.
logic: total_tokens should validate that it equals prompt_tokens + completion_tokens
| self._get_generation_client(event.span_id).update( | ||
| usage=usage, end_time=_get_timestamp() | ||
| usage=usage, usage_details=usage, end_time=_get_timestamp() | ||
| ) |
There was a problem hiding this comment.
logic: passing same 'usage' value to both usage and usage_details fields may cause issues with backward compatibility since usage field is deprecated
Important
Enhances cost-tracking by introducing detailed usage and cost tracking fields, updating OpenAI integration, and improving type safety with enum usage.
usage_detailsandcost_detailsfields inObservation,ObservationsView, andUsageclasses for detailed usage and cost tracking.usagefield in favor ofusage_detailsandcost_details.OpenAiUsageSchemaandUsageDetailsfor detailed usage tracking.MediaContentTypeto use an enum for better type safety.create_comment_request.py.langfuse/client.pyandlangfuse/decorators/langfuse_decorator.pyto support new usage tracking.langfuse/callback/langchain.pyandlangfuse/llama_index/_event_handler.pyto utilize new usage details.This description was created by
for 54fac8e. It will automatically update as commits are pushed.