Skip to content

Conversation

@cdepeuter
Copy link
Contributor

@cdepeuter cdepeuter commented Dec 3, 2025

Add optional timestamp parameter to score creation methods

This PR adds the ability to pass a custom timestamp parameter when creating scores, allowing users to backfill scores with historical timestamps or set specific evaluation times.

Motivation

Currently, scores are always created with the current UTC timestamp via _get_timestamp(). However, there are legitimate use cases where users need to specify a custom timestamp:

  • Backfilling historical evaluations - When importing scores from external systems or databases with existing timestamps
  • Batch evaluation workflows - When running evaluations asynchronously and wanting to preserve the original evaluation time rather than the ingestion time
  • Testing and development - When creating reproducible test scenarios with specific timestamps

Changes

This PR adds an optional timestamp: Optional[datetime] = None parameter to:

  • Langfuse.create_score() - Core score creation method in client.py
  • LangfuseSpan.score() - Convenience method for scoring a specific span
  • LangfuseSpan.score_trace() - Convenience method for scoring an entire trace

The implementation follows existing conventions in the codebase (similar to completion_start_time parameters) and uses the pattern timestamp or _get_timestamp() to maintain backward compatibility.

Example Usage

from datetime import datetime, timezone, timedelta

# Default behavior (unchanged)
langfuse.create_score(
    name="accuracy",
    value=0.92,
    trace_id="abcdef1234567890abcdef1234567890"
)

# New: Backfill a score with a historical timestamp
historical_time = datetime(2024, 1, 15, 10, 30, 0, tzinfo=timezone.utc)
langfuse.create_score(
    name="accuracy",
    value=0.92,
    trace_id="abcdef1234567890abcdef1234567890",
    timestamp=historical_time
)

# New: Use with batch evaluations
evaluation_time = datetime.now(timezone.utc) - timedelta(hours=1)
span.score(
    name="relevance",
    value=0.85,
    timestamp=evaluation_time
)

Important

Add optional timestamp parameter to score creation methods in client.py and span.py, allowing custom timestamps for scores.

  • Behavior:
    • Add optional timestamp parameter to Langfuse.create_score(), LangfuseSpan.score(), and LangfuseSpan.score_trace() to allow custom timestamps for scores.
    • Default behavior remains unchanged, using current UTC time if timestamp is not provided.
  • Implementation:
    • In client.py, create_score() method updated to accept timestamp and use timestamp or _get_timestamp().
    • In span.py, score() and score_trace() methods updated to accept timestamp.
  • Testing:
    • Add test_create_score_with_custom_timestamp() in test_core_sdk.py to verify custom timestamp functionality.

This description was created by Ellipsis for e158f0e. You can customize this summary. It will automatically update as commits are pushed.

Disclaimer: Experimental PR review

Greptile Overview

Greptile Summary

This PR adds an optional timestamp parameter to score creation methods, enabling users to backfill historical scores or set specific evaluation times.

  • Added timestamp: Optional[datetime] = None to Langfuse.create_score(), LangfuseSpan.score(), and LangfuseSpan.score_trace() methods
  • Uses timestamp or _get_timestamp() pattern for backward compatibility
  • Added comprehensive test coverage for the new functionality

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk - simple parameter addition with proper defaults.
  • Score reflects straightforward changes following existing patterns, proper backward compatibility via default values, and comprehensive test coverage.
  • No files require special attention.

Important Files Changed

File Analysis

Filename Score Overview
langfuse/_client/client.py 5/5 Added optional timestamp parameter to create_score() method and overloads, with proper fallback to _get_timestamp().
langfuse/_client/span.py 5/5 Added optional timestamp parameter to score() and score_trace() methods and overloads, passing through to create_score().
tests/test_core_sdk.py 4/5 Added test for custom timestamp functionality; contains inline import that should be moved to module level.

Sequence Diagram

sequenceDiagram
    participant User
    participant Langfuse as Langfuse Client
    participant Span as LangfuseSpan
    participant ResourceMgr as Resource Manager

    User->>Langfuse: create_score(name, value, timestamp)
    Langfuse->>Langfuse: timestamp or _get_timestamp()
    Langfuse->>Langfuse: Build ScoreBody event
    Langfuse->>ResourceMgr: add_score_task(event)
    ResourceMgr-->>User: Score created

    Note over User,ResourceMgr: Alternative: Score via Span

    User->>Span: span.score(name, value, timestamp)
    Span->>Langfuse: create_score(trace_id, observation_id, timestamp)
    Langfuse->>ResourceMgr: add_score_task(event)
    ResourceMgr-->>User: Score created
Loading

@CLAassistant
Copy link

CLAassistant commented Dec 3, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@cdepeuter cdepeuter force-pushed the create-score-timestamp branch from 10a2bd0 to e655cf8 Compare December 3, 2025 00:29
@hassiebp hassiebp self-requested a review December 3, 2025 09:52
@hassiebp
Copy link
Contributor

hassiebp commented Dec 3, 2025

Thanks a lot for your contribution, @cdepeuter !

@hassiebp hassiebp changed the base branch from main to add-score-timestamps December 3, 2025 09:57
@hassiebp hassiebp merged commit b29756c into langfuse:add-score-timestamps Dec 3, 2025
6 of 10 checks passed
hassiebp added a commit that referenced this pull request Dec 3, 2025
feat(client): Add optional timestamp parameter to score creation methods (#1463)

* add optional param to configure timestamp when scoring

* move inline imports to top

* alphabetic sorting of imports in case that matters

Co-authored-by: Conrad <conrad.depeuter@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants