Skip to content

Conversation

@hassiebp
Copy link
Contributor

@hassiebp hassiebp commented Jan 15, 2025

Important

Add debug level to spans tagged with langsmith:hidden in Langfuse Langchain integration and update dependencies.

  • Behavior:
    • Add level: "DEBUG" to spans with langsmith:hidden tag in on_chain_start(), on_tool_start(), on_retriever_start() in langchain.py.
  • Constants:
    • Define LANGSMITH_TAG_HIDDEN as "langsmith:hidden" in langchain.py.
  • Dependencies:
    • Add langgraph dependency to pyproject.toml.
  • Tests:
    • Add test_langgraph() in test_langchain.py to verify spans with langsmith:hidden tag are set to DEBUG level.

This description was created by Ellipsis for ded0985. It will automatically update as commits are pushed.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disclaimer: Experimental PR review

PR Summary

This PR adds support for debug level spans tagged with 'langsmith:hidden' in the Langchain callback handler, with corresponding test coverage.

  • Added langgraph v0.2.62 as dev dependency but needs proper version constraints and naming convention alignment with other langchain packages
  • Modified callback handler methods to set level='DEBUG' when spans are tagged with LANGSMITH_TAG_HIDDEN
  • Added new test using LangGraph to verify debug level behavior for hidden spans
  • Implemented consistent tag checking and debug level setting across chain, tool and retriever start methods
  • Consider moving langgraph from dev dependencies to main dependencies if it's required for core functionality

3 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disclaimer: Experimental PR review

PR Summary

(updates since last review)

This PR adds debug level functionality for spans tagged with 'langsmith:hidden' in the Langfuse Langchain integration. Here's a concise summary of the latest changes:

  • Fixed langgraph dependency placement in pyproject.toml by properly indenting under [tool.poetry.group.dev.dependencies]
  • Added LANGSMITH_TAG_HIDDEN constant and debug level setting in langchain.py for tagged spans
  • Implemented test coverage in test_langchain.py to verify debug level behavior
  • Ensured consistent debug level handling across chain, tool and retriever start methods

The implementation looks solid with good error handling and type safety. The main focus should be on properly organizing the dependencies and ensuring consistent behavior across different span types.

3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disclaimer: Experimental PR review

PR Summary

(updates since last review)

Based on the provided information, I'll summarize the most recent changes that haven't been covered in previous reviews:

Added null check safety for debug level tagging in Langchain integration, focusing on defensive programming and error prevention.

  • Added explicit null check for tags parameter before checking for LANGSMITH_TAG_HIDDEN in on_chain_start, on_tool_start, and on_retriever_start methods
  • Implemented consistent debug level logic: level="DEBUG" if tags and LANGSMITH_TAG_HIDDEN in tags else None
  • Enhanced runtime safety by preventing potential errors when tags parameter is None

The changes focus on improving code robustness while maintaining the core functionality of debug level tagging for langsmith hidden spans.

2 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile

@hassiebp hassiebp enabled auto-merge (squash) January 15, 2025 10:56
@hassiebp hassiebp disabled auto-merge January 15, 2025 10:56
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disclaimer: Experimental PR review

PR Summary

(updates since last review)

Based on the provided information, I'll summarize the latest changes that haven't been covered in previous reviews:

The PR adds debug level logging for spans tagged with 'langsmith:hidden' in the Langchain integration. Here are the key points:

  • Fixed inconsistency in on_retriever_start() method where debug level setting differs between parent and child spans
  • Ensured uniform debug level handling across all span types (chain, tool, retriever) for better consistency
  • Added proper type hints and docstrings for the new LANGSMITH_TAG_HIDDEN constant
  • Improved test coverage in test_langchain.py to verify debug level behavior for retriever spans specifically
  • Maintained backward compatibility while adding the new debug level functionality

The changes focus on standardizing the debug level implementation across different span types while ensuring proper error handling and type safety.

1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disclaimer: Experimental PR review

PR Summary

(updates since last review)

Based on the latest changes shown in the files and previous reviews, here's a concise summary of the new modifications:

  • Added small sleep delays (0.1s) in tests/test_core_sdk.py to improve test reliability for trace updates
  • Removed obsolete assertion in tests/test_datasets.py related to LlamaIndex query name validation
  • Fixed test stability issues by adding proper timing controls between sequential operations

These changes focus on test reliability improvements and are separate from the main debug level functionality changes covered in previous reviews. The modifications are minimal and targeted at ensuring consistent test behavior rather than adding new features.

Note: I've kept this summary focused only on the latest changes that weren't covered in previous reviews, specifically around test improvements and reliability fixes.

2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

@hassiebp hassiebp merged commit 6c74dc3 into main Jan 15, 2025
5 of 9 checks passed
@hassiebp hassiebp deleted the langchain-hidden-tag branch January 15, 2025 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apply debug level to langsmith:hidden observations of the langchain integration

2 participants