Skip to content

Conversation

@akshaylive
Copy link
Collaborator

@akshaylive akshaylive commented Jan 7, 2026

Initial implementation of context precision and faithfulness evaluators.

Development Package

  • Use uipath pack --nolock to get the latest dev build from this PR (requires version range).
  • Add this package as a dependency in your pyproject.toml:
[project]
dependencies = [
  # Exact version:
  "uipath==2.4.7.dev1010703663",

  # Any version from PR
  "uipath>=2.4.7.dev1010700000,<2.4.7.dev1010710000"
]

[[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
publish-url = "https://test.pypi.org/legacy/"
explicit = true

[tool.uv.sources]
uipath = { index = "testpypi" }

[tool.uv]
override-dependencies = [
    "uipath>=2.4.7.dev1010700000,<2.4.7.dev1010710000",
]

@github-actions github-actions bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Jan 7, 2026
@akshaylive akshaylive added the build:dev Create a dev build from the pr label Jan 7, 2026
@akshaylive akshaylive force-pushed the akshaya/context_precision branch 13 times, most recently from 563c0d0 to 00b6607 Compare January 8, 2026 21:11
@akshaylive akshaylive changed the title WIP feat(ContextPrecisionEvaluator): context precision evaluator feat(ContextPrecisionEvaluator): context precision evaluator Jan 8, 2026
@akshaylive akshaylive force-pushed the akshaya/context_precision branch from 00b6607 to 67390d3 Compare January 8, 2026 21:18
@akshaylive akshaylive changed the title feat(ContextPrecisionEvaluator): context precision evaluator feat(NewEvaluators): context precision and faithfulness Jan 8, 2026
@akshaylive akshaylive force-pushed the akshaya/context_precision branch from 67390d3 to 2aa6952 Compare January 8, 2026 21:28
@akshaylive
Copy link
Collaborator Author

Note: these evaluators seem to yield higher scores compared to production. We will debug in a follow-up PR.

Initial implementation of context precision and faithfulness evaluators.
@akshaylive akshaylive force-pushed the akshaya/context_precision branch from 2aa6952 to 3d845a5 Compare January 8, 2026 22:44
@akshaylive akshaylive enabled auto-merge (squash) January 8, 2026 22:44
@akshaylive akshaylive merged commit 17aa7e1 into main Jan 8, 2026
49 checks passed
@akshaylive akshaylive deleted the akshaya/context_precision branch January 8, 2026 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build:dev Create a dev build from the pr test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants