Commit c6cd5c3
fix: legacy evaluation reporting with Strategy Pattern
This PR fixes legacy evaluation reporting to the backend that was returning
HTTP 400 errors and implements the Strategy Pattern for cleaner code separation.
## Changes
### Strategy Pattern Implementation
- Created `EvalReportingStrategy` Protocol defining the interface for evaluation
reporting strategies
- Implemented `LegacyEvalReportingStrategy` for legacy evaluations:
- Converts string IDs to deterministic GUIDs using uuid5
- Uses endpoints without /coded/ prefix
- Uses assertionRuns format with assertionSnapshot
- Implemented `CodedEvalReportingStrategy` for coded evaluations:
- Keeps IDs as strings
- Uses /coded/ endpoint prefix
- Uses evaluatorRuns format with evaluationCriterias
### Bug Fixes
- Fixed legacy eval API payload structure for backend compatibility
- Added type assertion for project_id to fix mypy errors
- Removed unused ABC, abstractmethod imports after Protocol migration
### Test Results
- All 27 unit tests passing
- All linting checks (ruff, mypy) passing
- Integration testing with calculator sample: all API calls returning HTTP 200 OK
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1 parent d546ef4 commit c6cd5c3
File tree
7 files changed
+844
-805
lines changed- samples/calculator/evaluations
- eval-sets
- evaluators
- src/uipath/_cli/_evals
- tests/cli/eval
7 files changed
+844
-805
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | | - | |
| 48 | + | |
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
53 | | - | |
| 53 | + | |
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
| 2 | + | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
| 2 | + | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
| 2 | + | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
| 2 | + | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
0 commit comments