Skip to content

Commit cef6368

Browse files
salma-elshafeySalma Elshafey
andauthored
[Hot Fix] Ensure proper handling of agent response reformatting in Tool Input Accuracy evaluator (#44066)
* modify the method used to reformat agent response in tool input acc eval * To retrigger build pipeline --------- Co-authored-by: Salma Elshafey <selshafey@microsoft.com>
1 parent d11ce89 commit cef6368

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_tool_input_accuracy/_tool_input_accuracy.py

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ErrorTarget,
1515
EvaluationException,
1616
)
17-
from ..._common.utils import reformat_conversation_history, _get_agent_response
17+
from ..._common.utils import reformat_conversation_history, reformat_agent_response
1818
from azure.ai.evaluation._common._experimental import experimental
1919

2020
logger = logging.getLogger(__name__)
@@ -25,11 +25,11 @@ class _ToolInputAccuracyEvaluator(PromptyEvaluatorBase[Union[str, float]]):
2525
"""The Tool Input Accuracy evaluator performs a strict binary evaluation (PASS/FAIL) of parameters
2626
passed to tool calls. It ensures that ALL parameters meet ALL criteria:
2727
28-
- Parameter grounding: All parameters must be derived from conversation history/query
29-
- Type compliance: All parameters must match exact types specified in tool definitions
30-
- Format compliance: All parameters must follow exact format and structure requirements
31-
- Completeness: All required parameters must be provided
32-
- No unexpected parameters: Only defined parameters are allowed
28+
- Parameter grounding: All parameters must be derived from conversation history/query.
29+
- Type compliance: All parameters must match exact types specified in tool definitions.
30+
- Format compliance: All parameters must follow exact format and structure requirements.
31+
- Completeness: All required parameters must be provided.
32+
- No unexpected parameters: Only defined parameters are allowed.
3333
3434
The evaluator uses strict binary evaluation:
3535
- 1: Only when ALL criteria are satisfied perfectly for ALL parameters
@@ -135,8 +135,8 @@ def _convert_kwargs_to_eval_input(self, **kwargs):
135135
if len(needed_tool_definitions) == 0:
136136
return {"error_message": self._NO_TOOL_DEFINITIONS_MESSAGE}
137137

138-
# Get agent response with tool calls and results using _get_agent_response
139-
agent_response_with_tools = _get_agent_response(response, include_tool_messages=True)
138+
# Reformat agent response with tool calls and results using reformat_agent_response
139+
agent_response_with_tools = reformat_agent_response(response, include_tool_messages=True)
140140

141141
return {
142142
"query": query,

0 commit comments

Comments
 (0)