-
Notifications
You must be signed in to change notification settings - Fork 0
feat: allow hooks to retry model invocations on exceptions #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Add retry_model field to AfterModelCallEvent that enables hook providers to retry model invocations when exceptions occur. This provides flexibility for users to implement custom retry logic for any exception type. - Add retry_model: bool field to AfterModelCallEvent (default False) - Implement _can_write() to allow hooks to modify retry_model - Update _handle_model_execution() to check retry_model and retry when set - Hook retries integrate with existing ModelThrottledException retry logic - Hook retries respect MAX_ATTEMPTS limit - Add comprehensive test coverage for retry scenarios Resolves #9
|
/strands |
Hooks should be able to retry without framework-enforced limits. Removed the attempt count check that artificially limited hook retries. Hooks now control their own retry logic within the loop iterations.
src/strands/event_loop/event_loop.py
Outdated
| await agent.hooks.invoke_callbacks_async(after_model_call_event) | ||
|
|
||
| # Check if hooks want to retry the model call | ||
| if after_model_call_event.retry_model and after_model_call_event.exception: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go through the exercise of adding support for retrying the the model call even when there aren't exceptions. Add appropriate plumbing and test cases for that scenario
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added support for retrying on successful calls with test demonstrating content-based retry.
|
/strands implement |
Extend retry_model functionality to work on both successful and failed model calls, enabling hooks to retry based on response content validation or other custom criteria. - Remove exception requirement from retry_model check - Add retry check on successful model calls - Add test_hook_retry_on_successful_call demonstrating content-based retry - Update test_hook_retry_ignored_without_exception -> test_hook_retry_not_set_on_success - Update AfterModelCallEvent docstring to reflect retry on success
Motivation
Users need the ability to retry model calls on arbitrary exceptions beyond just
ModelThrottledException, and also retry based on response validation. Issue strands-agents#370 requests retrying on ServiceUnavailableException (503 errors), and this feature enables that and more by letting hooks implement custom retry logic for both exceptions and successful responses.Resolves #9
Public API Changes
New Field:
AfterModelCallEvent.retry_modelHook providers can now set
retry_model=Trueto retry model invocations on both exceptions and successful calls:The
retry_modelfield is writable within hook callbacks and defaults toFalse. It can be set for both successful calls (to validate response content) and failed calls (to retry exceptions).Use Cases
Implementation Notes
Hook Retry Integration
Hook-initiated retries work alongside the existing
ModelThrottledExceptionretry mechanism:retry_model=True, the retry loop continuesModelThrottledExceptionas beforeReverse Callback Ordering
AfterModelCallEventuses reverse callback ordering (cleanup pattern). When multiple hooks modifyretry_model, the first-registered hook's value wins because it's called last.No Framework-Enforced Limits
The framework doesn't enforce retry count limits or delays for hook-initiated retries - hooks manage their own state and logic. This provides maximum flexibility while keeping the framework simple.