Skip to content

Conversation

@github-actions
Copy link

@github-actions github-actions bot commented Dec 30, 2025

Motivation

Users need the ability to retry model calls on arbitrary exceptions beyond just ModelThrottledException, and also retry based on response validation. Issue strands-agents#370 requests retrying on ServiceUnavailableException (503 errors), and this feature enables that and more by letting hooks implement custom retry logic for both exceptions and successful responses.

Resolves #9

Public API Changes

New Field: AfterModelCallEvent.retry_model

Hook providers can now set retry_model=True to retry model invocations on both exceptions and successful calls:

# Example 1: Retry on exceptions
class RetryOnServiceUnavailable(HookProvider):
    def __init__(self, max_retries=3):
        self.max_retries = max_retries
        self.retry_counts = {}
    
    def register_hooks(self, registry):
        registry.add_callback(AfterModelCallEvent, self.handle_retry)
    
    async def handle_retry(self, event):
        if event.exception and "ServiceUnavailable" in str(event.exception):
            request_id = id(event)
            count = self.retry_counts.get(request_id, 0)
            
            if count < self.max_retries:
                self.retry_counts[request_id] = count + 1
                await asyncio.sleep(2 ** count)  # Exponential backoff
                event.retry_model = True

# Example 2: Retry on successful calls based on response validation
class MinimumResponseLengthHook(HookProvider):
    def __init__(self, min_length=50):
        self.min_length = min_length
        self.retry_count = 0
        self.max_retries = 2
    
    def register_hooks(self, registry):
        registry.add_callback(AfterModelCallEvent, self.handle_after_model_call)
    
    async def handle_after_model_call(self, event):
        if event.stop_response:
            text = "".join(b.get("text", "") for b in event.stop_response.message.get("content", []))
            if len(text) < self.min_length and self.retry_count < self.max_retries:
                self.retry_count += 1
                event.retry_model = True

The retry_model field is writable within hook callbacks and defaults to False. It can be set for both successful calls (to validate response content) and failed calls (to retry exceptions).

Use Cases

  • Custom Exception Retry: Retry on ServiceUnavailableException (503), rate limit errors, or any application-specific exceptions
  • Response Validation: Retry if response doesn't meet quality criteria (length, format, content)
  • Flexible Retry Logic: Hooks control retry count, delay strategy (exponential backoff, jitter), and conditions
  • Request-Specific Retry: Different retry strategies based on request context, exception type, or response content

Implementation Notes

Hook Retry Integration

Hook-initiated retries work alongside the existing ModelThrottledException retry mechanism:

  • Hook retries are checked on both successful and failed model calls
  • If hooks set retry_model=True, the retry loop continues
  • Throttle retries handle ModelThrottledException as before
  • Hooks control their own retry limits without framework-enforced constraints

Reverse Callback Ordering

AfterModelCallEvent uses reverse callback ordering (cleanup pattern). When multiple hooks modify retry_model, the first-registered hook's value wins because it's called last.

No Framework-Enforced Limits

The framework doesn't enforce retry count limits or delays for hook-initiated retries - hooks manage their own state and logic. This provides maximum flexibility while keeping the framework simple.

zastrowm and others added 2 commits December 30, 2025 13:55
Add retry_model field to AfterModelCallEvent that enables hook providers
to retry model invocations when exceptions occur. This provides flexibility
for users to implement custom retry logic for any exception type.

- Add retry_model: bool field to AfterModelCallEvent (default False)
- Implement _can_write() to allow hooks to modify retry_model
- Update _handle_model_execution() to check retry_model and retry when set
- Hook retries integrate with existing ModelThrottledException retry logic
- Hook retries respect MAX_ATTEMPTS limit
- Add comprehensive test coverage for retry scenarios

Resolves #9
@zastrowm
Copy link
Owner

/strands

Hooks should be able to retry without framework-enforced limits.
Removed the attempt count check that artificially limited hook retries.
Hooks now control their own retry logic within the loop iterations.
await agent.hooks.invoke_callbacks_async(after_model_call_event)

# Check if hooks want to retry the model call
if after_model_call_event.retry_model and after_model_call_event.exception:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go through the exercise of adding support for retrying the the model call even when there aren't exceptions. Add appropriate plumbing and test cases for that scenario

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added support for retrying on successful calls with test demonstrating content-based retry.

@zastrowm
Copy link
Owner

/strands implement

Extend retry_model functionality to work on both successful and failed
model calls, enabling hooks to retry based on response content validation
or other custom criteria.

- Remove exception requirement from retry_model check
- Add retry check on successful model calls
- Add test_hook_retry_on_successful_call demonstrating content-based retry
- Update test_hook_retry_ignored_without_exception -> test_hook_retry_not_set_on_success
- Update AfterModelCallEvent docstring to reflect retry on success
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Allow hooks to retry model invocations

3 participants