-
Notifications
You must be signed in to change notification settings - Fork 874
Description
Is your feature request related to a problem? Please describe.
Running FastAPI‑MCP’s Streamable HTTP transport behind reverse proxies (e.g., Envoy/Contour) results in long‑lived streams being dropped during quiet periods due to proxy stream idle timeouts (often ~5 minutes). In many managed platforms, changing global proxy settings is not possible. Even with per‑route ingress timeouts tuned (e.g., response: infinity and a large idle), if the app emits no bytes for an extended period, the connection can still be closed as “idle.” The current Streamable HTTP path does not emit periodic keepalive bytes, so legitimate idle periods cause client disconnects and break MCP workflows.
Describe the solution you'd like
Two complementary enhancements:
- Configurable heartbeats for Streamable HTTP
- Add an opt‑in heartbeat mechanism that emits harmless bytes when no data has been sent for a configurable interval.
- Configuration:
enabled: bool (defaultfalseto preserve current behavior)interval_seconds: int (e.g., default 25)payload: bytes (e.g., defaultb":hb\n"or a single newline)
- Behavior: only emit heartbeats if no “real” data has been sent within the interval; never delay or coalesce normal frames.
- Public stream wrapper hook
- Expose a documented hook to wrap the transport’s async iterator or streaming response so users can inject behaviors (heartbeats, logging, tracing, framing tweaks) without forking.
- Example API:
mount_http(stream_wrapper: Callable[[AsyncIterator[bytes]], AsyncIterator[bytes]] = None)
This keeps transport internals encapsulated while enabling advanced customization for real‑world deployments.
Describe alternatives you've considered
- Per‑route ingress timeouts: Setting
response: infinityand a largeidlehelps but does not prevent idle closures if no bytes are sent for long stretches. - SSE transport: Viable, but Streamable HTTP is preferred for current MCP transport guidance and client compatibility.
- Emitting synthetic domain events: Not always acceptable; many streams are legitimately quiet and should not send fake business events.
- Local wrapper route: Implementable via ASGI re‑dispatch and a
StreamingResponsewrapper that injects heartbeats, but this is brittle and depends on internal behavior. A first‑class hook and built‑in heartbeat would be cleaner and more maintainable.
Additional context
- Environment constraints: Cannot modify platform‑level Envoy/Contour configuration; control is limited to application code and per‑route ingress config.
- Current mitigations: Route timeouts configured to
response: infinityand a largeidle; still observe disconnects during quiet periods without periodic bytes. - Proposed defaults: Heartbeat disabled by default. When enabled, default interval 25s and payload
b":hb\n"(harmless for generic stream readers and SSE‑style consumers). - Documentation request: Please document proxy considerations and recommend tuning heartbeat interval to be comfortably below typical idle timeouts. Clarify interaction with compression/buffering (advise disabling non‑streaming compression on this route or using streaming‑capable compression).