Skip to content

Add configurable heartbeats and a public streaming hook for Streamable HTTP to prevent idle timeouts behind proxies #245

@SheGe

Description

@SheGe

Is your feature request related to a problem? Please describe.
Running FastAPI‑MCP’s Streamable HTTP transport behind reverse proxies (e.g., Envoy/Contour) results in long‑lived streams being dropped during quiet periods due to proxy stream idle timeouts (often ~5 minutes). In many managed platforms, changing global proxy settings is not possible. Even with per‑route ingress timeouts tuned (e.g., response: infinity and a large idle), if the app emits no bytes for an extended period, the connection can still be closed as “idle.” The current Streamable HTTP path does not emit periodic keepalive bytes, so legitimate idle periods cause client disconnects and break MCP workflows.

Describe the solution you'd like
Two complementary enhancements:

  1. Configurable heartbeats for Streamable HTTP
  • Add an opt‑in heartbeat mechanism that emits harmless bytes when no data has been sent for a configurable interval.
  • Configuration:
    • enabled: bool (default false to preserve current behavior)
    • interval_seconds: int (e.g., default 25)
    • payload: bytes (e.g., default b":hb\n" or a single newline)
  • Behavior: only emit heartbeats if no “real” data has been sent within the interval; never delay or coalesce normal frames.
  1. Public stream wrapper hook
  • Expose a documented hook to wrap the transport’s async iterator or streaming response so users can inject behaviors (heartbeats, logging, tracing, framing tweaks) without forking.
  • Example API:
    • mount_http(stream_wrapper: Callable[[AsyncIterator[bytes]], AsyncIterator[bytes]] = None)

This keeps transport internals encapsulated while enabling advanced customization for real‑world deployments.

Describe alternatives you've considered

  • Per‑route ingress timeouts: Setting response: infinity and a large idle helps but does not prevent idle closures if no bytes are sent for long stretches.
  • SSE transport: Viable, but Streamable HTTP is preferred for current MCP transport guidance and client compatibility.
  • Emitting synthetic domain events: Not always acceptable; many streams are legitimately quiet and should not send fake business events.
  • Local wrapper route: Implementable via ASGI re‑dispatch and a StreamingResponse wrapper that injects heartbeats, but this is brittle and depends on internal behavior. A first‑class hook and built‑in heartbeat would be cleaner and more maintainable.

Additional context

  • Environment constraints: Cannot modify platform‑level Envoy/Contour configuration; control is limited to application code and per‑route ingress config.
  • Current mitigations: Route timeouts configured to response: infinity and a large idle; still observe disconnects during quiet periods without periodic bytes.
  • Proposed defaults: Heartbeat disabled by default. When enabled, default interval 25s and payload b":hb\n" (harmless for generic stream readers and SSE‑style consumers).
  • Documentation request: Please document proxy considerations and recommend tuning heartbeat interval to be comfortably below typical idle timeouts. Clarify interaction with compression/buffering (advise disabling non‑streaming compression on this route or using streaming‑capable compression).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions