Skip to content

Implement WebSocket message streaming for large payloads #2256

@sanity

Description

@sanity

Problem

Currently, the WebSocket client and server use large message size limits (100MB) to handle contract data. This is a workaround - the proper solution is to break up large messages into manageable chunks and stream them.

Current State

The 100MB limit works but has drawbacks:

  • Memory pressure from buffering large messages
  • Potential timeout issues on slow connections
  • Not scalable for very large contracts

Proposed Solution

Implement message chunking/streaming for large payloads:

  1. Define chunk size - e.g., 1MB chunks
  2. Chunking protocol - Add message framing to indicate:
    • Start of chunked transfer
    • Chunk sequence number
    • End of chunked transfer
  3. Server-side streaming - Break large responses into chunks
  4. Client-side reassembly - Reassemble chunks into complete messages
  5. Backpressure handling - Flow control for slow consumers

Benefits

  • Reduced memory footprint
  • Better handling of slow/unreliable connections
  • More predictable latency
  • Scalable to arbitrarily large contracts

Related

[AI-assisted - Claude]

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-contractsArea: Contract runtime, SDK, and executionA-networkingArea: Networking, ring protocol, peer discoveryE-mediumExperience needed to fix/implement: Medium / intermediateS-needs-designStatus: Needs architectural design or RFCT-enhancementType: Improvement to existing functionality

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions