-
-
Notifications
You must be signed in to change notification settings - Fork 108
Open
Labels
A-contractsArea: Contract runtime, SDK, and executionArea: Contract runtime, SDK, and executionA-networkingArea: Networking, ring protocol, peer discoveryArea: Networking, ring protocol, peer discoveryE-mediumExperience needed to fix/implement: Medium / intermediateExperience needed to fix/implement: Medium / intermediateS-needs-designStatus: Needs architectural design or RFCStatus: Needs architectural design or RFCT-enhancementType: Improvement to existing functionalityType: Improvement to existing functionality
Description
Problem
Currently, the WebSocket client and server use large message size limits (100MB) to handle contract data. This is a workaround - the proper solution is to break up large messages into manageable chunks and stream them.
Current State
- Server-side (
crates/core/src/client_events/websocket.rs): Configured for 100MB messages - Client-side: Now also configured for 100MB to match (PR fix: increase WebSocket client message size limit to 100MB #2252)
The 100MB limit works but has drawbacks:
- Memory pressure from buffering large messages
- Potential timeout issues on slow connections
- Not scalable for very large contracts
Proposed Solution
Implement message chunking/streaming for large payloads:
- Define chunk size - e.g., 1MB chunks
- Chunking protocol - Add message framing to indicate:
- Start of chunked transfer
- Chunk sequence number
- End of chunked transfer
- Server-side streaming - Break large responses into chunks
- Client-side reassembly - Reassemble chunks into complete messages
- Backpressure handling - Flow control for slow consumers
Benefits
- Reduced memory footprint
- Better handling of slow/unreliable connections
- More predictable latency
- Scalable to arbitrarily large contracts
Related
- PR fix: increase WebSocket client message size limit to 100MB #2252: Temporary fix increasing client message size limit to 100MB
[AI-assisted - Claude]
Metadata
Metadata
Assignees
Labels
A-contractsArea: Contract runtime, SDK, and executionArea: Contract runtime, SDK, and executionA-networkingArea: Networking, ring protocol, peer discoveryArea: Networking, ring protocol, peer discoveryE-mediumExperience needed to fix/implement: Medium / intermediateExperience needed to fix/implement: Medium / intermediateS-needs-designStatus: Needs architectural design or RFCStatus: Needs architectural design or RFCT-enhancementType: Improvement to existing functionalityType: Improvement to existing functionality
Type
Projects
Status
Triage