Skip to content

Conversation

@GrapeBaBa
Copy link
Member

Fix the snappy frame issues found by interop.

  1. Broken chunk-length maths: upstream’s getChunkSize shifts by <<9 instead of <<8, so lengths are mis-read. Any chunk larger than 511 bytes will decode incorrectly or terminate early.
  2. CRC handling is off-spec: the Snappy frame format requires a masked CRC-32C. Their encoder writes the raw CRC via snappy.crc, and the decoder drops it entirely—no masking, no validation. Interop peers will reject our frames, and we wouldn’t catch corrupt payloads.
  3. Decoder ignores framing invariants: it never checks the stream identifier and immediately falls back to writing payload bytes, so malformed headers slip through and the caller can’t detect it.

Signed-off-by: Chen Kai <281165273grape@gmail.com>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the Snappy framing implementation to fix critical bugs that prevent proper interoperability with external Snappy frame implementations. The changes address broken chunk-length calculations, incorrect CRC handling, and missing stream identifier validation.

  • Fixes chunk length calculation that was using incorrect bit shifts, causing chunks >511 bytes to decode incorrectly
  • Implements proper masked CRC-32C validation per Snappy frame specification
  • Adds stream identifier validation and proper frame format enforcement

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Signed-off-by: Chen Kai <281165273grape@gmail.com>
@GrapeBaBa GrapeBaBa requested a review from Copilot October 11, 2025 05:33
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 1 out of 3 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

const compressed = try snappyz.encode(self.allocator, chunk_input);
defer self.allocator.free(compressed);

const compress_threshold = chunk_input.len - (chunk_input.len / 8);
Copy link

Copilot AI Oct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The magic number 8 used for compression threshold calculation should be extracted as a named constant (e.g., compression_ratio_threshold = 8) to clarify its purpose and make it configurable.

Copilot uses AI. Check for mistakes.
Comment on lines +495 to +496
for (large_data, 0..) |*byte, idx| {
byte.* = @intCast((idx * 31) % 251);
Copy link

Copilot AI Oct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The magic numbers 31 and 251 used for test data generation should be extracted as named constants to clarify their purpose as pseudo-random data generation parameters.

Suggested change
for (large_data, 0..) |*byte, idx| {
byte.* = @intCast((idx * 31) % 251);
const test_data_multiplier = 31;
const test_data_modulus = 251;
for (large_data, 0..) |*byte, idx| {
byte.* = @intCast((idx * test_data_multiplier) % test_data_modulus);

Copilot uses AI. Check for mistakes.
@g11tech g11tech merged commit 78e66c2 into main Oct 11, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants