Commit 8f4de31
committed
Add upper limit on initial buffer size in MessagePack::Unpacker
Currently, the initial buffer size is specified in rb_ary_new2() or rb_hash_new_capa().
If a huge size is specified, a large amount of memory is allocated and system memory might be depleted.
We want to unpack the data received over the network.
However the service may stop due to large amount of memory allocation with crafted data.
So this patch add upper limit on initial buffer size.
If the buffer runs out, Ruby API will be reallocated automatically.
## Test code
```ruby
require "msgpack"
puts "msgpack version: #{MessagePack::VERSION}"
unpacker = MessagePack::Unpacker.new
unpacker.feed_each("\xDF\x20\x00\x00\x00") {}
puts "Memory Usage: #{`ps -o rss= -p #{Process.pid}`.strip} KB"
```
## Before
Before it apply this patch, it allocates 8 GB memory on my environment.
```
$ ruby -v test.rb
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
msgpack version: 1.7.2
Memory Usage: 8403320 KB
```
## After
```
ruby -v test.rb
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
msgpack version: 1.7.2
Memory Usage: 14480 KB
```1 parent 9330593 commit 8f4de31
1 file changed
+13
-6
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| 31 | + | |
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| |||
37 | 39 | | |
38 | 40 | | |
39 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
40 | 47 | | |
41 | 48 | | |
42 | 49 | | |
| |||
375 | 382 | | |
376 | 383 | | |
377 | 384 | | |
378 | | - | |
| 385 | + | |
379 | 386 | | |
380 | 387 | | |
381 | 388 | | |
382 | 389 | | |
383 | 390 | | |
384 | 391 | | |
385 | | - | |
| 392 | + | |
386 | 393 | | |
387 | 394 | | |
388 | 395 | | |
| |||
605 | 612 | | |
606 | 613 | | |
607 | 614 | | |
608 | | - | |
| 615 | + | |
609 | 616 | | |
610 | 617 | | |
611 | 618 | | |
| |||
615 | 622 | | |
616 | 623 | | |
617 | 624 | | |
618 | | - | |
| 625 | + | |
619 | 626 | | |
620 | 627 | | |
621 | 628 | | |
| |||
625 | 632 | | |
626 | 633 | | |
627 | 634 | | |
628 | | - | |
| 635 | + | |
629 | 636 | | |
630 | 637 | | |
631 | 638 | | |
| |||
635 | 642 | | |
636 | 643 | | |
637 | 644 | | |
638 | | - | |
| 645 | + | |
639 | 646 | | |
640 | 647 | | |
641 | 648 | | |
| |||
0 commit comments