[BREAKING] Conv1D manages its own ring buffer #181
Merged
+2,878
−547
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add RingBuffer class to manage write/read pointers for Eigen::MatrixXf buffers - Add Reset() method to Conv1D to initialize ring buffer and pre-allocate output - Add get_output() method to Conv1D to access output buffer - Rename process_() to Process() and update to use internal ring buffer - Update ConvNet to use Conv1D internal buffers via Process() and get_output() - Update WaveNet to use Conv1D internal buffers and propagate Reset() - Add comprehensive tests for Conv1D in test_conv1d.cpp Implements issue #145
- Fix Reset() to zero buffer behind starting write position - Fix Rewind() to copy receptive_field (max lookback) samples to start - Set write position after receptive_field when rewinding - Add comprehensive test suite for RingBuffer (12 tests) - Tests cover: construction, reset, write/read, advance, rewind, lookback, etc. Fixes issues with RingBuffer not properly handling max lookback when rewinding.
- Rename _receptive_field member to _max_lookback - Rename SetReceptiveField() to SetMaxLookback() - Update all comments and documentation - Update all test references from receptive_field to max_lookback - More descriptive name for ring buffer context
- Create NAM/ring_buffer.h and NAM/ring_buffer.cpp for RingBuffer class - Create NAM/conv1d.h and NAM/conv1d.cpp for Conv1D class - Remove RingBuffer and Conv1D from NAM/dsp.h and NAM/dsp.cpp - Update includes in convnet.h, wavenet.h, and test files - All tests pass after refactoring
- Remove GetCapacity() from public interface (storage size is internal detail) - Add GetMaxBufferSize() to return the max_buffer_size passed to Reset() - Store max_buffer_size as member variable - Update tests to use GetMaxBufferSize() instead of GetCapacity() - External code should trust that storage is sized correctly
- Assert that write pointer is at least 2 * max_lookback to avoid aliasing - Assert that copy start position is within storage bounds - Remove silent failure condition - now asserts instead of silently skipping - Prevents data corruption from overlapping copy operations
- Renamed method in conv1d.h and conv1d.cpp - Updated all usages in convnet.cpp and wavenet.cpp - Updated all test cases in test_conv1d.cpp - Matches naming convention with Conv1x1::GetOutput()
- Remove _total_written member variable and all references - Remove GetReadPos() method (read_pos calculation now inline in Read()) - Remove test_get_read_pos() test function - Simplify RingBuffer for exclusive use by Conv1D layer - Update tests to work without GetReadPos()
…dd GetHeadOutputs() - Remove head_outputs parameter from both Process() overloads - Add GetHeadOutputs() method to retrieve head outputs from _head_rechannel - Update WaveNet::process() to use GetHeadOutputs() directly from layer arrays - Remove deprecated set_num_frames_() methods from _Layer and _LayerArray - Update _Layer::SetMaxBufferSize() to use Conv1D::SetMaxBufferSize() instead of Reset() - Simplify head input zeroing in LayerArray::Process()
- Remove _DilatedConv wrapper class, use Conv1D directly in _Layer - Refactor LayerArray::Process() to extract common logic into ProcessInner() - Make _Layer::Process() RT-safe by removing resize in Process() - Update _Layer::SetMaxBufferSize() to use Conv1D::SetMaxBufferSize() - Update tests and ConvNet to use SetMaxBufferSize() instead of Reset() - Simplify _Layer::Process() by combining conv and input_mixin processing
- Split WaveNet tests into three files by component: - test_layer.cpp: Tests for individual WaveNet layers - test_layer_array.cpp: Tests for layer arrays - test_full.cpp: Tests for full WaveNet models - Use nested namespaces: test_wavenet::test_layer, test_wavenet::test_layer_array, test_wavenet::test_full - Remove old monolithic test_wavenet.cpp file - Update test runner to include new test files directly - Add 12 new comprehensive tests covering: - Gated and non-gated layers - Different activations - Multi-channel layers - Layer array processing - Receptive field calculations - Full model processing - Edge cases (zero input, different buffer sizes) - Prewarm functionality
- Add Conv1D constructor with shape parameters - Rename Conv1D::Reset() to SetMaxBufferSize(), remove unused sampleRate param - Add Conv1D::has_bias() method - Reorganize RingBuffer public/private interface (move internal methods to private) - Improve test assertions with numerical accuracy checks - Clean up ring buffer tests, remove internal state checks - Remove commented code from WaveNet - Update NOTES with completed tasks
- Refactor ConvNetBlock to add Process() and GetOutput() methods using new Conv1D API - Simplify ConvNet::process() to eliminate _block_vals for Conv1D layers - Simplify buffer management - Conv1D handles its own ring buffers - Fix test expectations in test_conv1d.cpp (corrected weight ordering) - Fix test bug in test_conv1d.cpp (output2 -> output) - Fix ring buffer test assertion - Add comprehensive ConvNet tests (test_convnet.cpp) - Update test runner to include ConvNet tests This completes the refactoring work for issue #145, making ConvNet use Conv1D's new ring buffer API similar to how WaveNet was refactored.
Fixed assertion failure 'DenseBase::resize() does not actually allow to resize' at wavenet.cpp:61. The issue occurred when assigning _z.leftCols() to _output_head.leftCols() for gated layers, where _z has 2*channels rows but _output_head has only channels rows. Fix: Use _z.topRows(channels).leftCols(num_frames) for gated layers to correctly select only the first channels rows that should be copied to _output_head.
Fixed incorrect weight counts in ConvNet tests. The tests were missing bias weights when batchnorm=false (since !batchnorm=true means do_bias=true in ConvNetBlock::set_weights_). Changes: - test_convnet_basic: Added 4 bias weights (2 per block) - 15 to 19 weights - test_convnet_batchnorm: Removed 1 extra bias (batchnorm=true means no bias) - 10 to 9 weights - test_convnet_multiple_blocks: Added 6 bias weights (2 per block) - 23 to 29 weights - test_convnet_zero_input: Added 1 bias weight - 3 to 4 weights - test_convnet_different_buffer_sizes: Added 1 bias weight - 3 to 4 weights - test_convnet_prewarm: Added 6 bias weights (2 per block) - 23 to 29 weights - test_convnet_multiple_calls: Added 1 bias weight - 3 to 4 weights All tests now pass successfully.
Added SetMaxBufferSize and Process methods to ConvNetBlock to manage output buffers independently from Conv1D, allowing proper batchnorm and activation application on block-owned buffers.
- Remove WaveNet::_head_output member variable (never read) - Remove WaveNet::_set_num_frames_ method declaration (never implemented) - Remove _LayerArray::_get_receptive_field() private method (has TODO to remove) - Remove _Head::_head member variable (never used) - Remove entire _Head class (never instantiated in WaveNet)
- Add .noalias() to matrix assignments for better performance - Remove unnecessary _z.setZero() call (matrix is initialized as needed) - Remove redundant comment in SetMaxBufferSize - Add build workaround for conv1d.cpp Eigen warnings
- Add test_process_realtime_safe() to verify no allocations during process() - Add allocation tracking using malloc/free hooks to catch Eigen allocations - Add helper tests to verify allocation tracking works correctly - Test processes buffers of multiple sizes with two layer arrays - Ensures WaveNet::process() is real-time safe (no allocations/frees)
- Add test_conv1d_process_realtime_safe() to test Conv1D with full matrix input - Add test_conv1d_process_block_realtime_safe() to test Conv1D with Block input - Add test_layer_process_realtime_safe() to test Layer::Process() - Add test_layer_array_process_realtime_safe() to test LayerArray::Process() - Tests confirm Conv1D allocates when passed Block expressions - Attempt to fix RingBuffer::Write() to avoid allocations (work in progress)
- Remove Conv1D Block-input real-time test and rely on full-matrix path - Ensure tests and runtime code pass full buffers between layers, slicing only inside - Simplify RingBuffer::Write to take MatrixXf and add note about full-buffer requirement
- Add full-buffer getters for Conv1x1, _Layer, and _LayerArray - Change LayerArray and WaveNet to pass full MatrixXf buffers and slice internally - Simplify RingBuffer::Write API and document the full-buffer requirement - Restore real-time safety test file and keep only full-matrix Conv1D test in runner
- Remove GetOutputHead(num_frames) and GetOutputNextLayer(num_frames) from _Layer - Remove GetLayerOutputs(num_frames) and GetHeadOutputs(num_frames) from _LayerArray - Rename *Full() methods to original names (GetOutputHead, GetOutputNextLayer, etc.) - Update Conv1D and Conv1x1 GetOutput() to return full buffer (no num_frames param) - Update all internal code and tests to use .leftCols(num_frames) on full buffers - All methods now return pre-allocated full buffers; callers slice as needed
Owner
Author
Owner
Author
|
Deep breath |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a bit of a big one.
The main change is to get each
Conv1Dto hold a ring buffer inside it that's managed automatically. The main methods for working with it are now more idiomatic, I think:SetMaxBufferSize()(Should be called from `Reset() on modules that own convolution modules)Process()to process an input bufferGetOutput()to access the results of aProcess()call.This causes quite a bit of re-work in the WaveNet (and ConvNet) classes.
I also made sure that the WaveNet is real-time safe when using this new setup, including some new tests for real-time safety. I think that the checking here may be better than we had before, and it sniffed out a few spots where Eigen's block functionality might have been causing some blocks to be evaluated, causing (de)allocations during code that should have been real-time safe (i.e. WaveNet's
process()method). Possibly related, I also saw that The WaveNet model in this change is about 5-15% faster than currentmainwhen this is built into REAPER and I try rendering a minimal project (just one track with an instance of NeuralAmpModelerPlugin, with all the "extras" disabled except the DC blocker). I checked with buffer sizes of 64 and 1024.So, Conv1D now lives in its own source/header files (not
dsp.{h,cpp}). There's also a pair ofring_bufferfiles for that part, but since it's a bit specialized for use by the Conv1D class it might make sense to put it inside theconv1dfiles.There's also lots more tests for the ConvNet and WaveNet models. I still didn't look super-close at the ConvNet model (and it's not guaranteed RT-safe), but since it's not a very interesting model I'm not focusing much on it at the moment.
Lots of changes to the WaveNet implementation, but it's all below the public interfaces, amazingly (since
_Layerand_LayerArrayare meant to be private).Some of the main ones to call out are that the outputs of various objects are now generally "owned" by said object and accessed via
GetOutput()-like methods. WaveNet layers and layer arrays have two different inputs and outputs.Also, since the convolution ring buffers are now encapsulated within the convolution objects that own them, there aren't any "start/end" arguments that need to be provided to the layer array and layer
Process()calls--much cleaner and less error-prone, I hope.This is also where I ended up being a bit shy on the use of "block" operations with Eigen, which seems to have been causing RT safety issues in intermediate stages of developing this. I wonder if they may have caused RT safety issues in earlier versions of this code that weren't caught. Regardless, the test are much better at catching them.
Finally, lots more tests--about 50 of them. Hopefully this improves the airtightness of things moving forward :) Most were AI-drafted, but I've read them all to ensure that they are sane, and I made some changes to get there :)
Developed with support and sponsorship from TONE3000