Added gating activation classes #180
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
By the way, this might have an impact on performance since I am calling |
This was referenced Jan 14, 2026
Closed
added 3 commits
January 14, 2026 09:46
sdatkinson
requested changes
Jan 14, 2026
Owner
sdatkinson
left a comment
sdatkinson
left a comment
There was a problem hiding this comment.
Haven't gotten to the tests yet, but blending activation isn't quite right.
…ctivation. Removed all runtime checks and replaced with asserts.
sdatkinson
approved these changes
Jan 14, 2026
Owner
sdatkinson
left a comment
sdatkinson
left a comment
There was a problem hiding this comment.
Nits but nothing critical 👍🏻
|
|
||
| nam::gating_activations::GatingActivation gating_act(nullptr, nullptr, channels); | ||
|
|
||
| // This should not crash or produce incorrect results due to memory contiguity issues |
Owner
There was a problem hiding this comment.
Nit: This might be done better...but the numerical assertions are reasonable, and when this gets combined with the better real-time saftey testing in the Conv1D PR we'll be in good shape.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding abstractions for gating activations and blending activations. These classes are both very similar: they take two activations (and a blending coefficient alpha for blending activations). One activation is used for channel 0 of the input (the "input activation" in the code) and the other is used for channel 1 (the "gating activation" in the code). The difference between the two is that gating activation uses the activation of the second channel as a gate, while blending activation sums both channels scaled by alpha and (1-alpha).
Developed with support and sponsorship from TONE3000