Skip to content

Conversation

@Ray0907
Copy link

@Ray0907 Ray0907 commented Jan 13, 2026


Summary

  • Fix thread safety issue in fast_tanh toggle by using std::atomic instead of modifying shared state
  • Remove mutable global _activations map modification that could cause data races

Test plan

  • Verify fast_tanh enable/disable works correctly in single-threaded context
  • Test concurrent access from multiple threads toggling fast_tanh
  • Confirm no performance regression with atomic load in hot path

  Use atomic<bool> and runtime check instead of modifying shared map.
Copy link
Owner

@sdatkinson sdatkinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

If you can check to make sure that this doesn't clash with the real-time safety of any of the nets, then this seems ok.

Also, some testing to cement the intent of the change would be appreciated.

{
if (_activations.find(name) == _activations.end())
// Return FastTanh when Tanh is requested and fast_tanh mode is enabled
if (name == "Tanh" && using_fast_tanh.load(std::memory_order_relaxed))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this change make the WaveNet's getting of the sigmoid activation for its gating activation no longer real-time safe?

activations::Activation::get_activation("Sigmoid")->apply(this->_z.block(channels, i, channels, 1));

@Ray0907
Copy link
Author

Ray0907 commented Jan 15, 2026

Thanks for the review!

Real-time safety:

For non-"Tanh" lookups like WaveNet's "Sigmoid":

  if (name == "Tanh" && using_fast_tanh.load(...))
  //  "Sigmoid" != "Tanh" → short-circuits, atomic load never executed

It just hits the string comparison (false), then goes straight to find() - same as before, actually one fewer map lookup since we reuse the iterator now.

For "Tanh" lookups, the atomic load is lock-free with memory_order_relaxed - compiles down to a plain memory read on x86/ARM.

One thing I did consider: _activations.at("Fasttanh") technically can throw, though "Fasttanh" is guaranteed to exist. If you'd prefer, I can cache the pointer or use find() instead to be more defensive.

Tests:

Happy to add tests - what would you like to see? Unit tests for toggle behavior, multi-threaded stress tests, or both?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants