audioWorklets proposal 2

[Jonathhhan]

This time only one Emscripten file is changed: emscripten-core/emscripten@main...Jonathhhan:emscripten:audioWorkletOF_3.1.32-2
Maybe it is possible to replace that file automatically from the OF side?
It still can be optimized, but I think it is on the right track (and it works quite well)...
Here an example: https://audioplayer.handmadeproductions.de/

[ofTheo]

@Jonathhhan I am guessing this isn't working with the current core emscripten release yet?
Do you see your changes getting merged into the core emscripten? or is there a way to use audio worklets without modifying their files?

Thanks!

[Jonathhhan]

@ofTheo
hi, sorry initially I read your questions wrong.
no, it is not working with the current core emscripten (but it does, if you replace the audio_worklet.js file).

Do you see your changes getting merged into the core emscripten?

No, I do not think so (but have not asked yet)...

or is there a way to use audio worklets without modifying their files?

Thats what I am asking myself for a while, it would be much better and I am quite sure it is. But I already tried that without success (which does not mean that it is not possible). Here is shown how to access audio data with processAudio() from the c++ side without modifying anything: https://github.com/emscripten-core/emscripten/blob/main/test/webaudio/tone_generator.c
I would be one step further, if I would know how to access the ofBuffer (inbuffer/outbuffer) in processAudio(), but then I still have to pass the number of inputs etc. somehow...

[ofTheo]

Could you use the last argument UserData for the inputs outputs etc?

ie:

struct AudioStruct{
  int numInputs = 0; 
  int numOutputs = 0; 
  ofBuffer inBuffer; 
  ofBuffer outBuffer; 
}

Then when you start it you pass in the data as the last argument like:

       AudioStruct audioData;  //defined somewhere in the class 

        audioData.inputs = 2; 
        audioData.outputs = 0; 
	emscripten_start_wasm_audio_worklet_thread_async(context, wasmAudioWorkletStack, sizeof(wasmAudioWorkletStack), WebAudioWorkletThreadInitialized, (void *)&audioData);

Then in your callback:

// This function will be called for every fixed 128 samples of audio to be processed.
EM_BOOL ProcessAudio(int numInputs, const AudioSampleFrame *inputs, int numOutputs, AudioSampleFrame *outputs, int numParams, const AudioParamFrame *params, void *userData)
{
#ifdef REPORT_RESULT
	++audioProcessedCount;
#endif

AudioStruct * audioData = (AudioStruct *) userData; //everything you need in the struct 
audioData->outBuffer = ..... 

[Jonathhhan]

@ofTheo thanks a lot, that looks promising and I will try that.
Edit: It does not give any errors, but it also does not output the values yet. I still think this is the right direction.

[ofTheo]

@Jonathhhan awesome! Glad to hear! 👍

[Jonathhhan]

There is a recent Emscripten change, which makes it more clear which userData flows where:
emscripten-core/emscripten#18989

But I still did not found a way to pass the data where I want.

However the userData pointer that is passed to a call to emscripten_start_wasm_audio_worklet_thread_async() is not the userData pointer that will flow in to ProcessAudio().

[Jonathhhan]

Closed because of a new draft: #7727