Audio JavaScript library interfering with Shiny JavaScript?

shiny

#1

I’m trying to make an audio input Shiny app that uses this audio recording demo app as its basis.

I’ve tried to intercept where it outputs the audio wav file, and that seems to work and writes the wave file to the disk, so it all seems to work great - until I want to update Shiny with the results :frowning:

All the Shiny update functions seem to fail after the audio is captured: tableOutput(), textOutput() etc. don’t fire.

I think perhaps the audio recording JS may be interfering with the Shiny JS? But can’t get to the bottom of it or know how to debug it.

Perhaps I have put the Shiny.onInputChange() in the wrong place? I put it in this:

If anyone can point me in the right direction it would be appreciated, the full app code is in this branch of googleLanguageR : https://github.com/ropensci/googleLanguageR/tree/shiny-demo/inst/shiny/capture_speech


#2

When a recording ends, that worker is getting two messages. The first contains the raw data buffer (what you really want). The second contains the processed WAV, a binary object which can’t be directly serialized to JSON. If you check Shiny’s trace logs, it’s being sent as an empty object. I think you really just want to not export that WAV.

I poked around to see what runs when the recording stops.

https://github.com/ropensci/googleLanguageR/blob/shiny-demo/inst/shiny/capture_speech/www/main.js#L57-L61

https://github.com/ropensci/googleLanguageR/blob/shiny-demo/inst/shiny/capture_speech/www/main.js#L41-L49

I can’t confirm if this works, but try placing the Shiny.onInputChange here and taking out the exportWAV.

function gotBuffers( buffers ) {
    var canvas = document.getElementById( "wavedisplay" );

    drawBuffer( canvas.width, canvas.height, canvas.getContext('2d'), buffers[0] );

    // the ONLY time gotBuffers is called is right after a new recording is completed -
    // so here's where we should set up the download.
    
    // audioRecorder.exportWAV( doneEncoding );
    Shiny.onInputChange("audio", buffers);
}

#3

Thanks Greg, that did it! Please have my gratitude!


#4

I got it all working and the solution is here https://github.com/ropensci/googleLanguageR/tree/master/inst/shiny/capture_speech