Yes I did figure out what the issue was. For me it was due to using a long list (~14000 items) in a SelectizeInput() widget. For some reason, Chrome could handle it quite quickly whereas the other browsers could not. As a work around, I created a word filter textInput widget, that prefiltered the list, and then I used updateSelectizeInput(...) to update the much smaller list. This significantly sped things up across all browsers.
If your issue is similar to mine I could show you my code snippets that may help you implement such a solution.
If your issue is elsewhere, I recommend using the browser() function to go through your code line by line (deploy locally in the slow browser) and figure out what code chunk/widget is making things really slow.