Rstudio SparkUI button unresponsive while R is busy (with a spark session)

Hello,

I'm not sure if it's just me, but I commonly kick off a query through sparklyr and I'd like to see the Spark UI in order to monitor its progress. The problem becomes that once I have kicked it off in R, the Rstudio IDE basically makes this "Spark UI" button unresponsive. It's really counter-intuitive, because I don't need to be monitoring my spark session unless I am curious about the progress. It appears that if I click it, it will wait, and then after the spark job is done, it will pop open the new window.

Should I file a git request/bug somewhere for an upgrade on this feature? It seems that it should be disconnected from the current R session so that it can run when the session is busy.

Appreciate any advice! Thanks!

I think this is worth filing an issue for. Would you mind filing this as a feature request at https://github.com/rstudio/rstudio/issues?

1 Like

A fix for this issue is available in sparklyr 1.0.1.9003, to make use of this improvement install sparklyr from GitHub:

install.packages("remotes")
remotes::install_github("rstudio/sparklyr")

See https://github.com/rstudio/sparklyr/pull/2044

1 Like

Thanks @javierluraschi !

Also thanks to @kevinushey!

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.