RStudio Connect with Azure Databricks

Posit Professional & Hosted Products Posit Connect
1

Hello Team,

For one of our premier Reinsurance Organization in USA, we have proposed Azure Databricks as a processing cluster.

At present Azure Databricks supports integrating RStudio IDE however does not support Shiny Apps and RStudio Connect for single push button publishing.

Both Shiny apps and RStudio Connect are mandatory requirements from customer.

Considering this, we have proposed a RStudio Connect for Azure as a separate server on the dedicated VM on Azure.

Now the question that we have is, can we establish the connectivity between RStudio integrated with Azure Databricks and RStudio Connect for Azure on separate VM in Azure using rsconnect package(Tools->Global Options->Publishing)?

Also apart form the approach mentioned above, is there any other better way to accomplish the stated functionality?

I would be very thankful to hear from the experts on th ground!

Regards,
Santosh

2

This should be possible to do.

However, keep in mind that the RStudio Connect deployment will not be able to trigger Spark jobs inside DataBricks.

This means you will have to set up the workloads as follows:

  • Use DataBricks spark jobs to summarize, aggregate and process the data, and write these summaries to a database.

  • Use shiny apps and R Markdown reports that read these summaries from the database, and publish these apps to Connect.

In other words, your architecture will have separation between the processing cluster (DataBricks) and the data repository (your database).

3

Is there an update as now Databricks does support RShiny