Thanks Cole for your reply.
I currently have RStudio on edge node, but we are planning to have transient aws emr cluster for each data scientist, but use one RStudio. I thought Livy would be good for this so each user makes spark connection to their dedicated transient cluster's Livy server.
I guess i can also setup edge note for each user with different environment variables, so each user connects to different EMR cluster's 'YARN'. I'm trying this but having issues with spark lib jars "Failed during initialize_connection: org.apache.hadoop.ipc.RemoteException(java.io.IOException)"...trying to investigate this now.
But is this the only way to have multiple RStudio users to different EMR Cluster independently or is there any other option? please advise.
Currently this is a POC, but my client may be interested to buy Pro once this is proved.
Thanks,
Asif