When am trying to connect to sparklyr from r-studio am facing this error could anyone help me on this.
R-Studio installed in remote desktop and trying to connect spark on hdp3.0 cluster.
Generated kerberos ticket but even though getting invalid token error.
*Error in livy_validate_master(master, config) : *
Livy connections are generally not recommended. It is best to run sparklyr on an edge node of the Spark cluster.
I have tagged your question with "Spark" and "Sparklyr," so hopefully some more experienced "sparkers" will find it! Also be sure to check out spark.rstudio.com for some best practices!
What is the output from connecting to Livy using curl? From the terminal try,
curl https://abcxzy:8999/sessions
Can you execute the examples say, from http://gethue.com/how-to-use-the-livy-spark-rest-job-server-for-interactive-spark-2-2/ ?
Since you might have a complex authentication setup, I would recommend to take sparklyr out of the picture first and try to get simple commands working. Once you know what authentication is in-place and the HTTP headers you might need when using curl, you can map those headers to sparklyr by setting sparklyr.livy.headers, for instance:
config$sparklyr.livy.headers <- list(
livy_header1 = "value1",
livy_header2 = "value1"
)
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.