We have an RStudio server instance on an EC2 instance on AWS. We try to use the necessary commands to connect to a local Spark instance:
sc <- spark_connect(master = "local")
We are getting the following error:
Error in force(code) :
Failed during initialize_connection: java.lang.IllegalArgumentException: java.net.UnknownHostException: ip-10-204-183-142.us-east-1.orbis-nonprod.aws.adp
In our EC2 instance, the command hostname returns the following:
We need to understand from which config file is the spark_connect function getting the incorrect ip-10-204-183-142 address so that we can modify it to use the correct IP address.