I am having trouble connecting MySQL to spark, possible reasons are, for instance, java version, java files location, connector files location, MySQL version, environment variable location, the use of jdbc or odbc, and so on. My questions are:
Do we need to install hadoop and java before installing sparklyr? I am using R base, not Rstudio.
Which version of each of these package are stable for successful installation and connection, if anyone had any possible experience? (the solutions online might worked on older version of these packages, but seems not working anymore in my case, I’m on mac by the way).
So far, the only way I tried successfully is to utilize the sqldf package on SparkR to connect MySQL, but I am not sure if spark was working (to speed up the process) when I run the sql queries with sqldf package on SparkR. Can I do the same with sparklyr? Then how do I know if it is spark that is working behind or the R that is working behind?
I hope I described my questions clearly. Thank you very much for the help.