I'm starting to work with R in databricks and have an issue using the geohashTools package. I am trying to convert lat/long to geohashes usign the gh_encode function in geohashTools library within databricks. And while I have solved one problem I have created another.
Using the tbl function I first access a hive table then I attempted to use the follow example code to manipulate the data:
my_tble <- my_tbl %>%
mutate(geo_hash = gh_encode(lat, long, percision = 9L)
but this results in some sort of parse error. So what I did was to use the collect function to convert the hive table into a R data.frame. Runnign the same command the gh_encode function works. But now I have an R dataframe so I though I could convert it to a Spark dataframe using SparkR::as.DataFrame(acled_temp). But now I have a Spark data.frame.
What I can't figure out is why the mutate command as worked just fine except when using the geohoashTools fnctions. Converting object types causes another set of problems. Any idea why mutate(geo_hash = gh_encode(lat, long, percision = 9L) would error out in the first place.
Jeff