Hello I am just getting started using Sparklyr and I am getting an error when trying to use dplyr to wrangle some data. I'm sure its something stupid I am overlooking but I have searched in multiple areas for an answer and I don't see anyone else getting this error
library(sparklyr) sc <- spark_connect(master = "local") spark_read_csv(sc, "df2_tbl", "C:/Users/...csv") spark_read_csv(sc, "df_n2_tbl", "C:/Users/...csv")
I see the objects "df2_tbl" and "df2_n2_tbl" in the "Connections" tab next to "Environment" and "History" as well as on the Spark UI, but when I run the following
match_cat <- df_n2_tbl %>% filter(var1 %in% df2_tbl) %>% collect()
I get the error -
"Error in eval(lhs, parent, parent) : object 'df_n2_tbl' not found"