Hi
We installed Arrow in the Rstudio environment. We are able to validate Arrow functionality without SparklyR context. With SparklyR we are getting error.
Please help.
Standalone Arrow Test ---------------------------------------------------
library(arrow)
a <- read_csv_arrow(file = "/home/rmunda1/installed_packages.csv")
head(a)
#> # A tibble: 6 x 4
#> `` Package Version Priority
#>
#> 1 abind abind 1.4-5
#> 2 acepack acepack 1.4.1
#> 3 actuar actuar 2.3-1
#> 4 ada ada 2.0-5
#> 5 adabag adabag 4.2
#> 6 ade4 ade4 1.7-13
detach("package:arrow")
SparklyR Arrow Test ---------------------------------------------------With SparklyR:
library(arrow)
ptm <- proc.time()
collected <- sdf_len(sc, 10^6) %>% collect()
#> Error in record_batch_stream_reader(stream): could not find function "record_batch_stream_reader"
proc.time() - ptm
#> user system elapsed
#> 0.120 0.021 5.416
detach("package:arrow")
ptm <- proc.time()
collected <- sdf_len(sc, 10^6) %>% collect()
proc.time() - ptm
#> user system elapsed
#> 0.066 0.015 0.619
Thanks
Karan