Hi
We installed Arrow in the Rstudio environment. We are able to validate Arrow functionality without SparklyR context. With SparklyR we are getting error.
Please help.

Standalone Arrow Test ---------------------------------------------------

library(arrow)

a <- read_csv_arrow(file = "/home/rmunda1/installed_packages.csv")
head(a)
#> # A tibble: 6 x 4
#> `` Package Version Priority
#>
#> 1 abind abind 1.4-5
#> 2 acepack acepack 1.4.1
#> 3 actuar actuar 2.3-1
#> 4 ada ada 2.0-5
#> 5 adabag adabag 4.2
#> 6 ade4 ade4 1.7-13

detach("package:arrow")

SparklyR Arrow Test ---------------------------------------------------With SparklyR:

library(arrow)
ptm <- proc.time()
collected <- sdf_len(sc, 10^6) %>% collect()
#> Error in record_batch_stream_reader(stream): could not find function "record_batch_stream_reader"
proc.time() - ptm
#>    user  system elapsed 
#>   0.120   0.021   5.416

detach("package:arrow")
ptm <- proc.time()
collected <- sdf_len(sc, 10^6) %>% collect()
proc.time() - ptm
#>    user  system elapsed 
#>   0.066   0.015   0.619

Thanks
Karan

Could you share the output of sessioninfo::session_info()?

─ Session info ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 3.6.0 (2019-04-26)
os CentOS Linux 7 (Core)
system x86_64, linux-gnu
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/Chicago
date 2020-04-14

─ Packages ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
package * version date lib source
arrow 0.16.0.1 2020-02-10 [2] CRAN (R 3.6.0)
askpass 1.1 2019-01-13 [2] CRAN (R 3.6.0)
assertthat 0.2.1 2019-03-21 [2] CRAN (R 3.6.0)
backports 1.1.4 2019-04-10 [2] CRAN (R 3.6.0)
base64enc 0.1-3 2015-07-28 [2] CRAN (R 3.6.0)
bit 1.1-14 2018-05-29 [2] CRAN (R 3.6.0)
bit64 0.9-7 2017-05-08 [2] CRAN (R 3.6.0)
bitops 1.0-6 2013-08-17 [2] CRAN (R 3.6.0)
broom 0.5.2 2019-04-07 [2] CRAN (R 3.6.0)
cellranger 1.1.0 2016-07-27 [2] CRAN (R 3.6.0)
cli 1.1.0 2019-03-19 [2] CRAN (R 3.6.0)
colorspace 1.4-1 2019-03-18 [2] CRAN (R 3.6.0)
crayon 1.3.4 2017-09-16 [2] CRAN (R 3.6.0)
DBI * 1.0.0 2018-05-02 [2] CRAN (R 3.6.0)
dbplyr 1.4.2 2019-06-17 [2] CRAN (R 3.6.0)
digest 0.6.19 2019-05-20 [2] CRAN (R 3.6.0)
dplyr * 0.8.2 2019-06-29 [2] CRAN (R 3.6.0)
ellipsis 0.3.0 2019-09-20 [2] CRAN (R 3.6.0)
fansi 0.4.0 2018-10-05 [2] CRAN (R 3.6.0)
forcats * 0.4.0 2019-02-17 [2] CRAN (R 3.6.0)
forge 0.2.0 2019-02-26 [2] CRAN (R 3.6.0)
generics 0.0.2 2018-11-29 [2] CRAN (R 3.6.0)
ggplot2 * 3.2.0 2019-06-16 [2] CRAN (R 3.6.0)
glue 1.3.1 2019-03-12 [2] CRAN (R 3.6.0)
gtable 0.3.0 2019-03-25 [2] CRAN (R 3.6.0)
h2o 3.24.0.4 2019-07-02 [2] local
haven 2.1.1 2019-07-04 [2] CRAN (R 3.6.0)
hms 0.5.0 2019-07-09 [2] CRAN (R 3.6.0)
htmltools 0.3.6 2017-04-28 [2] CRAN (R 3.6.0)
htmlwidgets 1.3 2018-09-30 [2] CRAN (R 3.6.0)
httr 1.4.1 2019-08-05 [2] CRAN (R 3.6.0)
jsonlite 1.6 2018-12-07 [2] CRAN (R 3.6.0)
lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.0)
lazyeval 0.2.2 2019-03-15 [2] CRAN (R 3.6.0)
lubridate 1.7.4 2018-04-11 [2] CRAN (R 3.6.0)
magrittr * 1.5 2014-11-22 [2] CRAN (R 3.6.0)
modelr 0.1.4 2019-02-18 [2] CRAN (R 3.6.0)
munsell 0.5.0 2018-06-12 [2] CRAN (R 3.6.0)
nlme 3.1-139 2019-04-09 [2] CRAN (R 3.6.0)
openssl 1.4 2019-05-31 [2] CRAN (R 3.6.0)
pillar 1.4.2 2019-06-29 [2] CRAN (R 3.6.0)
pkgconfig 2.0.2 2018-08-16 [2] CRAN (R 3.6.0)
purrr * 0.3.2 2019-03-15 [2] CRAN (R 3.6.0)
r2d3 0.2.3 2018-12-18 [2] CRAN (R 3.6.0)
R6 2.4.0 2019-02-14 [2] CRAN (R 3.6.0)
Rcpp 1.0.1 2019-03-17 [2] CRAN (R 3.6.0)
RCurl 1.95-4.12 2019-03-04 [2] CRAN (R 3.6.0)
readr * 1.3.1 2018-12-21 [2] CRAN (R 3.6.0)
readxl 1.3.1 2019-03-13 [2] CRAN (R 3.6.0)
rJava * 0.9-11 2019-03-29 [2] CRAN (R 3.6.0)
RJDBC * 0.2-7.1 2018-04-16 [2] CRAN (R 3.6.0)
rlang * 0.4.0 2019-06-25 [2] CRAN (R 3.6.0)
rprojroot 1.3-2 2018-01-03 [2] CRAN (R 3.6.0)
rsparkling * 0.2.25 2019-07-02 [2] local
rstudioapi 0.10 2019-03-19 [2] CRAN (R 3.6.0)
rvest 0.3.4 2019-05-15 [2] CRAN (R 3.6.0)
scales 1.0.0 2018-08-09 [2] CRAN (R 3.6.0)
sessioninfo 1.1.1 2018-11-05 [2] CRAN (R 3.6.0)
sparklyr * 1.0.1 2019-05-17 [2] CRAN (R 3.6.0)
stringi 1.4.3 2019-03-12 [2] CRAN (R 3.6.0)
stringr * 1.4.0 2019-02-10 [2] CRAN (R 3.6.0)
tibble * 2.1.3 2019-06-06 [2] CRAN (R 3.6.0)
tidyr * 0.8.3 2019-03-01 [2] CRAN (R 3.6.0)
tidyselect 0.2.5 2018-10-11 [2] CRAN (R 3.6.0)
tidyverse * 1.2.1 2017-11-14 [2] CRAN (R 3.6.0)
utf8 1.1.4 2018-05-24 [2] CRAN (R 3.6.0)
vctrs 0.2.0 2019-07-05 [2] CRAN (R 3.6.0)
withr 2.1.2 2018-03-15 [2] CRAN (R 3.6.0)
xml2 1.2.0 2018-01-24 [2] CRAN (R 3.6.0)
zeallot 0.1.0 2018-01-28 [2] CRAN (R 3.6.0)

[1] /home/XXXXXXXX/R/x86_64-redhat-linux-gnu-library/3.6
[2] /usr/lib64/R/library
[3] /usr/share/R/library

@Blair09M - Hi James - Any update on this please?

I haven't been able to reproduce the issue. Do you notice any glaring differences in this reprex?

library(sparklyr)

sc <- spark_connect(master = "local")

library(arrow)
#> 
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#> 
#>     timestamp
ptm <- proc.time()
collected <- sdf_len(sc, 10^6) %>% collect()
proc.time() - ptm
#>    user  system elapsed 
#>   0.361   0.036   5.339

detach("package:arrow")
ptm <- proc.time()
collected <- sdf_len(sc, 10^6) %>% collect()
proc.time() - ptm
#>    user  system elapsed 
#>   0.080   0.005   0.585

Created on 2020-04-21 by the reprex package (v0.3.0)

Session info
sessionInfo()
#> R version 3.6.2 (2019-12-12)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: CentOS Linux 7 (Core)
#> 
#> Matrix products: default
#> BLAS/LAPACK: /usr/lib64/libopenblasp-r0.3.3.so
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices datasets  utils     methods   base     
#> 
#> other attached packages:
#> [1] sparklyr_1.1.0.9001
#> 
#> loaded via a namespace (and not attached):
#>  [1] Rcpp_1.0.4        compiler_3.6.2    pillar_1.4.3      dbplyr_1.4.2     
#>  [5] highr_0.8         r2d3_0.2.3        base64enc_0.1-3   tools_3.6.2      
#>  [9] bit_1.1-15.2      digest_0.6.25     jsonlite_1.6.1    evaluate_0.14    
#> [13] tibble_2.1.3      pkgconfig_2.0.3   rlang_0.4.5       DBI_1.1.0        
#> [17] rstudioapi_0.11   yaml_2.2.1        parallel_3.6.2    xfun_0.12        
#> [21] withr_2.1.2       httr_1.4.1        stringr_1.4.0     dplyr_0.8.5      
#> [25] knitr_1.28        vctrs_0.2.4       askpass_1.1       rappdirs_0.3.1   
#> [29] generics_0.0.2    htmlwidgets_1.5.1 bit64_0.9-7       rprojroot_1.3-2  
#> [33] tidyselect_1.0.0  glue_1.3.2        forge_0.2.0       R6_2.4.1         
#> [37] rmarkdown_2.1     purrr_0.3.3       magrittr_1.5      backports_1.1.5  
#> [41] htmltools_0.4.0   ellipsis_0.3.0    assertthat_0.2.1  renv_0.9.3-58    
#> [45] arrow_0.17.0      config_0.3        stringi_1.4.6     openssl_1.4.1    
#> [49] crayon_1.3.4