Sparklyr: spark_apply (Error in file(con, "r") : cannot open the connection)

[ahmadamado]

Hello I've been trying to use spark_apply to run the (treeClust) function r package on spark but it gives the following error:

Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file 'C:\Users\Ahmad\AppData\Local\Temp\RtmpUx7cfr\file444c1e7b6fb7_spark.log': Permission denied

I am using R Studio Desktop version for Windows on Windows 10 64 Bit.

I wrote the function as follows:

U <- CrimesCompleteCasesSDF %>% 
     spark_apply(function(x) treeclust::TreeClusteringCLARA <- treeClust(d.num = 2, final.algorithm = "clara, k= 34, x))

[harryzhu]

@ahmadamado
You should open RStudio as administration in Windows

[ahmadamado]

Thanks for your help, but the error still exists even after running R Studio as administrator.

[harryzhu]

@ahmadamado

Permission denied

As the log says, it is a Permission denied problem.

[cc20002002]

Same problem here, my code is

require(sparklyr)
sc <- spark_connect(master='local')
airquality2=as.data.frame(sapply(airquality,as.numeric))
airquality2$Month=as.character(airquality2$Month)
df <- sdf_copy_to(sc,na.omit(airquality2))

spark_apply(
df,
function(df) {

        data.frame(month = df %>% select('Month'), mse = svm(Ozone ~ Solar_R+Wind+Temp+Day, df, cross = 3)$tot.MSE)
  },
  names = c("Month", "MSE"),
  packages='e1071',
  group_by = "Month")