Package- SparkR

rstudio

#1

Hi,
Could anyone please help me? I have RStudio in my local machine and running into memory issues when reading a huge data file. Planning to use SparkR. But, want to know on the below.
1.Is there any difference between Sparklyr and SparkR?
2. Since , both are used for Spark Integration from R, when to use which package?
3. Since, both have active community, trying to see which one is the best?
4. Also , are there any limitations or challenges on these two?

If anyone could provide details on this, that would be great.
Thanks!


#2

Things may have changed since May, but thought I’d point to @kevinushey’s outline of differences/motivations here:


#3

About this, there another topic on the community that could have interesting information for you

And also the two topic are related, it is good to link them.


#4

Hi all,

I recently wrote a blog post comparing sparklyr and SparkR on a range of criteria.

The table at the top of the post provides a rough overview of my conclusions. Essentially sparklyr is already nicer to use even at this early stage of its development (compared to SparkR).


#5

Great job on the article, I recently read it and shared it internally. Thanks!