Hi i was attempting to do sentiment analysis on a large comments file (approx 47 mb). It was taking over two hours. kindly advise on this.
How long does it take on your local machine? Is it something that benefits from multiple cores normally?
it is on cloud right so why the need for my local machine
It's a way to isolate variables: if the code is running very quickly on your machine locally, then that helps us identify what might be going on with rstudio cloud.
It's not running very quickly on the local machine. I got a memory error but the time taken on r cloud far exceeded the time on local machine. It was like 2 to 3 hours even after shortening the file by half.
You need to be more specific.
Are you saying that when you run it on the local machine after a while the code fails with a memory error but on rstudio cloud the code runs to completion but takes a long time?
When you say you cut the file in half and it still took more than 2 hours... were you talking about the code running on the local machine and failing, or on rstudio cloud and completing?
Hi Sir, i tried two packages, sentimentr and sentimentanalysis. When running the command sentiment(b) where b was my file about 203000 comments, it did not reach completion. On using the command analyzesentiment in sentimentanalysis, again it did not reach completion. Then i tried splitting the file and then ran sentimentanalysis(b) where b had 100000, it did not reach completion. same outcome for 50000 also. finally i tried sentimentanalysis(b) with 1000 comment files and it worked, reaching completion. All of this was on rcloud. I tried only once on my local system where it directly gave me error, unable to allocate vector of size 50.5 mb(memory error)
We still need more specifics.
What you need to do first is to determine if R Studio cloud has an issue that is making it run slowly or if your code has an issue that is making it run slowly.
Are you saying the 1000 comment file ran on rstudio cloud to completion but took over 2 hours to run?
Try running the 1000 comment file on your local machine to see how long that takes.
You need to do is an apples to apples comparison on both the cloud and your local machine.