ELI5 system.time() and understanding going parallel using foreach

parallel
#1

Hi all!

I have a nested loop that I wanted to speed up using foreach. I am using RStudio server with 24 cores. I was using system.time() to see how much I was speeding up the process and got the following results:

## system time 

#default empty registerdoParallel()
#   user  system elapsed 
#974.884  49.516 281.730 

#12 cores parallel 
#   user  system elapsed 
#395.024  38.268 443.409 

#16 cores parallel
#   user  system elapsed 
#402.524  35.576 444.529 

##pure loop
#   user  system elapsed 
#634.660   1.644 636.098

I soon discovered that I have no idea what system.time actually does. I looked on stackoverflow, but I'm still not sure I understand things completely. I'm feeling really dense right now... could someone explain this to me like I'm 5 years old? Hopefully, I'll be able to make heads or tails of my little parallel experiment. Also, why is there such a big difference between defining 12 cores:

no_cores <- detectCores() / 2  
cl <- makeCluster(no_cores, type="FORK") 
registerDoParallel(cl)

and just using:
registerDoParallel()

Shouldn't they be the same? Thanks again and sorry for the newb questions!

0 Likes