I have a grouped dataframe grouped by gene (~20 rows) and I did a group_by(gene) and nest() to have a list column data.
I recently learned about furrr and tried to run future_map but it is running slower than purrr map. After reading the document on furrr I ungrouped my input before the call to future_map. But the sequential is still faster than the multicore plan.
I am not really sure what's going on.
library(tidyverse) library(furrr) #sequential time # input is grouped datframe with list-column data from previous group_by, nest #sequential plan plan(sequential) t1 <- proc.time() result_series <- exposure_response_tpm_sample %>% # first 1000 rows of original input (~20k) rows mutate(wilcox_result = map(data, wilcox_baseline_tpm_reponse), log2FC_baseline=map(data, calculate_TPM_foldchange_baseline)) #t1 sequential time is 570 seconds
# multicore plan plan(multicore, workers=8) t2 <- proc.time() result_series <- exposure_response_tpm_sample %>% ungroup() %>% #furrr doc said to ungroup mutate(wilcox_result = future_map(data, wilcox_baseline_tpm_reponse), log2FC_baseline=future_map(data, calculate_TPM_foldchange_baseline)) #t2 multicore time is 624 user seconds