Problem with parallel computing. When I run the pdp::partial() functions on a workflow object

I am using the partial function to calculate two dimensional dependence. The code works fine. Unfortunately, when I add parallel computation commands, an error pops up. I have no idea how to solve it :(.

library(tidymodels)
library(doParallel)
#> Ładowanie wymaganego pakietu: foreach
#> 
#> Dołączanie pakietu: 'foreach'
#> Następujące obiekty zostały zakryte z 'package:purrr':
#> 
#>     accumulate, when
#> Ładowanie wymaganego pakietu: iterators
#> Ładowanie wymaganego pakietu: parallel
library(pdp)
#> 
#> Dołączanie pakietu: 'pdp'
#> Następujący obiekt został zakryty z 'package:purrr':
#> 
#>     partial
data("mtcars")

# tune:
workflow_fit <-
  workflow() %>%
  add_model(linear_reg() %>% set_engine("lm")) %>%
  add_formula(mpg ~ wt + I(1/disp^2)) %>%
  fit(data = mtcars)


# function pred.var:
pdp_pred_fun <- function(object, newdata) {
  predict(object, newdata, type = "numeric")$.pred
}

# testing:
pdp_pred_fun(workflow_fit, mtcars[1:3,])
#> [1] 21.02471 20.37330 24.74476

# Why do not ?

cl <- makeCluster(4) 
doParallel::registerDoParallel(cl)

workflow_partial <-
  partial(workflow_fit,
          pred.var = "disp",
          pred.fun = pdp_pred_fun,
          train = mtcars,
          parallel = TRUE,
          paropts = list(.packages = "earth"))
#> Error in {: task 1 failed - "niestosowalna metoda dla 'predict' zastosowana do obiektu klasy "workflow""

doParallel::stopImplicitCluster()


sessionInfo()
#> R version 4.2.1 (2022-06-23 ucrt)
#> Platform: x86_64-w64-mingw32/x64 (64-bit)
#> Running under: Windows 10 x64 (build 22000)
#> 
#> Matrix products: default
#> 
#> locale:
#> [1] LC_COLLATE=Polish_Poland.utf8  LC_CTYPE=Polish_Poland.utf8   
#> [3] LC_MONETARY=Polish_Poland.utf8 LC_NUMERIC=C                  
#> [5] LC_TIME=Polish_Poland.utf8    
#> 
#> attached base packages:
#> [1] parallel  stats     graphics  grDevices utils     datasets  methods  
#> [8] base     
#> 
#> other attached packages:
#>  [1] pdp_0.8.1          doParallel_1.0.17  iterators_1.0.14   foreach_1.5.2     
#>  [5] yardstick_1.1.0    workflowsets_1.0.0 workflows_1.0.0    tune_1.0.0        
#>  [9] tidyr_1.2.1        tibble_3.1.8       rsample_1.1.0      recipes_1.0.1     
#> [13] purrr_0.3.4        parsnip_1.0.1      modeldata_1.0.1    infer_1.0.3       
#> [17] ggplot2_3.3.6      dplyr_1.0.10       dials_1.0.0        scales_1.2.1      
#> [21] broom_1.0.1        tidymodels_1.0.0  
#> 
#> loaded via a namespace (and not attached):
#>  [1] splines_4.2.1      R.utils_2.12.0     prodlim_2019.11.13 assertthat_0.2.1  
#>  [5] highr_0.9          GPfit_1.0-8        yaml_2.3.5         globals_0.16.1    
#>  [9] ipred_0.9-13       pillar_1.8.1       backports_1.4.1    lattice_0.20-45   
#> [13] glue_1.6.2         digest_0.6.29      hardhat_1.2.0      colorspace_2.0-3  
#> [17] htmltools_0.5.3    Matrix_1.5-1       R.oo_1.25.0        timeDate_4021.104 
#> [21] pkgconfig_2.0.3    lhs_1.1.5          DiceDesign_1.9     listenv_0.8.0     
#> [25] gower_1.0.0        lava_1.6.10        styler_1.7.0       generics_0.1.3    
#> [29] ellipsis_0.3.2     furrr_0.3.1        withr_2.5.0        nnet_7.3-17       
#> [33] cli_3.4.0          survival_3.3-1     magrittr_2.0.3     evaluate_0.16     
#> [37] R.methodsS3_1.8.2  fs_1.5.2           fansi_1.0.3        future_1.28.0     
#> [41] parallelly_1.32.1  R.cache_0.16.0     MASS_7.3-57        class_7.3-20      
#> [45] tools_4.2.1        lifecycle_1.0.2    stringr_1.4.1      munsell_0.5.0     
#> [49] reprex_2.0.2       compiler_4.2.1     rlang_1.0.5        grid_4.2.1        
#> [53] rstudioapi_0.14    rmarkdown_2.16     gtable_0.3.1       codetools_0.2-18  
#> [57] DBI_1.1.3          R6_2.5.1           lubridate_1.8.0    knitr_1.40        
#> [61] fastmap_1.1.0      future.apply_1.9.1 utf8_1.2.2         stringi_1.7.8     
#> [65] Rcpp_1.0.9         vctrs_0.4.1        rpart_4.1.16       tidyselect_1.1.2  
#> [69] xfun_0.33

Thanks for the post! Just noting the cross-posted StackOverflow post where this was answered.

1 Like

Sorry i'm new. I posted this question in two places. It took a while to get this post. Meanwhile, there was an answer. I will not be duplicating anymore. :innocent:
@simoncouch

No worries! We look in a few different places.

2 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.