stacks: [model] doesn't make use of the same resampling object as the existing candidates

I'm using {stacks} for the first time, and I'm hitting an error regarding the resampling object.

Error: It seems like the new candidate member 'nnet_res' doesn't make use of the same resampling object as the existing candidates.

My reproducible example comes from the docs: https://stacks.tidymodels.org/articles/classification.html. I modified the mlp model to use tune_bayes() instead of tune_grid() as in the other models. This is the model that's giving me problems. Could someone explain why the resampling object is different?

library(tidymodels)
library(tidyverse)
library(stacks)

data("tree_frogs")

# subset the data
tree_frogs <- tree_frogs %>%
  dplyr::select(-c(clutch, latency))

set.seed(1)

tree_frogs_split <- initial_split(tree_frogs)
tree_frogs_train <- training(tree_frogs_split)
tree_frogs_test  <- testing(tree_frogs_split)

folds <- rsample::vfold_cv(tree_frogs_train, v = 5)

tree_frogs_rec <- 
  recipe(reflex ~ ., data = tree_frogs_train) %>%
  step_dummy(all_nominal(), -reflex) %>%
  step_zv(all_predictors())

tree_frogs_wflow <- 
  workflow() %>% 
  add_recipe(tree_frogs_rec)

ctrl_grid <- control_stack_grid()


rand_forest_spec <- 
  rand_forest(
    mtry = tune(),
    min_n = tune(),
    trees = 500
  ) %>%
  set_mode("classification") %>%
  set_engine("ranger")

rand_forest_wflow <-
  tree_frogs_wflow %>%
  add_model(rand_forest_spec)

rand_forest_res <- 
  tune_grid(
    object = rand_forest_wflow, 
    resamples = folds, 
    grid = 10,
    control = ctrl_grid
  )

# from original example

# nnet_spec <-
#   mlp(hidden_units = tune(), penalty = tune(), epochs = tune()) %>%
#   set_mode("classification") %>%
#   set_engine("nnet")
# 
# nnet_rec <- 
#   tree_frogs_rec %>% 
#   step_normalize(all_predictors())
# 
# nnet_wflow <- 
#   tree_frogs_wflow %>%
#   add_model(nnet_spec)
# 
# nnet_res <-
#   tune_grid(
#     object = nnet_wflow, 
#     resamples = folds, 
#     grid = 10,
#     control = ctrl_grid
#   )

nnet_spec <-
  mlp(hidden_units = tune(), penalty = tune(), epochs = tune()) %>%
  set_mode("classification") %>%
  set_engine("nnet")

nnet_rec <- 
  tree_frogs_rec %>% 
  step_normalize(all_predictors())

nnet_wflow <- 
  tree_frogs_wflow %>%
  add_model(nnet_spec)

par_nnet <- dials::parameters(nnet_wflow)

nnet_res <-
  tune_bayes(
    object = nnet_wflow, 
    param_info = par_nnet,
    resamples = folds, 
    control = control_bayes(verbose=TRUE,
                            save_pred = TRUE,
                            save_workflow = TRUE,
                            no_improve = 10,
                            seed = 1)
  )


tree_frogs_model_st <- 
  # initialize the stack
  stacks() %>%
  # add candidate members
  add_candidates(rand_forest_res) %>%
  add_candidates(nnet_res) %>%
  # determine how to combine their predictions
  blend_predictions() %>%
  # fit the candidates with nonzero stacking coefficients
  fit_members()

Thanks for noting this! This should now be fixed on the main branch of stacks!

More discussion here: https://github.com/tidymodels/stacks/issues/47

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.