Hi.
It's this code block below. Note that I ran this very code block further up in my script. It runs fine. I should have made a function but was in a rush having been held up so much fighting with R's ram limitations.
What I'm doing here is predicting on my training data so as to make some visualizations with the predictions later on. So, I had to make some transformations to the training data so it's in the same format as the data passed to train earlier:
```{r model_predictions, echo=F, message=F, warning=F}
# (Me desperately trying to get things to run with samples of data) Transform original data to right format to predict
# model_img_data <- churn_data %>% sample_n(500000)
# model_img_data <- churn_data %>% sample_n(10000)
# clear memory for remaining steps
rm(list = setdiff(ls(), c("model_img_data", "quadratic_model")))
## make month just a categorical not ordered factor
model_img_data <- model_img_data %>% mutate(month = as.character(month)) %>%
mutate_at(vars(matches("v_")), funs(as.numeric)) %>%
mutate(auto_renewal_flag = as.numeric(auto_renewal_flag))
## make dummy vars
dummy <- caret::dummyVars(~ month + product_pnl_line_name + shopper_region_1_name + hosting_provider,
data = model_img_data, fullRank = F, sep = ".")
dummy_df <- predict(dummy, model_img_data) %>% as.data.frame()
model_img_data <- c(model_img_data, dummy_df) %>% data.frame() %>%
mutate(count_domains_x_hosting_provider = v_count_domains * hosting_provider.Top.Ten.Hosting.Competitor.on.Venture) %>%
mutate(quadratic_term_tenure = tenure_months^2)
# predict onto original data for visualizing probability
preds <- predict(quadratic_model,
newdata = model_img_data,
type = "prob")
modeling_data <- model_img_data %>% mutate(Predicted_Probability_Churn = preds$X1)
rm(list = setdiff(ls(), "modeling_data"))
So I'm able to fit the model but not predict.
Yes, that is correct.
I think that pasting my code block above might be a diversion to the issue. On it's own section c which includes the block above is not particularly big or challenging for r to work with. It's only when I try to run all the sections together that I hit the dreaded cannot allocate memory error. This is even after clearing memory with rm()