Applying post probability adjustments to models that use SMOTE

[john.smith]

Hi,

I am reading the webpage on tidymodels where its mentioned that the probabilities of a trained model
using up/down/SMOTE sampling will not be representative of the actual probabilities in the dataset. I have seen there are some adjustments that can be made post modelling such as Platt Scaling. I have a number of questions, I was hoping you can give me help on

• Is the adjustment on the probabilities done after you apply your model to the test set and to any new and unseen data going forward. Im thinking about putting a model into production.
• Is this something that can be done within the tidymodels framework

Thank you for your time

[Max]

If you did something like that, it would basically undo the subsampling. That's all subsampling methods (like SMOTE) do: make the class probability estimates less consistent with the class prevalence in the actual data.

Postprocessing is on our roadmap and probably won't start until later this year.