I have been looking at the latest updates for tidymodels around smoothing the probabilities. I have what might seem like a very basic question. If i have a model where on training produces a sensitivity 60% and recall of 90%. If i then put this model into production and it produces a score on an item of 95%. Is this the true probability.
Im not a huge probability expert but was wondering when the model makes its prediction of 95% does it factor in the precision/recall of the model from the training.
Is it worth if I know the prior probability of the disease/failures/rate in the population to use bayes on top of the probability outputted by the model to give true probabilities.
P(Disease|Test) = (P(Test|Disease) * P(Disease)) / P(Test)
Thank you for your time