Optimizing on partial AUC during model parameter tuning?

Machine Learning and Modeling
1

Rather than using AUC as my optimization parameter during hyper parameter tuning in the tidy models framework, I would like to optimize on partial AUC. I am most interested in differential parameter/tuning performance at very high levels of specificity. I've searched a bit but I'm stuck on how to use/implement such a strategy. Any quick pointers?

2

There is the pROC package, which is very good, where you can use that method (and others).

yardstick used to use pROC but does not anymore since we needed support for case weights and wanted fewer dependencies. If you are using tidymodels, you would have to make your own yardstick metric. We used to have that implemented (maybe in version 0.0.9) so there is a starting point.

3

Thanks, Max. I assume this is reading I should do (Custom metrics • yardstick), and my apologies for not finding it before I posted.

1 Like
4

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.