How can I calculate Sensitivity, Specificity, F1, etc. for the overall database not by class in confusion matrix


Model <- train(Movement~., tuneLength = 5, data = Database, method = 
                   trControl = trainControl(method = "cv",
                                            number = 10,
                                            savePredictions = "final",
                                            classProbs = T))
confusionMatrix(Model$pred[order(Model $pred$rowIndex),2], Database$Movement)

After running my code for my multiclass database (11 different classes), I receive the following results for Overall Statistics:

               Accuracy : 0.5808          
                 95% CI : (0.5649, 0.5965)
    No Information Rate : 0.1681          
    P-Value [Acc > NIR] : < 2.2e-16                                                
                  Kappa : 0.5301                                                 
 Mcnemar's Test P-Value : NA

but I do have more performance measures for each class

> Statistics by Class:
>                         Class AccelerationAg      Class: AccelerationNon   Class: BackGround
> Sensitivity                     0.91929825             0.00000000        0.83685801
> Specificity                     0.91368541             1.00000000        0.98676640
> Pos Pred Value                  0.46289753                    NaN        0.85758514
> Neg Pred Value                  0.99290342             0.92907801        0.98450057
> Precision                       0.46289753                     NA        0.85758514
> Recall                          0.91929825             0.00000000        0.83685801
> F1                              0.61574618                     NA        0.84709480
> Prevalence                      0.07486210             0.07092199        0.08694510
> Detection Rate                  0.06882059             0.00000000        0.07276070
> Detection Prevalence            0.14867350             0.00000000        0.08484371
> Balanced Accuracy               0.91649183             0.50000000        0.91181220

I am wondering how can I calculate other performance measures such as precision, recall, F1, sensitivity, etc for the overall database not by each class such as the accuracy and Kappa that I have already calculated for in the overall statistics. I mean, the confusionmatrix function is calculating everything that I need but by each class and I want those statistics for the overall (whole) database. For instance, as you can see above, the accuracy of my model is 0.5808 which indicates the overall accuracy of my database for prediction. I want to calculate the overall Recall, F1, Precision, Sensitivity, Specificity, etc. of my entire database not by class.
I do appreciate your help in advance,

Those are all measure that are only defined for two-class problems.

caret is restricted in that way but yardstick has methods for multiclass averaging that may do what you want.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.