This R package provides a torch efficient extension of the AUM for multi-class classification, which was created to be used as a surrogate loss for optimizing Area Under the ROC Curve (AUC) in supervised binary classification and changepoint detection problems. # Installation
if(!requireNamespace("remotes"))install.packages("remotes")
::install_github("tdhock/aum") remotes
To extend AUM for multi-class classification , the OvR(One-Versus-Rest) approach was used , to average results , both macro and micro averages were used and there is one funtion for each averaging method( see Scikit-learn for more details and equations) Given a labels tensor:
= torch::torch_tensor(c(1,3,2,2),dtype=torch::torch_long()) four_labels
and a prediction tensor:
= torch::torch_tensor(matrix(c(0.4, 0.3, 0.3,
four_pred 0.2, 0.1, 0.7,
0.5,0.2,0.3,
0.3,0.4,0.3),
ncol=3,byrow=TRUE))
We could : - Plot ROC curves using either macro or micro averaging - Compute AUC either macro or micro - Compute the AUM either macro or micro, micro AUM supports weighting . ### ROC curves ROC macro:
::Draw_ROC_curve_macro(four_pred,four_labels)) (torchMAUM
ROC micro:
::Draw_ROC_curve_micro(four_pred,four_labels)) (torchMAUM
### AUC
value AUC macro:
::ROC_AUC_macro(four_pred,four_labels)) (torchMAUM
torch_tensor0.805556
[ CPUFloatType{} ]
AUC micro:
::ROC_AUC_micro(four_pred,four_labels)) (torchMAUM
torch_tensor0.734375
[ CPUFloatType{} ]
AUM macro:
::ROC_AUM_macro(four_pred,four_labels)) (torchMAUM
torch_tensor0.0277778
[ CPUFloatType{} ]
AUM micro: The AUM micro has two variants: either weighted or unweighted . To use the weighted version we need to pass the counts of each class in the dataset to the function :
unweighted:
::ROC_AUM_micro(four_pred,four_labels,counts=NULL)) (torchMAUM
torch_tensor0.05
[ CPUFloatType{} ]
weighted:
=torch::torch_tensor(c(1,2,1))
counts::ROC_AUM_micro(four_pred,four_labels,counts=counts)) (torchMAUM
torch_tensor0.0388889
[ CPUFloatType{} ]