Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [auc]

66 questions
7
votes
2 answers

AUC-ROC for Multi-Label Classification

Hey guys I'm currently reading about AUC-ROC and I have understood the binary case and I think that I understand the multi-classification case. Now I'm a bit confused on how to generalize it to the multi-label case, and I can't find any intuitive…
NotoriousFunk
  • 71
  • 1
  • 1
  • 3
6
votes
1 answer

Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional neural network model for classification of the data.…
machinery
  • 236
  • 2
  • 9
6
votes
1 answer

How to choose between different models with similar results? RF, GLM and XGBoost

I am a medical doctor trying to make prediction models based on a database of approximately 1500 patients with 60+ parameters each. I am dealing with a classification problem (mortality at 1, 3, 6 and 12 months) and have made stratified splits (70…
user145725
  • 61
  • 1
5
votes
2 answers

Confused AUC ROC score

I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...). I am using a cross validation technique (to avoid over-fitting) with AUC ROC as scoring…
Ak.tech
  • 103
  • 1
  • 6
3
votes
1 answer

At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - validation - final test I do K-Fold cross validation using the combined train + validation set During that phase we can…
erotavlas
  • 130
  • 4
3
votes
2 answers

Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not been able to prove or disprove it yet. The reason…
Borun Chowdhury
  • 187
  • 7
3
votes
0 answers

Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those calculations with standards implementations (I am using…
Lucas Morin
  • 2,513
  • 5
  • 19
  • 39
3
votes
3 answers

What is AUC - ROC Curve?

AUC - ROC curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability. Is Roc the same as AUC?
user87246
3
votes
1 answer

Area Under the Precision Recall Curve

I have got the following Precision Recall Curve for a classifier I built using AutoML. Most of the Precisio-Recall curves tend to start from (0, 1) go towards (1,0). But mine is the opposite. But I feel like, similar to the ROC curve it is actually…
user77005
  • 151
  • 4
2
votes
0 answers

AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?

I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows: precision recall f1-score support …
Coldchain9
  • 159
  • 5
2
votes
2 answers

Don't understand why I get an inverse ROC curve for SVM (Python)

I build an SVM classifier but get an inverse ROC curve. The AUC is only 0.08. I've used the same datasets to build a Logistic Regression classifier and a Decision Tree classifier, and the ROC curves for them look good. Here are my codes for…
MMMMMay
  • 31
  • 3
2
votes
2 answers

Main options on how to deal with imbalanced data

As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets: Option 1: Create k-fold Cross-Validation samples randomly (or even better create k-fold samples using Stratified k-fold:…
Newbie
  • 21
  • 3
2
votes
0 answers

Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to be sick range from 0-100% (of course). What I want…
Luis Pinto
  • 31
  • 3
2
votes
2 answers

How much can the AUC improve comparing the raw dataset and the feature engineered dataset?

Let's say I put the following two datasets in the best possible model (same model for both): A raw dataset, the variables as they came just from the query. A feature-engineered dataset, with hundreds of created variables, which came from the same…
Juan Esteban de la Calle
  • 2,262
  • 7
  • 28
2
votes
0 answers

Smallest possible difference between AUC of two ranker

If there are 10 positive examples, and 90 negative examples in the test set, what is the smallest possible difference in AUC, between two rankers giving different AUC?
wrek
  • 123
  • 3
1
2 3 4 5