Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [accuracy]

In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset.

In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset. It is defined as:

Accuracy = Number of correct predictions / Total number of predictions

In binary classification we can further define it in terms of positives and negatives:

Accuracy = (TP + TN) / (TP + TN + FP + FN)
398 questions
98
votes
4 answers

Advantages of AUC vs standard accuracy

I was starting to look into area under curve(AUC) and am a little confused about its usefulness. When first explained to me, AUC seemed to be a great measure of performance but in my research I've found that some claim its advantage is mostly…
aidankmcl
  • 1,083
  • 1
  • 8
  • 6
55
votes
5 answers

Is it always better to use the whole dataset to train the final model?

A common technique after training, validating and testing the Machine Learning model of preference is to use the complete dataset, including the testing subset, to train a final model to deploy it on, e.g. a product. My question is: Is it always…
pcko1
  • 3,910
  • 1
  • 14
  • 29
41
votes
8 answers

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an accuracy of 83% on training set and 83% on test set.…
EitanT
  • 519
  • 4
  • 3
31
votes
1 answer

What is a LB score in machine learning?

I was going through an article on kaggle blogs. Repeatedly, the author mentions 'LB score' and 'LB fit') as a metric for effectiveness of machine learning (along with cross validation (CV) score). With a research for the meaning of 'LB' I spent…
user345394
  • 505
  • 1
  • 4
  • 8
27
votes
4 answers

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification Classification Report : precision recall f1-score support 0 1.00…
user10296606
  • 1,784
  • 5
  • 17
  • 31
25
votes
3 answers

How do you manage expectations at work?

With all the hoopla around Data Science, Machine Learning, and all the success stories around, there are a lot of both justified, as well as overinflated, expectations from Data Scientists and their predictive models. My question to practicing…
neuron
  • 664
  • 1
  • 6
  • 9
22
votes
2 answers

How to increase accuracy of classifiers?

I am using OpenCV letter_recog.cpp example to experiment on random trees and other classifiers. This example has implementations of six classifiers - random trees, boosting, MLP, kNN, naive Bayes and SVM. UCI letter recognition dataset with 20000…
Mika
  • 323
  • 1
  • 2
  • 7
16
votes
1 answer

Train Accuracy vs Test Accuracy vs Confusion matrix

After I developed my predictive model using Random Forest I get the following metrics: Train Accuracy :: 0.9764634601043997 Test Accuracy :: 0.7933284397683713 Confusion matrix [[28292 1474] …
Pedro Alves
  • 367
  • 2
  • 3
  • 11
13
votes
1 answer

How to know if a model is overfitting or underfitting by looking at graph

Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy) And please do enlighten me about the concept of underfitting as well.
Nikhil.Nixel
  • 319
  • 1
  • 2
  • 9
13
votes
2 answers

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which two accuracies/errors amoung…
A.B
  • 316
  • 1
  • 3
  • 12
12
votes
1 answer

Balanced Accuracy vs. F1 Score

I've read plenty of online posts with clear explanations about the difference between accuracy and F1 score in a binary classification context. However, when I came across the concept of balanced accuracy, explained e.g. in the following image…
Ric S
  • 257
  • 2
  • 11
12
votes
3 answers

What are the disadvantages of accuracy?

I have been reading about evaluating a model with accuracy only and I have found some disadvantages. Among them, I read that it equates all errors. How could this problem be solved? Maybe assigning costs to each type of failure? Thank you very much…
PicaR
  • 304
  • 2
  • 13
11
votes
3 answers

Inverse Relationship Between Precision and Recall

I made some search to learn precision and recall and I saw some graphs represents inverse relationship between precision and recall and I started to think about it to clarify subject. I wonder the inverse relationship always hold? Suppose I have a…
tkarahan
  • 422
  • 5
  • 14
11
votes
7 answers

I got 100% accuracy on my test set,is there something wrong?

I got 100% accuracy on my test set when trained using decision tree algorithm.but only got 85% accuracy on random forest Is there something wrong with my model or is decision tree best suited for the dataset provided. Code: from…
Harigovind Valsakumar
  • 113
  • 1
  • 1
  • 8
11
votes
3 answers

Relationship between KS, AUROC, and Gini

Common model validation statistics like the Kolmogorov–Smirnov test (KS), AUROC, and Gini coefficient are all functionally related. However, my question has to do with proving how these are all related. I am curious if anyone can help me prove these…
Steven
  • 111
  • 1
  • 1
  • 3
1
2 3
…
26 27