Questions tagged [logistic-regression]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

The logistic function is $$ f(x) = \frac{1}{1+e^{-x}}, $$ which maps real numbers to $(0,1)$. One common use of the logistic function is logistic regression, which is a standard method of quantifying the effect of a set of predictors $\{X_1, ..., X_p\}$ on a binary outcome, $Y$. The model can be written as $$ P(Y=1|X) = f(\beta_0 + \beta_1X_1 + ... + \beta_p X_p)$$ The logistic regression model has the nice property that the exponentiated regression coefficients can be interpreted as odds ratios associated with a one unit increase in the predictor.

A second use of the logistic function (but unrelated to logistic regression) is the logistic distribution, which has $f(x)$ as its quantile function.

693 questions
30
votes
3 answers

How to get p-value and confident interval in LogisticRegression with sklearn?

I am building a multinomial logistic regression with sklearn (LogisticRegression). But after it finishes, how can I get a p-value and confident interval of my model? It only appears that sklearn only provides coefficient and intercept. Thank you a…
hminle
  • 401
  • 1
  • 4
  • 4
28
votes
4 answers

Scikit-learn: Getting SGDClassifier to predict as well as a Logistic Regression

A way to train a Logistic Regression is by using stochastic gradient descent, which scikit-learn offers an interface to. What I would like to do is take a scikit-learn's SGDClassifier and have it score the same as a Logistic Regression here.…
hlin117
  • 675
  • 1
  • 8
  • 11
26
votes
2 answers

Text categorization: combining different kind of features

The problem I am tackling is categorizing short texts into multiple classes. My current approach is to use tf-idf weighted term frequencies and learn a simple linear classifier (logistic regression). This works reasonably well (around 90% macro F-1…
24
votes
3 answers

Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

I have a very basic question which relates to Python, numpy and multiplication of matrices in the setting of logistic regression. First, let me apologise for not using math notation. I am confused about the use of matrix dot multiplication versus…
GhostRider
  • 353
  • 1
  • 2
  • 8
23
votes
4 answers

Is logistic regression actually a regression algorithm?

The usual definition of regression (as far as I am aware) is predicting a continuous output variable from a given set of input variables. Logistic regression is a binary classification algorithm, so it produces a categorical output. Is it really a…
joews
  • 526
  • 1
  • 4
  • 10
22
votes
3 answers

Linear regression with non-symmetric cost function?

I want to predict some value $Y(x)$ and I am trying to get some prediction $\hat Y(x)$ that optimizes between being as low as possible, but still being larger than $Y(x)$. In other words: $$\text{cost}\left\{ Y(x) \gtrsim \hat Y(x) \right\} >>…
asPlankBridge
  • 323
  • 2
  • 6
21
votes
1 answer

What does it mean to "share parameters between features and classes"

When reading this paper there is a line which says "linear classifiers do not share parameters among features and classes." What is the meaning of this statement? Does it mean that linear classifiers such as logistic regression need features…
20
votes
5 answers

Choose binary classification algorithm

I have a binary classification problem: Approximately 1000 samples in training set 10 attributes, including binary, numeric and categorical Which algorithm is the best choice for this type of problem? By default I'm going to start with SVM…
IgorS
  • 5,444
  • 11
  • 31
  • 43
17
votes
5 answers

Should I use a decision tree or logistic regression for classification?

I am working on a classification problem. I have a dataset containing equal numbers of categorical variables and continuous variables. How do I decide which technique to use, between a decision tree and logistic regression? Is it right to assume…
Arun
  • 717
  • 3
  • 10
  • 27
16
votes
2 answers

Binary classification model for unbalanced data

I have a dataset with the following specifications: Training dataset with 193,176 samples with 2,821 positives Test Dataset with 82,887 samples with 673 positives There are 10 features. I want to perform a binary classification (0 or 1). The issue…
tejaskhot
  • 3,935
  • 7
  • 20
  • 18
15
votes
2 answers

Does scikit-learn use regularization by default?

I just fitted a logistic curve to some fake data. I made the data essentially a step function. data = -------------++++++++++++++ But when I look at the fitted curve, the slope is very small. The function that best minimizes the cost function,…
sebastianspiegel
  • 871
  • 3
  • 11
  • 16
13
votes
1 answer

What is the difference in xgboost binary:logistic and reg:logistic

What is the difference in R in xgboost between binary:logistic and reg:logistic? Is it only in evaluation metric? If yes, how does RMSE on binary classification compare to error rate? Is the relationship between the metrics more or less monotonic,…
user2530062
  • 297
  • 2
  • 8
12
votes
1 answer

What's the relationship between an SVM and hinge loss?

My colleague and I are trying to wrap our heads around the difference between logistic regression and an SVM. Clearly they are optimizing different objective functions. Is an SVM as simple as saying it's a discriminative classifier that simply…
Simon
  • 1,026
  • 10
  • 10
12
votes
2 answers

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, is the only difference between an SVM and logistic regression the criterium to choose the boundary? Apparently, SVM…
12
votes
1 answer

How do I implement the sigmoid function in Octave?

so given that the sigmoid function is defined as hθ(x) = g(θ^(T)x), how can I implement this funcion in Octave given that g = zeros(size(z)) ?
Shuryu Kisuke
  • 223
  • 1
  • 2
  • 5
1
2 3
46 47