1

Question: Why is my predictor value (continuous) perfectly correlated with my logit value (when testing logistic regression model assumptions)?

Code:

# linearity in the logit for continuous var: check the linear relationship bw continuous predictor var and the logit of the outcome: inspect scatter plot bw each predictor and logit value
# Select only continuous predictors
glm_h2_a1 <- df_master_aus %>%
  dplyr::select(c(c_ns2)) 
predictors <- colnames(df_master_aus)
# bind the logit and tidying the data for plot
glm_h2_a1 <- glm_h2_a1 %>%
  mutate(logit = log(probabilities/(1-probabilities))) %>%
  gather(key = "predictors", value = "predictor.value", -logit)

# create the Scatter Plots:
ggplot(glm_h2_a1, aes(logit, predictor.value))+
  geom_point(size = 0.5, alpha = 0.5) +
  geom_smooth(method = "loess") + 
  theme_bw() + 
  facet_wrap(~predictors, scales = "free_y")

Image: enter image description here

Note: More complex model with additional predictors do not all show such linearity: enter image description here

In_cognito
  • 11
  • 2

1 Answers1

0

$$ \text{logit}=\hat\beta_0+\hat\beta_1x\\ \text{cor}(x, \text{logit})\\ =\text{cor}(x, \hat\beta_0+\hat\beta_1x)\\ =\text{cor}(x, \hat\beta_1x) $$

If the estimated slope coefficient $\hat\beta_1>0$, then $\text{cor}(x,\hat\beta_1x)=\text{cor}(x,x)=1$.

Consequently, this does not test any assumptions: by definition, the linear prediction of your logistic regression model has a perfect (perhaps negative) correlation with the feature. If you understand why the feature in a simple linear regression is perfectly correlated (perhaps negatively) with the predictions, the same idea applies here.

Dave
  • 3,841
  • 1
  • 8
  • 23
  • Thank you! I am not familiar with the "feature" term. What is the feature in a simple linear regression and why is it perfectly correlated with preditions? I thought that in running this test I would be testing the linear relationship between the continuous predictor and hte logit of the outcome (and the assumption is that is is lineaR). – In_cognito Mar 16 '23 at 06:22
  • A feature is a predictor ($x$) variable. In simple linear regression, there is just the one feature, so the prediction is $\hat y=\hat\beta_0+\hat\beta_1x$. Do you see why there is perfect correlation ($\pm 1$) between $x$ and $\hat y?$ Let’s start there. – Dave Mar 16 '23 at 10:37
  • I believe so - as every 1-unit increase in x increases/decreases y by a certain amount? So is this a case of perfect correlation as there is only one predictor in the model? I aask as my models with more than one predictor showed a less perfectly linear correlation. – In_cognito Mar 16 '23 at 21:39
  • Less perfect correlation between each feature and the logit value? – Dave Mar 16 '23 at 22:07
  • Sorry, that wasn't very clear: the image is a better explainer of what I mean re the fact that some trends aren't perfectly linear: see post above as couldn't post hre. – In_cognito Mar 17 '23 at 00:24
  • In the case with one predictor $x$, you are calculating $\text{cor}(x, \hat\beta_0+\hat\beta_1x)$. If $x$ is not constant, then $\left\vert\text{cor}(x, \hat\beta_0+\hat\beta_1x)\right\vert = 1$ no matter what, and $\text{cor}(x, \hat\beta_0+\hat\beta_1x) = \text{sign}(\hat\beta_1)$. Does this make sense so far? – Dave Mar 17 '23 at 00:29
  • I believe so, although am still unsure why this changes for adding more predictors into the model (so perhaps understanding is not good yet)... – In_cognito Mar 21 '23 at 10:08
  • Now what can you say about $\text{cor}(x_1, \hat\beta_0+\hat\beta_1x_1+\hat\beta_2x_2)?$ – Dave Mar 21 '23 at 11:29