Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [collinearity]

26 questions
9
votes
2 answers

What is the meaning of a quadratic relation when r = 0?

A website (on page 4) says: The correlation coefficient is a measure of linear relationship and thus a value of r = 0 does not imply there is no relationship between the variables. For example in the following scatterplot which implies no…
Subhash C. Davar
  • 578
  • 4
  • 18
7
votes
2 answers

Possible harm in standardizing one-hot encoded features

While there may not be any added value in standardizing one-hot encoded features prior to applying linear models, is there is any harm in doing so (i.e., affecting model performance)? Standardizing definition: applying (x - mean) / std to make the…
thereandhere1
  • 715
  • 1
  • 7
  • 22
6
votes
3 answers

Correlation vs Multicollinearity

I have been taught to check correlation matrix before going for any algorithm. I have a few questions around the same: Pearson Correlation is for numerical variables only. What if we have to check the correlation between a continuous and…
Payal Bhatia
  • 159
  • 7
4
votes
2 answers

what is the difference in terms namely Correlation, correlated and collinearity?

A website says Correlation refers to an increase/decrease in a dependent variable with an increase/decrease in an independent variable. Collinearity refers to two or more independent variables acting in concert to explain the variation in a…
Subhash C. Davar
  • 578
  • 4
  • 18
3
votes
1 answer

How to interpret Variance Inflation Factor (VIF) results?

From various books and blog posts, I understood that the Variance Inflation Factor (VIF) is used to calculate collinearity. They say that VIF till 10 is good. But I have a question. As we can see in the below output, the rad feature has the highest…
thewhitetulip
  • 153
  • 1
  • 6
3
votes
2 answers

How to measure variable contribution to an observation in a non-linear model?

Based on my model, if I decline someone due to their score, it should be able to provide some reasoning as to which variables mainly contributed to the decision to decline. Typically in Logistic Regression models, this is a simple exercise where you…
rayven1lk
  • 361
  • 2
  • 8
3
votes
1 answer

Does Multicollinearity affect Neural Networks?

Can someone explain to me like I'm five on why multicollinearity does not affect neural networks? I've done some research and neural networks are basically linear functions being stacked with activation functions in between, now if the original…
Chukwudi Ogbonna
  • 35
  • 3
3
votes
4 answers

Multicollinearity vs Perfect multicollinearity for Linear regression

I have been trying to understand how multicollinearity within the independent variables would affect the Linear regression model. Wikipedia page suggests that only when there is a "perfect" multicollinearity, one of the independent variables would…
ak1431
  • 31
  • 2
3
votes
2 answers

Does PCA helps to include all the variables even if there is high collinearity among variables?

I have a dataset that has high collinearity among variables. When I created the linear regression model, I could not include more than five variables ( I eliminated the feature whenever VIF>5). But I need to have all the variables in the model and…
NAS_2339
  • 233
  • 2
  • 11
2
votes
2 answers

Checking linearity for a linear regression model?

I've read that there are various assumptions associated with a multiple linear regression model which you should check/validate before getting too excited about your model results. One of these is the assumption of linearity. I get that you would…
lukeweatherstone
  • 21
  • 1
2
votes
2 answers

Can GLM( generalized linear method) handle the collinearity between the predictor variables in a regression-analysis?

I'm a beginner in Machine learning and I've studied that collinearity among the predictor variables of a model is a huge problem since it can lead to unpredictable model behaviour and a large error. But, are there some models (say GLM) that are…
Bharathi
  • 277
  • 6
  • 15
2
votes
2 answers

Transforming negative correlated non linear variable to linear positive correlated variable

At my office, I am stuck in a weird situation. I am asked to perform a regression algorithm on the data, in which the target variable is continuous having values range between 0.6 to 0.9 with 8 digits of precision after the decimal. Although I know…
shivanshu dhawan
  • 178
  • 1
  • 9
2
votes
2 answers

Multicolinearity & accurate weights of predictors

Let’s suppose that the stock value of various companies is the target of my models. I have some “internal” predictors e.g. yearly sales of each company, sum of salaries at each company etc. I have some “external” predictors e.g. geographical…
Outcast
  • 1,037
  • 2
  • 11
  • 27
2
votes
1 answer

Collinearity and Outlier Removal

I am playing with a credit fraud detection dataset at Kaggle. An imbalanced dataset with about 0.1% of fraud transaction. The features are 28 PCs out from a PCA exercise done by the data publisher + time & txn amount and a class variable of 0/1 for…
Harris
  • 21
  • 3
2
votes
0 answers

Deriving VIF equation from the matrix form of Least Squares equation

I have been working through the derivation of the formula used to calculate the Variance Inflation Factor associated with a model. I am hoping to start with the Least Squares equation as defined in matrix form and find a proof that derives this,…
Erin
  • 81
  • 4
1
2