1

Let's say I build a binary classification model to predict survival on the Titanic. I then use SHAP to get feature importance for each feature. I see that the SHAP importance for the top feature, sex, is greater than 1. How can that be? Since the target variable can only take a value of 0 and 1, I would've thought SHAP values for a given feature cannot be greater than 1. How am I wrong?

from seaborn import load_dataset
from sklearn.model_selection import train_test_split
from lightgbm import LGBMClassifier
import shap
import numpy as np
import pandas as pd

#Load and curate data
titanic = load_dataset("titanic")
X = titanic.drop(["survived","alive","adult_male","who",'deck'], 1)
y = titanic["survived"]

features = X.columns
cat_features = []
for cat in X.select_dtypes(exclude="number"):
    cat_features.append(cat)
    X[cat] = X[cat].astype("category").cat.codes.astype("category")

# Split train/test data and build model
X_train, X_val, y_train, y_val = train_test_split(X,y,train_size=.8, random_state=42)

clf = LGBMClassifier(max_depth=3, n_estimators=1000, objective="binary")
clf.fit(X_train,y_train, eval_set=(X_val,y_val), early_stopping_rounds=100, verbose=100)

# Get SHAP feature importances
explainer = shap.TreeExplainer(clf)
shap_values = explainer.shap_values(X_train)
rf_resultX = pd.DataFrame(shap_values[1], columns = features)
vals = np.abs(rf_resultX.values).mean(0)
shap_importance = pd.DataFrame(list(zip(features, vals)), columns=['col_name', 'feature_importance_vals'])
shap_importance.sort_values(by=['feature_importance_vals'], ascending=False, inplace=True)
print(shap_importance)
    col_name    feature_importance_vals
1   sex         1.374387  <-- SHAP feature importance greater than 1
0   pclass      0.558869
2   age         0.436174
5   fare        0.322730
3   sibsp       0.197708
6   embarked    0.167459
7   class       0.093250
4   parch       0.027302
9   alone       0.003941
8   embark_town 0.000000
Gaurav Bansal
  • 111
  • 2

1 Answers1

3

First, SHAP values are not directed translated as probabilities, they are marginal contributions for model's output. As explained in this post, we can't interpret SHAP values from raw predictions.

Also, if you check shap.TreeExplainer

model_output : "raw", "probability", "log_loss", or model method name What output of the model should be explained. If "raw" then we explain the raw output of the trees, which varies by model.

LGBMClassifier applies a transformation in model's output, so raw predictions are not translated as probabilities as well. Therefore, it is important to consider model's output in order to interpret SHAP values correctly.

Finally, when you calculate feature importance, you calculate the average contribution for all instances in dataset, so values are not summing to 1 necessarily, because you have negative and positive contributions, and your average output is not 1 (predicting a single class).

Victor Oliveira
  • 800
  • 3
  • 10