Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [explainable-ai]

Use for questions about explainable artificial intelligence (AI), which aims at understanding, interpreting, and explaining the decisions that have been made by complex AI systems

The goal of Explainable Artificial Intelligence (AI) is to better understand the decisions that are made by complex AI systems.

In contrast to common machine learning approaches, which are often a "black box", explainable AI systems should be modeled in a way that allows humans to retrace how the system arrived at a certain decision.

The overarching goal of Explainable AI is to build trust in complex AI systems and increase the overall accountability and transparency of these systems.

68 questions
20
votes
5 answers

What is the difference between explainable and interpretable machine learning?

O’Rourke says that explainable ML uses a black box model and explains it afterwards, whereas interpretable ML uses models that are no black boxes. Christoph Molnar says interpretable ML refers to the degree to which a human can understand the cause…
Funkwecker
  • 585
  • 1
  • 5
  • 12
9
votes
2 answers

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one hot encoding (for better interpretation in my…
Shahab Kazemi
  • 103
  • 4
7
votes
1 answer

Evaluating machine learning explainers?

I'm working on a project where multiple machine learning explainers (LIME and SHAP, potentially more coming) are applied to pre-trained models (neural networks) to help explain the predictions of those 'black boxes'. The explainers assign each…
Notna
  • 171
  • 1
5
votes
2 answers

Different feature importance results between DNN, Random Forests and Gradient Boosted Decision Trees

I've been modeling metabolite data with 3 different regressor models. I get similar results from running feature importance with Random Forest model and Gradient Boosted Decision Trees (where I used the scikit-learn built-in feature importance), but…
Orion
  • 51
  • 1
5
votes
1 answer

Shapley values without intercept (or without `expected_value`)

I have a model and I want to derive its interpretability by using feature contributions. In the end, I want to have some contribution per feature such that the sum of contributions equals the prediction of the model. One approach may be to use…
David Masip
  • 5,981
  • 2
  • 23
  • 61
5
votes
1 answer

How is the "base value" of SHAP values calculated?

I'm trying to understand how the base value is calculated. So I used an example from SHAP's github notebook, Census income classification with LightGBM. Right after I trained the lightgbm model, I applied explainer.shap_values() on each row of the…
David293836
  • 197
  • 1
  • 6
5
votes
1 answer

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following the SHAP Python library. Now I'd like learn the logic behind DE more. From the relevant paper it is not clear to me how SHAP values are…
mlee_jordan
  • 153
  • 1
  • 8
5
votes
1 answer

Is it valid to compare SHAP values across models?

Let's say I have three models: a random forest with 100 trees a random forest with 1000 trees an xgboost model. I can rank the importance of my features on my dataset for each model using SHAP, and compare relative importance across models. What…
DKL
  • 78
  • 5
3
votes
1 answer

How to evaluate the "importance" of a variable in a function

Let's say that we have $$f(x,y,z) = x/k - (y/k) ((z - x/k)/(z - y/k))$$ $$k = constant \in ]0,1[$$ And I need to show in some way that the variable $x$ is more important in some metric that I don't know which one could be good. I thought about …
Allan Araujo
  • 51
  • 2
3
votes
2 answers

What is a "surrogate model"?

While reading about model explainability and model accountability, the term surrogate model keeps appearing. I had an idea about what it is but it does not seem to make sense anymore: What is a surrogate models? Why are surrogate models good for…
Carlos Mougan
  • 6,011
  • 2
  • 15
  • 45
3
votes
3 answers

SHAP Explanations in case of repeated train/test split

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that follow (respectively a summary plot Figure A and a…
Forinstance
  • 133
  • 5
3
votes
1 answer

What is the meaning of an empty SHAP graph in Explainable AI?

Using Python, I created a neural network to perform predictions on a binary class dataset (e.g. will a passenger survive the Titanic?). I am using the SHAP package to explain individual predictions. For all of the instances in this dataset, the…
caspar
  • 31
  • 1
3
votes
1 answer

Is multicollinarity a problem when interpreting SHAP values from an XGBoost model?

I'm using an XGBoost model for multi-class classification and is looking at feature importance by using SHAP values. I'm curious if multicollinarity is a problem for the interpretation of the SHAP values? As far as I know, XGB is not affected by…
hideonbush
  • 31
  • 1
2
votes
0 answers

How SHAP value explains contribution of features for outliers event?

I'm trying to understand and experiment with how the SHAP value can explain behaviour for each outlier events (rows) and how it can be related to shap.force_plot(). I already created a simple synthetic dataset with 7 outliers. I didn't get how 4.85…
Mario
  • 335
  • 5
  • 18
2
votes
1 answer

Shapley contribution when coalition is 0

I am exploring Shapley for channel attribution based on [here][1] Consider C1, C2, C3, C4 as 4 channels in question. Some of the coalition does not have value, such as (C1, C2) -> 20 (C1, C3, C4) -> 10 (C1, C2, C3, C4) -> 0 The reason being there…
Kenny
  • 121
  • 1
1
2 3 4 5