Why does the SHAP library in python seem to be so poorly coded/maintained, and how do I get a working environment?

Question

I suppose this is a general question involving dependency conflicts. Without fail, every time I attempt to include the shap library into my project to examine explainability, it creates python environment-destroying dependency conflicts.

What I've tried:

1. Installing the shap library into an existing project conda environment X. This creates irreparable dependency conflicts

2. Manually resolving dependency conflicts X. There are too many packages to manage that shap requires to be feasible manually

3. Creating a stripped-down new environment exclusively for testing shapley values

4. Installing shap with conda-forge

5. Installing tensorflow with conda-forge (I still need to load the stored model) X. This creates irreparable dependency conflicts

6. Creating a new environment like above, only with pip installation

7. Installing tensorflow and shap X. Irreparable dependency conflicts

I would provide the conflicts I'm getting, but they vary so much in each trial that it would not be helpful.

I don't mind if tensorflow runs slowly without my GPU for shapley testing, so no need to install cudatoolkit or cudnn

Is there an existing docker image, or fully setup environment available that doesn't have dependency conflicts that I'm able to use online?

Answer 1

It seems like you want to vent. The first part of your title is not necessary (and the answer is because it is free). Would you rather have nothing at all ?

Regarding your question, it is a bit too vague as it doesn't include one specific problem. Regarding the usage of shap + tensorflow: my solution is generally to let conda solve the problems at the beginning. Generally speaking I don't install a new library in an existing project. That is I build an environnement with everything needed at the beginning... I know it is not always possible, but it is good practice as it will require you to think a bit about where you are going.

If you actually want to install a new library that mean creating a new environnement with conda solving all dependency problems. (yes it means that another library can be downgraded and can be a headache).