Add SHAP tutorial for Transformers

[lisa-sousa]

Summary

Adds a new SHAP-based explainability tutorial for Transformer models, extending the existing SHAP coverage (currently used for Random Forests and CNNs) to transformer architectures.

Changes

• New notebook(s)/content under xai-for-transformer/ demonstrating SHAP applied to a transformer model
• Updates to support loading a pretrained transformer model for the tutorial

⚠️ Notes for the merger @ema265

1. Pin shap version in requirements
   requirements_xai-for-transformer.txt needs shap==0.52.0 pinned. Please also check whether requirements_xai-for-cnn.txt and requirements_xai-for-random-forest.txt can be pinned to the same version — if so, update them too and re-run those notebooks end-to-end to confirm nothing breaks with the newer SHAP release before merging.

2. Enable automatic model weights download
   The transformer notebook currently expects model weights to be available locally/manually placed. Model weights will be published as part of a GitHub Release — once that release is live, switch on/uncomment the automatic download step (pointing at the release asset URL) so users don't have to source weights manually.

3. Double-check reported train/test accuracy for the loaded transformer model
   When running the intro notebook, I get:

   • TRAIN accuracy: 0.9498
   • TEST accuracy: 0.8776

   using balanced accuracy. Can you confirm whether the notebook's original reported numbers were computed with balanced accuracy or a different metric (plain accuracy, F1, etc.)? If the notebook text states different figures, we should either update the text to match, or check if there's a metric/data-split mismatch causing the discrepancy.

Testing checklist

• Notebooks in xai-for-transformer/ run end-to-end with shap==0.52.0
• Notebooks in xai-for-cnn/ and xai-for-random-forest/ re-tested with pinned SHAP version
• Model weights auto-download verified against the GitHub Release once published
• Accuracy figures in notebook text reconciled with actual output (confirm metric used)

[Copilot]

🟡 Not ready to approve

The PR introduces shap usage in code, but the transformer requirements file does not include/pin shap==0.52.0 (and SHAP versions are inconsistent across tutorials), which blocks reproducible execution as described in the PR notes.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR extends the transformer tutorial utilities to support SHAP-based explainability workflows (model evaluation with balanced accuracy and helper plotting for SHAP token attributions), aligning the transformer track with existing SHAP coverage in other tutorials.

Changes:

• Added evaluate_model() to compute balanced accuracy for transformer text classification pipelines.
• Added plot_shap_values() to compare mean SHAP token attributions between train/test splits.
• Introduced new dependencies in the transformer utils module (shap, balanced_accuracy_score).

File summaries

File	Description
xai-for-transformer/utils.py	Adds SHAP evaluation/plotting helpers and related imports for the transformer SHAP tutorial workflow.

Review details

• Files reviewed: 1/4 changed files
• Comments generated: 4
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[donatella-cea]

Really nice and easy to follow notebook!
I have a few suggestions on the figure and some small clarifications.

[ema265]

Hi @lisa-sousa, I started on merger note #1. I pinned shap==0.52.0 in the transformer requirements, then tested it end-to-end, and a few things came up I'd like to align on before changing anything else:

1. shap 0.52.0 requires Python ≥ 3.12. On 3.11 the pin makes pip install -r requirements_xai-for-transformer.txt fail (the newest shap for 3.11 is 0.51.0). So this effectively moves the transformer track to 3.12, and bumping the CNN/RF requirements to 0.52.0 would force those tracks to 3.12 as well. Are we OK moving them to 3.12?

2. requirements_xai-for-transformer.txt is missing datasets. The notebook imports it (to load dair-ai/emotion), so it won't run from the requirements alone. I can add it if that's fine.

3. token_type_ids crash. With the pinned transformers==4.51.3 and the hosted DistilBERT weights, the pipeline fails with DistilBertForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids' (the saved tokenizer lists token_type_ids, which DistilBERT doesn't accept). It's fixable by re-saving the tokenizer in the weights Release, or a one-line workaround in the notebook (tokenizer.model_input_names = ["input_ids", "attention_mask"]). Which transformers version did you test the notebook with? That would tell me whether it's just a version difference on my side.

The SHAP part itself works fine on 0.52.0 (PartitionExplainer, the text masker, and the plots all run), so the pin is the right version. I just wanted to agree on the Python-3.12 move and the token_type_ids fix before touching CNN/RF. Let me know how you'd like to proceed.

[ema265]

On #3 (accuracy check): it's just the metric, not a data/split mismatch. On the same test split I get:

• plain accuracy: 0.9295
• balanced accuracy: 0.8776 (your number)
• weighted F1: 0.929

So the 0.9498 / 0.8776 the notebook prints are balanced accuracy (that's what utils.evaluate_model computes), while the 92.95% in the training summary is plain accuracy. Both are correct, same split.

The notebook was labeling them just "accuracy", so I relabeled the prints to "balanced accuracy" and added a short note explaining why we use it (the emotion classes are imbalanced). Pushed to the branch (76711dc).

[ema265]

@donatella-cea when you have a moment, could you publish the draft weights Release?

Task #2 on this PR (automatic download of the fine-tuned DistilBERT weights) is ready in the notebook, but it is blocked because the "DistilBERT emotion weights" release (tag distilbert-emotion-weights) is still a draft, so its download URL returns 404. Once it is published the notebook's download step should work as is, since the URL already matches the tag.

Publishing it also unblocks the "Load the fully fine-tuned model" cell in notebook 1, so one publish covers both notebooks.

You can find it under the repo's Releases tab, named "DistilBERT emotion weights". Thanks!

[lisa-sousa]

@donatella-cea when you have a moment, could you publish the draft weights Release?

Task #2 on this PR (automatic download of the fine-tuned DistilBERT weights) is ready in the notebook, but it is blocked because the "DistilBERT emotion weights" release (tag distilbert-emotion-weights) is still a draft, so its download URL returns 404. Once it is published the notebook's download step should work as is, since the URL already matches the tag.

Publishing it also unblocks the "Load the fully fine-tuned model" cell in notebook 1, so one publish covers both notebooks.

You can find it under the repo's Releases tab, named "DistilBERT emotion weights". Thanks!

Hi @ema265 I published the model weights as a new release. I'm following our release versioning, hence the new release is v2.1.0: https://github.com/HelmholtzAI-Consultants-Munich/XAI-Tutorials/releases/tag/v2.1.0

[lisa-sousa]

Hi @lisa-sousa, I started on merger note #1. I pinned shap==0.52.0 in the transformer requirements, then tested it end-to-end, and a few things came up I'd like to align on before changing anything else:

1. **shap 0.52.0 requires Python ≥ 3.12.** On 3.11 the pin makes `pip install -r requirements_xai-for-transformer.txt` fail (the newest shap for 3.11 is 0.51.0). So this effectively moves the transformer track to 3.12, and bumping the CNN/RF requirements to 0.52.0 would force those tracks to 3.12 as well. Are we OK moving them to 3.12?

2. **`requirements_xai-for-transformer.txt` is missing `datasets`.** The notebook imports it (to load `dair-ai/emotion`), so it won't run from the requirements alone. I can add it if that's fine.

3. **token_type_ids crash.** With the pinned `transformers==4.51.3` and the hosted DistilBERT weights, the pipeline fails with `DistilBertForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'` (the saved tokenizer lists `token_type_ids`, which DistilBERT doesn't accept). It's fixable by re-saving the tokenizer in the weights Release, or a one-line workaround in the notebook (`tokenizer.model_input_names = ["input_ids", "attention_mask"]`). Which transformers version did you test the notebook with? That would tell me whether it's just a version difference on my side.

The SHAP part itself works fine on 0.52.0 (PartitionExplainer, the text masker, and the plots all run), so the pin is the right version. I just wanted to agree on the Python-3.12 move and the token_type_ids fix before touching CNN/RF. Let me know how you'd like to proceed.

Hi @ema265
regarding your questions:

1. Yes, please pin Python version 3.12 for all requirements (RF, CNNs, Transformers), so we have the same Pyhton version everywhere.
2. Yes, please add. Also in case any other requirement is missing. Please pin versions for all requirements (this can be updated when other notebooks are merged).
3. I used: transformers==5.13.0

[ema265]

Hi @lisa-sousa, status from the merger side. Thanks for the answers and for publishing the weights.

#1 pin shap 0.52.0: done. I pinned shap==0.52.0, bumped transformers to 5.13.0 (that cleared the token_type_ids crash, so it was purely the version, thanks for the pointer), and added the missing deps (datasets, pandas, requests), all pinned. I also added a "Requires Python 3.12" marker to all three requirements files (transformer, CNN, RF) as you asked. I re-tested the SHAP notebook end-to-end on a clean Python 3.12 env: full run, no errors, and it reproduces the published accuracy (TRAIN 0.9498, TEST 0.8776).

#2 weights auto-download: done. I pointed the download at the new v2.1.0 release and verified it from a clean state (the notebook downloads, unzips, and loads the model).

#3 accuracy: done earlier. The prints are balanced accuracy (from utils.evaluate_model), relabeled with a short note on why we use it.

Testing checklist:

• Transformer notebooks with shap 0.52.0: checked. Of the notebooks in the folder only the SHAP one uses shap, and it runs clean on 3.12.
• CNN and RF re-tested with pinned SHAP: I left this unchecked on purpose. Per your note the SHAP version alignment happens once all notebooks are integrated, so I only added the 3.12 marker to their requirements and did not bump or re-run them. Flagging it as your integration item so the empty box is not read as forgotten.
• Weights auto-download and accuracy reconciled: both checked.

Before I merge: Donatella left two review threads that are still open, and since they are both on content you authored, you might want to take a look before I merge:

• the SHAP figure (shap_partition_explainer.png) readability notes (panels 2/4/5, "Obtain" to "Visualize", and so on)
• the notebook text tweaks (the "acending" typo, the "number of explained texts" wording, the base value clarification sentence, and the bar plot mean wording)

It would be good to have these resolved first. The figure is best updated on your side, but I am happy to apply the small text fixes myself if you would prefer.

[lisa-sousa]

@ema265 thanks for the addressing all those points.

I still have to update the notebook content according to Donatellas comments and I also reworked the full video slides and reda the docs content, so I will tailor the notebook accordingly. I'll let you and Donatella know when I updated the notebook.

[lisa-sousa]

@donatella-cea could you please have a look at the updated transformer notebook? I’ve addressed your comments and aligned the content with the new Read the Docs structure for SHAP.

In addition, could you briefly review the updated SHAP content in the Random Forest and CNN notebooks? These have also been revised to follow the new structure and now include additional explainers: KernelSHAP for the Random Forest and PartitionExplainer for the CNN.

@ema265 once Donatella has approved the changes, this should be ready to merge. Before merging, could you please double-check that all three notebooks still run correctly with the updated requirements? Thanks :)

[donatella-cea]

@donatella-cea could you please have a look at the updated transformer notebook? I’ve addressed your comments and aligned the content with the new Read the Docs structure for SHAP.

In addition, could you briefly review the updated SHAP content in the Random Forest and CNN notebooks? These have also been revised to follow the new structure and now include additional explainers: KernelSHAP for the Random Forest and PartitionExplainer for the CNN.

@ema265 once Donatella has approved the changes, this should be ready to merge. Before merging, could you please double-check that all three notebooks still run correctly with the updated requirements? Thanks :)

Everything looks good to me! @ema265, as Lisa mentioned, you can go on with the merging after double-checking that the notebooks run with the updated requirements. Thanks!

[lisa-sousa]

@ema265

Regarding google collab section, could you try out this version:

# Move to Colab's temporary working directory
%cd /content

# Remove any existing copy and clone the specified course branch
!rm -rf XAI-Tutorials
!git clone --depth 1 --branch main https://github.com/HelmholtzAI-Consultants-Munich/XAI-Tutorials.git

Like this the student should not be ofrced to share their google drive for the course purpose and with --depth 1 it should create a shallow copy of the branch such that less data needs to be downloaded (important for on-site courses where the bandwidth might be liited).

Thank you!