When Shift Happens - Confounding is to Blame

---

Steps to reproduce synthetic dataset results.

---

To run the experiments on synthetic data, install the dependices: xgboost, npeet, scikitlearn along with basic modules such as numpy, pandas, and matplotlib.

1. Change directory to code/synthetic
2. To reproduce the results in Figure 2 of the main paper, run: python introfigure.py, , resulting plot will be saved to code/synthetic/results.
3. To reproduce the results in Figure 4 of the main paper, run: python varyingS.py, resulting plot will be saved to code/synthetic/results.
4. To reproduce the results in Figure D16 of the appendix, run: python varyingnoise.py, resulting plot will be saved to code/synthetic/results.
5. To reproduce the results in Figure D17 of the appendix, run: python varyingU.py, resulting plot will be saved to code/synthetic/results.
6. To reproduce the results in Figures D18 - D21 of the appendix, run: python synthetic_xgboost.py and python synthetic_xgboost.py by toggling the variable named 'which' to run experiments for low and high overlapping confounding support. Resulting plots will be saved to code/synthetic/results.

---

Steps to reproduce real-world dataset results.

---

To run the experiments on real-world datasets, setup the runtime environment following the instructions at https://github.com/socialfoundations/causal-features/tree/main. Additionally install npeet library for mutual information estimation.

1. Change directory to code/realworld
2. To reproduce the results from Table 2, cd into experiments_causal and run: python confoundingshiftmeasures_data.py
3. To reproduce the resulst from Table 3, Figures C2, C3, C6-C15 run bash examples/script.sh by uncommenting the respective script. Create a JSON with results. Readymade results are already present in results/hyperparamsetting. To visualize the results, run: python shiftmeasuresplots_realworld_hyperparametersetting.py
4. To perform hyperparameter tuning, run: bash examples/ray_train_example.sh. Save the best hyperparameters to examples/hps.py (this step is already performed and best hyperparameters are saved). To reproduce the resulst from Figures C4, C5, and Table C2, run bash examples/script.sh by uncommenting the respective script. Create a JSON with results. Readymade results are already present in results/hyperparamsetting_besthps. To visualize the results, run: python shiftmeasuresplots_realworld_best_hyperparametersetting.py
5. To reproduce the results in D22-D24, run: python plotresults_pmaood.py and python plotresults_rho.py

If you use this project, please cite:

@inproceedings{
reddy2026when,
title={When Shift Happens - Confounding Is to Blame},
author={Abbavaram Gowtham Reddy and Celia Rubio-Madrigal and Rebekka Burkholz and Krikamol Muandet},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=sFjxg8cyJS}
}