A Functional Information Perspective on Model Interpretation
Published in ICML, 2022
Recommended citation: Itai Gat, Nitay Calderon, Roi Reichart and Tamir Hazan https://proceedings.mlr.press/v162/gat22a.html
Abstract
Contemporary predictive models are hard to interpret as their deep nets exploit numerous complex relations between input elements. This work suggests a theoretical framework for model interpretability by measuring the contribution of relevant features to the functional entropy of the network with respect to the input. We rely on the log-Sobolev inequality that bounds the functional entropy by the functional Fisher information with respect to the covariance of the data. This provides a principled way to measure the amount of information contribution of a subset of features to the decision function. Through extensive experiments, we show that our method surpasses existing interpretability sampling-based methods on various data signals such as image, text, and audio.bibtex
@inproceedings{gat2022functional,
author = {Itai Gat and
Nitay Calderon and
Roi Reichart and
Tamir Hazan},
editor = {Kamalika Chaudhuri and
Stefanie Jegelka and
Le Song and
Csaba Szepesv{\'{a}}ri and
Gang Niu and
Sivan Sabato},
title = {A Functional Information Perspective on Model Interpretation},
booktitle = {International Conference on Machine Learning, {ICML} 2022, 17-23 July
2022, Baltimore, Maryland, {USA}},
series = {Proceedings of Machine Learning Research},
volume = {162},
pages = {7266--7278},
publisher = ,
year = {2022},
url = {https://proceedings.mlr.press/v162/gat22a.html},
timestamp = {Tue, 12 Jul 2022 17:36:52 +0200},
biburl = {https://dblp.org/rec/conf/icml/GatCRH22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}