SyntaxShap: Syntax-aware Explainability Method for Text Generation

SyntaxShap is a model agnostic attribution-based explainability method for text generation. It accounts for the syntax in the text data by extending Shapley values to consider parsing-based syntactic dependencies. We adopt a game-theoretic approach to compute the contributions of each token in the input text to the model's prediction. We scope ourselves to explaining text generation by autoregressive language models. We compare SyntaxShap to the random baseline and popular explainers for text data, including LIME, SHAP, Partition. In addition, we also extend our method to a weighted variant SyntaxShap-w.

Abstract

To harness the power of large language models in safety-critical domains we need to ensure the explainability of their predictions. However, despite the significant attention to model interpretability, there remains an unexplored domain in explaining sequence-to-sequence tasks using methods tailored for textual data. This paper introduces SyntaxShap, a local, model-agnostic explainability method for text generation that takes into consideration the syntax in the text data. The presented work extends Shapley values to account for parsing-based syntactic dependencies. Taking a game theoric approach, SyntaxShap only considers coalitions constraint by the dependency tree. We adopt a model-based evaluation to compare SyntaxShap and its weighted form to state-of-the-art explainability methods adapted to text generation tasks, using diverse metrics including faithfulness, complexity, coherency, and semantic alignment of the explanations to the model. We show that our syntax-aware method produces explanations that help build more faithful, coherent, and interpretable explanations for predictions by autoregressive models.

Citation

Consider citing our whitepaper if you want to reference our work or if you are using SyntaxShap explainability method:

@article{amara2022SyntaxShap,
    title=SyntaxShap: Syntax-aware Explainability Method for Text Generation},
    author={Amara, Kenza and Sevastjanova, Rita and El-Assady, Mennatallah },
    journal={arXiv preprint https://arxiv.org/pdf/2402.09259},
    year={2024}
}

Contribute to SyntaxShap!

Contribute to SyntaxShap. We welcome any contribution in terms of both new explainability methods and new evaluation metrics. Please check here for more details and raise issues if you have any questions.

Feel free to contact us at kenza.amara@ai.ethz.ch

Maintainers