Interpreting Pretrained Language Models via Concept Bottlenecks

Citation

• Authors: Zhen Tan, Lu Cheng, Song Wang, Bo Yuan, Jundong Li, and Huan Liu
• Year: 2024
• Venue: PAKDD 2024
• arXiv: arXiv:2311.05014
• URL: https://doi.org/10.1007/978-981-97-2259-4_5
• Citation count: 11 (Scopus via Arizona State University publication page, retrieved 2026-04-20)

One-Line Takeaway

This paper adapts concept bottleneck models to pretrained language models by inserting a concept layer $x \to z \to c \to y$, then uses LLM-assisted concept augmentation to improve the interpretability-utility tradeoff.

Technical Core

• Start from a fine-tuned PLM encoder $$z = f_\theta(x),$$ then add a concept projector $$\hat{c} = p_\psi(z),$$ and finally a task head $$\hat{y} = g_\phi(\hat{c}).$$
• The resulting model is a Concept-Bottleneck-Enabled PLM (CBE-PLM) with computation $$x \to z \to c \to y.$$
• Unlike classic image CBMs, the paper keeps the PLM representation large and inserts a separate concept projector rather than shrinking the main encoder layer directly.
• Joint training combines label prediction and concept prediction losses so the concept layer stays predictive rather than decorative.

Method

• Benchmark standard CBM training strategies for PLMs: independent, sequential, and joint training.
• Introduce C3M: ChatGPT-guided Concept augmentation with Concept-level Mixup.
• Use ChatGPT both to expand the concept set and to produce noisy pseudo-labels for unlabeled concept data.
• Train with concept-level MixUp so the model can learn from both clean human-annotated concepts and noisier machine-generated concept labels.
• Evaluate on text classification datasets with human concept annotations such as CEBaB and IMDB.

Empirical Evaluation

• Among the standard CBM-style training strategies, joint training gives the best interpretability-utility tradeoff for PLMs in the reported experiments.
• C3M improves concept prediction and downstream task performance relative to the simpler supervised bottleneck setups.
• The paper highlights test-time concept intervention as both a diagnostic tool and a way to correct some wrong task predictions.

Assumptions And Limits

This is a supervised concept interface, not an unsupervised latent-discovery method. Its success depends on having a meaningful concept ontology, reasonable concept labels, and a task where a concept bottleneck is appropriate. It is also mainly a classification paper, so it does not address latent-variable control of open-ended generation directly.

Relevance To This Project

This paper is useful mainly as a contrast class. It shows one clean route to interpretability: choose human-readable concepts first, then force the model to communicate through them.

That is not the repo's goal, but it sharpens the distinction:

• concept bottlenecks use predefined semantic handles;
• this repo aims to recover latent strategy structure that may not be known or enumerable in advance;
• the repo also cares about generation-time factorization, not only classifier explanations.

Comparison Notes

Galichin et al. (2025) is a useful foil. Concept bottlenecks are supervised and architecture-level: the model is trained to expose specific concept variables. Sparse-autoencoder features are discovered post hoc inside an already trained model. For this repo, the concept-bottleneck line is closer to "human-labeled strategy probes," while the SAE line is closer to "discover reasoning structure inside a pretrained model after the fact."

Related Notes

• I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
• Abstract Strategy vs Solution
• Challenges of Learning a Latent Variable Factorization of Pre-trained Language Models
• Contributions