Literature: Interpretability

Papers on interpretable internal structure in language models that are relevant to this project's longer-term interpretability motivation. Scope includes:

• supervised concept interfaces for pretrained language models;
• post hoc feature decomposition methods such as sparse autoencoders;
• papers that expose internal reasoning structure rather than only improving task accuracy.

Papers

Concept-Supervised Interfaces

• Tan et al. (2024) - Interpreting Pretrained Language Models via Concept Bottlenecks

This paper is the clearest nearby reference when the question is "what if the interpretable latent variables are predefined human concepts rather than discovered strategy modes?"

Sparse Features And Mechanistic Reasoning

• Galichin et al. (2025) - I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders

This paper is the strongest nearby reference when the question is "can we locate reasoning behavior inside an already trained model without retraining it as a VAE or bottleneck model?"

Closely Related Project Notes

• Abstract Strategy vs Solution
• Challenges of Learning a Latent Variable Factorization of Pre-trained Language Models
• Contributions