Contributions

An initial paper submission can focus on the problem of learning a disentangled factorization of a language model whose distribution is implicitly an entangled mixture of multiple "modes" of reasoning. Leave applications to exploration, interpretability, etc for later papers or versions of the paper. Keep paper self-contained, focusing on this general problem.

Contributions:

• Proposing and formalizing the problem; identifying key challenges
• A benchmark dataset: collection of tasks with solutions generated by distribution with mixture of latent strategies.
• A class of methods to solve the problem
• Experimental evaluation of methods, identifying key requirements for solutions to such a problem and failure modes
• Proof of concept for applications to directed active exploration (perhaps with some simple formalism or theory)