Abstract

This note is the working outline and candidate paragraph for the paper abstract. It replaces the earlier abstract structure note.

Abstract Outline

1. Problem / setting: A fixed language model defines a conditional distribution $p_\theta(y|x)$ that can contain multiple solution strategies for the same problem instance. The paper should frame the difficulty as implicit, entangled strategy structure in the response distribution.
2. Goal: State the goal as learning a latent-variable router-generator factorization that exposes this structure. The router $r_\phi(z|x)$ selects a latent strategy variable and the generator $g_\phi(y|x,z)$ produces a strategy-conditioned solution trace.
3. Obstacle: Explain why the standard conditional ELBO is insufficient for strategy-structure recovery: it can match the marginal distribution while leaving $z$ inert or unaligned.
4. Method: Introduce base-directed reconstruction as a $p_\theta$-directed objective. The abstract should mention the two mechanisms only if space permits: a global base-model surprisal scale and token-level $p_\theta$ surprisal weighting.
5. Evaluation setup: Briefly name the controlled multi-strategy algorithmic benchmark and explain why it matters: strategy annotations make it possible to measure strategy alignment and cross-input semantic consistency.
6. Main findings: End with the core empirical message: base-directed reconstruction learns strategy-aligned router-generator factorizations, traces an alignment-reconstruction frontier, and shows evidence of cross-input semantic consistency via analogical agreement.

Required Messages

• The paper is about exposing latent strategy structure in a fixed or base language-model distribution, not simply improving task accuracy.
• The formal object is a latent-variable router-generator factorization, with $r_\phi(z|x)$ and $g_\phi(y|x,z)$.
• The standard conditional ELBO is under-specified for this problem because marginal fidelity does not imply strategy-aligned use of $z$.
• The proposed methodology is base-directed reconstruction: reconstruction pressure is defined relative to the base model $p_\theta$.
• The benchmark is controlled because strategy annotations are needed to evaluate strategy alignment and cross-input semantic consistency.
• The strongest empirical claim should be phrased as strategy-aligned factorization, with measured evidence for analogical agreement rather than an overbroad claim of fully recovering true strategies.

Candidate Abstract

Language models often represent multiple ways of solving the same problem, but this strategy structure is implicit in the conditional distribution $p_\theta(y|x)$ rather than exposed as a controllable variable. We study whether a fixed base model's strategy-entangled response distribution can be factored into a latent-variable router-generator model, where a router $r_\phi(z|x)$ selects a latent strategy variable and a generator $g_\phi(y|x,z)$ produces a strategy-conditioned solution trace. A standard conditional ELBO is not enough for this goal: it can preserve the marginal distribution over solutions while allowing the latent variable to remain inert or unaligned with strategy. We therefore introduce base-directed reconstruction, a $p_\theta$-directed objective that replaces uniform reconstruction pressure with a global base-model surprisal scale and token-level $p_\theta$ surprisal weighting. This objective measures reconstruction in base-model reference units and concentrates pressure on tokens where the base model has high residual surprisal. On a controlled multi-strategy benchmark, base-directed reconstruction learns strategy-aligned factorizations that expose reusable cross-input strategy structure while maintaining marginal fidelity.

Open Decisions

• Decide whether the first sentence should emphasize language models broadly or fixed base-model distributions more narrowly.
• Decide how much space the abstract should spend on ELBO under-specification versus the positive method.
• Decide whether to mention pretrained Qwen / LoRA adaptation explicitly or keep the abstract architecture-agnostic.
• Decide whether the final sentence should say "while maintaining marginal fidelity" or use a more cautious phrase such as "without optimizing for a different task distribution."