Methodology Outline

This note records the approved narrative structure for path:paper/conference-paper/sections/03-methodology.tex. The methodology section should explain how the paper parameterizes and trains a latent-variable router-generator factorization of a fixed base-model response distribution $p_\theta(y|x)$. Its central role is to bridge the problem statement and experiments: the problem statement says that marginal-preserving latent-variable factorizations are under-specified; the methodology section shows why the standard conditional ELBO does not solve that selection problem and introduces $p_\theta$-directed reconstruction as the proposed training pressure.

This is for a NeurIPS submission with a 9-page main-text limit. The methodology should therefore prioritize the architecture, the ELBO non-identification argument, and the proposed objective. Training-procedure details and objective variants should be compact, with full procedural or implementation details pushed to appendix if needed.

Main-Text Page Budget

Aim for roughly 2 pages total for the methodology section, with a soft target around 1.75 pages and a hard ceiling around 2.25 pages including figures, equations, and any algorithm box. A reasonable internal budget is 0.4–0.5 pages for the factorized Transformer parameterization and architecture figure, 0.45–0.6 pages for ELBO non-identification and the wrap-elbo-fails diagnostic, 0.7–0.9 pages for $p_\theta$-directed reconstruction and the token-weighting figure, and 0.2–0.3 pages for training procedure and objective variants. The main space risk is keeping the architecture figure, ELBO wrapfigure, token-weighting figure, and algorithm all in the main text; treat the algorithm as optional or highly compact if the section begins to exceed budget.

Related concept notes:

• Problem framing
• Router-generator factorization
• Base model and reference notation
• ELBO under-specification and the z-ignoring solution
• Base-directed reconstruction objective
• Base-directed reconstruction variants
• Metrics and frontier language
• Findings

Related figure notes:

• Architecture parameterization
• ELBO failure dynamics
• Token-level surprisal-share weighting

Section Role

The methodology section should answer: given the latent-variable factorization problem, how do we parameterize and train a factorization that selects for an informative, strategy-aligned $z$ rather than merely preserving the marginal distribution $p_\theta(y|x)$?

The section should include four ingredients:

1. The factorized Transformer architecture implementing the router $r_\phi(z|x)$, strategy-conditioned generator $g_\phi(y|x,z)$, and training-time posterior $q_\xi(z|x,y)$.
2. The conceptual and symbolic reason the standard conditional ELBO is non-identifying in the adapted-from-base setting.
3. The proposed $p_\theta$-directed reconstruction objective $\mathcal R_\theta$.
4. A compact training-procedure description emphasizing that factorization training samples responses from the base-model response distribution $p_\theta(y|x)$ and adapts a pretrained model, rather than learning the benchmark data distribution from scratch.

Approved Narrative Decisions

The section should keep the motivation primarily conceptual rather than fully empirical. The benchmark and reference strategy annotations should be introduced in the experiments section, where F1 can establish that strategy information is present in the base model before F2 describes ELBO failure.

The methodology section may still include fig:elbo-fails as a compact empirical preview near the ELBO analysis. The figure should be readable at a glance, but the surrounding text should not require the reader to understand the full benchmark setup yet. A caption or parenthetical can say that the representative controlled-benchmark run is described in the experiments section.

Do not place a boxed numbered Finding 2 in methodology before Finding 1 appears in experiments. In methodology, describe the result as an ELBO under-specification observation, failure mode, or diagnostic preview. The experiments section can later formalize F2 and refer back to the symbolic observation and fig:elbo-fails.

The experiment section should open with the benchmark setup and F1, Implicit Strategy Structure In The Base Model, then present F2, ELBO Under-Specification And Collapse, as the empirical counterpart of the methodology argument.

Proposed Visible Subsection Structure

Approved section title:

• Learning Strategy-Aligned Latent-Variable Factorizations

Approved visible subsection structure:

1. A Transformer Architecture for Router-Generator Factorization
2. ELBO Under-Specification and Latent Collapse
3. $p_\theta$-Directed Reconstruction

Use short \paragraph headings inside the third subsection if needed. Approved paragraph titles are \paragraph{Training Distribution.} and \paragraph{Objective Variants.}. Avoid adding many visible subsections unless the page budget changes.

Detailed Section Outline

3.1 A Transformer Architecture for Router-Generator Factorization

Purpose: Show how the abstract router-generator factorization is implemented in an autoregressive Transformer language model.

Reader question: What model class realizes $r_\phi(z|x)$ and $g_\phi(y|x,z)$?

Core concepts and notation:

• $p_\theta(y|x)$: fixed base model whose response distribution is being factorized.
• $\phi$: adapted router-generator parameters.
• $\xi$: training-time posterior parameters.
• $r_\phi(z|x)$: router, equivalent to the conditional prior in variational-model language.
• $g_\phi(y|x,z)$: strategy-conditioned generator.
• $q_\xi(z|x,y)$: training-time posterior used for variational training.
• $p_\phi(y|x)=\int r_\phi(z|x)g_\phi(y|x,z)\,d\mu(z)$: induced latent marginal.

Narrative content: Start from the factorization target defined in the problem statement and explain that the paper realizes this factorization by adapting a base Transformer into a shared router-generator architecture. The router reads the input and produces a distribution over $z$; the generator conditions on $x$ and $z$ while generating $y$ autoregressively; the posterior $q_\xi(z|x,y)$ is used only during training.

Figure support: Use fig:architecture-ar-compact to make clear that router and generator are roles of a shared adapted model rather than separate expert networks. The prose should reinforce the notation decisions in Router-generator factorization and Base model and reference notation.

Must establish before next subsection: The architecture class is expressive enough to preserve $p_\theta(y|x)$ while using or ignoring $z$. Therefore the training objective must solve a selection problem, not merely a fitting problem.

3.2 ELBO Under-Specification and Latent Collapse

Purpose: Explain why the standard variational objective does not identify a useful, strategy-aligned factorization of an already-capable base model.

Reader question: Why does ordinary conditional VAE training fail to solve the latent-variable factorization problem?

Core concepts and notation:

• Standard negative conditional ELBO:

$$\mathcal J_{\mathrm{cELBO}}(x,y;\phi,\xi)=\mathbb E_{z\sim q_\xi(\cdot|x,y)}[-\log g_\phi(y|x,z)]+\beta\,\mathrm{KL}(q_\xi(\cdot|x,y)\Vert r_\phi(\cdot|x)).$$

• Induced posterior $p_\phi(z|x,y)$ under the router-generator factorization.
• ELBO identity for $\beta=1$:

$$\mathcal J_{\mathrm{cELBO}}(x,y;\phi,\xi)=-\log p_\phi(y|x)+\mathrm{KL}(q_\xi(\cdot|x,y)\Vert p_\phi(\cdot|x,y)).$$

• z-ignoring factorization: $g_\phi(y|x,z)=p_\theta(y|x)$ for all $z$, with $q_\xi(z|x,y)=r_\phi(z|x)$.
• objective-level under-specification: matching $p_\theta(y|x)$ does not select a strategy-aligned latent.
• adapted-from-base optimization pathology: because $\phi$ is initialized from or near $p_\theta$, the z-ignoring solution is nearby at initialization.

Narrative content: Derive or state the ELBO identity compactly. The reader should see that the standard conditional ELBO asks the induced marginal $p_\phi(y|x)$ to match $p_\theta(y|x)$ and asks $q_\xi$ to match the posterior of whichever factorization produced that marginal. It does not ask whether $z$ is informative about $y$ beyond $x$ or whether $z$ aligns with reusable strategies.

Figure support: Include fig:elbo-fails near this discussion as a representative empirical preview. The text should describe only the minimum needed to read the figure: distributional fidelity improves, posterior-router KL collapses, and strategy alignment remains low. The full benchmark and metric definitions should come later.

Finding handling: Do not box F2 here. This subsection supports the later experimental finding F2 — ELBO Under-Specification And Collapse, which should appear after F1 in the experiments section.

Must establish before next subsection: The standard conditional ELBO can preserve observable behavior while leaving $z$ inert or unaligned. The proposed objective therefore needs to change the reconstruction pressure so that $z$ is encouraged to explain residual variation in $y$ beyond what $x$ and the z-free base model already explain.

3.3 $p_\theta$-Directed Reconstruction

Purpose: Introduce the proposed objective as the methodological response to ELBO under-specification in the adapted-from-base factorization setting.

Reader question: How does the proposed objective create pressure for $z$ to explain response variation beyond $x$?

Core concepts and notation:

• $p_\theta$-directed reconstruction objective: formal term for the objective modification.
• base-directed reconstruction: short prose term.
• z-free base-model surprisal: $b_{\theta,t}(x,y)=-\log p_\theta(y_t|x,y_{<t})$.
• z-free base-model reference scale: $c_\theta=\mathbb E_{x\sim\mathcal D_X,\;y\sim p_\theta(\cdot|x)}[b_\theta(x,y)]$.
• token-level $p_\theta$ surprisal weighting: weights derived from $b_{\theta,t}(x,y)$.
• global $p_\theta$ surprisal scale: normalization by $c_\theta$.
• $\mathcal R_\theta(x,y;\phi,\xi)$: base-relative reconstruction term.
• $\mathcal J_\theta(x,y;\phi,\xi)=\mathcal R_\theta(x,y;\phi,\xi)+\beta\,\mathrm{KL}(q_\xi(\cdot|x,y)\Vert r_\phi(\cdot|x))$: proposed training objective.
• $\alpha$: interpolation between uniform and surprisal-directed token pressure.
• $\gamma$: concentration exponent for token-level weighting.

Narrative content: State that standard reconstruction pressure predicts $y$ from $(x,z)$ but does not ask whether $z$ explains $y$ beyond $x$. The proposed $\mathcal R_\theta$ replaces the uniform reconstruction term with a base-directed reconstruction term that measures reconstruction in base-model reference units and concentrates token-level pressure where the frozen base model has high z-free surprisal.

Reference language: Because $y$ is generated left-to-right, much of the sequence can become predictable from $x$ and earlier tokens $y_{<t}$. Strategy information may be concentrated at branch points or early commitments, so a uniform reconstruction objective can spend most of its pressure on tokens where $z$ has little work to do. This is the methodological role of $p_\theta$-directed reconstruction and token surprisal weighting.

Figure support: Use fig:token-weighting to explain token-level $p_\theta$ surprisal weighting. The caption and nearby prose should use $p_\theta$, not $p_{\mathrm{base}}$, and should connect the figure to $b_{\theta,t}(x,y)$ and $a_{\theta,t}(x,y)$.

Must establish before training-procedure paragraph: The fixed base model $p_\theta$ is used in two roles: it is the response distribution being factorized and the frozen reference copy used to compute scores, weights, and normalization constants. The reference scores, token weights, and normalization constants are fixed with respect to optimization over $\phi$ and $\xi$, while $\mathcal R_\theta(x,y;\phi,\xi)$ and $\mathcal J_\theta(x,y;\phi,\xi)$ remain functions of the optimized parameters indicated after the semicolon.

Training Procedure

A brief algorithmic environment is useful if it stays compact. Its purpose is not to teach optimization; its purpose is to prevent a central misunderstanding about the training distribution. The algorithm should emphasize that factorization training samples $x\sim\mathcal D_X$ and $y\sim p_\theta(\cdot|x)$, not supervised responses from the benchmark data distribution, and that $\phi$ is an adapted router-generator parameterization initialized from or built on top of the pretrained base model.

The algorithm should make four facts visually explicit:

1. Initialize/adapt $\phi$ from the base model $p_\theta$.
2. Freeze $p_\theta$ as both target response distribution and scoring reference.
3. During factorization training, sample $x\sim\mathcal D_X$ and $y\sim p_\theta(\cdot|x)$.
4. Strategy annotations are withheld from training and used only for evaluation.

If space is tight, replace the algorithm with a short paragraph and the displayed training-distribution line:

$$x\sim\mathcal D_X,\qquad y\sim p_\theta(\cdot|x),\qquad z\sim q_\xi(\cdot|x,y).$$

Then state: all $p_\theta$ quantities are computed using a frozen reference copy, while $\phi$ denotes the adapted router-generator parameters optimized by the objective.

Objective Variants

The methodology section can briefly name the comparison set used later, without reporting results:

• standard conditional ELBO: uniform reconstruction pressure and standard variational regularizer.
• $\beta$-ELBO: low-$\beta$ conditional ELBO baseline with \beta&lt;1.
• global scale: global $p_\theta$ surprisal scale.
• token weighting: token-level $p_\theta$ surprisal weighting.
• global + token: combined base-directed variant.
• token inverse: inverse-surprisal ablation.

Do not spend main-text space on all variant details unless the experiments depend on them immediately. The important methodological point is that the base-directed reconstruction family contains two mechanisms: the global $p_\theta$ surprisal scale and token-level $p_\theta$ surprisal weighting.

Open Drafting Decisions

1. Whether to keep the current algorithm environment in the main text or compress it to prose plus the training-distribution display.
2. Whether fig:elbo-fails should remain in methodology as a wrapfigure or move to experiments while methodology keeps only the symbolic argument. Current decision: keep it in methodology as a diagnostic preview.
3. Whether fig:token-weighting is worth the main-text space or should be replaced with a compact equation-and-caption explanation. Current decision: retain it unless page pressure becomes severe.
4. Whether visible subsection titles should be exactly the recommended titles above or shortened during LaTeX drafting.

Reader State After This Section

By the end of methodology, the reader should understand:

1. how the router-generator factorization is parameterized in a Transformer language model;
2. why the standard conditional ELBO is non-identifying for the latent-variable factorization problem;
3. why the adapted-from-base setting makes z-ignoring collapse especially natural;
4. how $p_\theta$-directed reconstruction changes the reconstruction pressure;
5. what training distribution is used during factorization;
6. which objective variants will be compared empirically.