Router-Generator Factorization

This note describes the concept of a latent-variable router-generator factorization and gives reference language for writing about it in the paper. The core object is a pair $(r(z|x), g(y|x,z))$ : a router that maps an input to a distribution over latent strategies, and a generator that produces a response conditioned on the input and latent strategy.

Core Language

Use latent-variable router-generator factorization for the full model object: the pair $(r(z|x), g(y|x,z))$ together with the latent marginalization over $z$ .

Use router-generator factorization as the short form after the full term has been introduced.

Use router for $r(z|x)$ .

Use generator for $g(y|x,z)$ , with strategy-conditioned generator as the more precise first-use form.

Reference phrasing:

We seek a latent-variable router-generator factorization $(r(z|x), g(y|x,z))$ of a pretrained conditional language model $p_\theta(y|x)$ , where the router $r(z|x)$ defines a distribution over latent strategies for an input $x$ , and the strategy-conditioned generator $g(y|x,z)$ generates a response $y$ conditioned on both the input and the latent strategy.

Formal Referent

In the exact mathematical formulation, a router-generator pair $(r,g)$ is a latent-variable factorization of a conditional distribution $p(y|x)$ when

\int_{\mathcal Z} r(z|x) g(y|x,z)\,d\mu(z) = p(y|x).

In the empirical setting, the learned factorization is approximate: marginalizing over $z$ should preserve the base model's conditional distribution closely, while the latent variable should expose useful strategy structure. When precision matters, distinguish the exact formal factorization from the learned approximate factorization.

Router

The router $r(z|x)$ selects or represents a strategy given a problem. More precisely, it defines a conditional distribution over latent strategy values for each input $x$ .

The router language emphasizes the operational role: the input is routed into latent strategy space before generation. It is related to mixture-of-experts language, but here the routed object is a latent strategy value, not necessarily a separate expert network.

Use care with two possible misreadings:

Router does not imply hard discrete assignment. In continuous-latent models, it is a distribution over a continuous latent space.
Router does not imply separate expert networks. In this project, the generator may be a shared conditioned model rather than a bank of independent experts.

Bridge for variational-model readers:

In variational-model terminology, the router $r(z|x)$ plays the role of a conditional prior over the latent variable.

Use conditional prior when connecting to VAE notation, but prefer router in the paper's main prose because it communicates the strategy-selection role.

Generator

The generator $g(y|x,z)$ generates the solution or response conditioned on both the input and the latent strategy. At first use, prefer strategy-conditioned generator. After the definition is established, generator is concise and clear.

Use care with two possible misreadings:

Generator does not imply a separate full model for each strategy. It can be a shared model conditioned on $z$ .
Generator should not be called an expert unless the architecture actually uses separate expert modules.

Bridge for variational-model readers:

In VAE terminology, the generator $g(y|x,z)$ plays the role of the decoder.

Use decoder when connecting to VAE notation, but prefer generator in the paper's main prose. In language-model contexts, "decoder" is overloaded by decoder-only Transformer terminology and can obscure the sequence-generation role.

Factorization

Use factorization as the primary noun for the model object. It is mathematically appropriate because the paper rewrites a conditional distribution through a latent marginalization over a router and generator.

The main advantage of factorization is that it is neutral. It says that $(r,g)$ induces the same or approximately the same observable conditional law, without claiming by itself that the latent variable is meaningful, unique, causal, or disentangled. That neutrality is useful because a central message of the paper is that many latent-variable factorizations can preserve $p(y|x)$ , and the scientific problem is to select one with strategy structure.

Useful modifiers:

latent-variable factorization emphasizes the mathematical structure;
router-generator factorization emphasizes the named components;
latent-variable router-generator factorization is the full first-use form;
strategy-aligned factorization describes a factorization whose latent variable tracks the intended strategy structure;
disentangled factorization describes a factorization that separates strategies in the intended sense.

Avoid bare factorization near autoregressive language-model notation unless the referent is already clear, since language models also factorize $p(y|x)$ token by token as $\prod_t p(y_t|x,y_{<t})$ .

Decomposition is a looser prose synonym for factorization. It can help intuition, but it is less mathematically precise because it does not by itself imply preservation of the original conditional law under latent marginalization.

Strategy-aligned describes a factorization whose latent variable tracks the intended strategy structure.

Disentangled describes the property of separating strategies. Phrases such as disentangled factorization or strategy-disentangled factorization are useful when the intended sense has been defined. The standalone noun disentanglement should be used carefully because it invites comparison to beta-VAE and FactorVAE-style unsupervised disentanglement.

Strategy-structure recovery names an empirical or theoretical success claim: the learned latent recovers the strategy structure of the response distribution. This names an outcome, not the model object.

Unmixing is vivid but narrower than factorization. It can imply that the mixture components are already well-defined and simply need to be recovered.

Usage Across The Paper

In the title and abstract, prefer latent-variable router-generator factorization or router-generator factorization if space allows. Pair it with a strategy phrase so readers know what kind of latent structure matters.

In the problem statement, define the exact latent-variable factorization with the integral equation, then introduce the harder target: a strategy-aligned factorization.

In methodology, use router and strategy-conditioned generator for the architecture, and use variational terms such as conditional prior and decoder as bridges to standard latent-variable terminology.

In experiments, describe outcomes with strategy-aligned factorization, strategy-structure recovery, within-input strategy separation, and cross-input semantic stability.

In figures and captions, keep terms short:

router
generator
router probe
strategy-conditioned generation
strategy-aligned latent
router-generator factorization

Terms That Need Care

strategy selector: too deterministic and too discrete for continuous routers.
gating network: too tied to mixture-of-experts architecture and separate expert modules.
conditional prior: technically correct but too VAE-internal for the main conceptual story.
decoder: useful as a bridge, but overloaded in language-model architecture.
expert: misleading unless the model really has separate expert modules.
probabilistic decomposition: broadly reasonable, but less precise and less memorable than router-generator factorization.