Synthetic Sequences Disentangled Factorization

The second stage of the synthetic-sequences workflow applies the project's general latent-strategy methodology to the empirical synthetic setting. It replaces the entangled conditional model with an explicit factorization into a router over latent strategies and a latent-conditioned generator.

Relation To Methodology

This note focuses on how the synthetic-sequences experiment instantiates the method. The reusable method itself lives in:

Loss interventions such as baseline-normalized reconstruction, token-weighted reconstruction, router marginal KL, and posterior variance remain methodology notes rather than experiment-setting notes.

Stage-2 Construction

In the synthetic setting, the disentangling trainer combines three model roles:

a router $p_\phi(z \mid x)$ ,
a latent-conditioned generator $p_\psi(y \mid x, z)$ ,
a training-time posterior model $q_\xi(z \mid x, y)$ .

The experiment wiring expands the prompt format to expose the latent position and trains the factorized system against either gold synthetic supervision or model-generated supervision. The experiment-level package is documented in path:experiments/synthetic_sequences/disentangled/README.md.

Source Model Choices

The current setup supports two source-model lines:

entangled_run: start from a task-adapted entangled checkpoint,
pretrained_model: start directly from a raw pretrained model.

These choices define the main contrast between entangled-initialized and direct-pretrained disentangling experiments.

Supervision Choices

The synthetic setting supports two supervision sources:

groundtruth-data: use canonical task records with known gold strategies,
model-generated: use one sampled solution per input from the frozen entangled model and cache the resulting canonicalized records.

This matters because the project can study both disentangling gold strategy structure and disentangling the behavior distribution actually realized by the entangled initializer.

Latent Variants

The synthetic setting currently supports both:

discrete latents represented by learned latent tokens,
continuous latents injected at a designated latent position in the prompt.

The experiment-level distinction between these modes is documented in path:experiments/synthetic_sequences/disentangled/docs/latent_types_and_modes.md.