from-entangled-v2 continuous-v2 Results

Note

This is a partial-collection snapshot based on path:experiments/synthetic_sequences/disentangled/analysis/aggregates/synthetic-pretrained-disentangled-from-entangled-v2-continuous-v2/. The matched continuous-v1 collection is used only through direct dim=8 vs dim=64 matched-cell comparisons. Older synthetic-pretrained-disentangled-from-entangled-v{X} collections remain figure-layout references only.

Collection State

total planned cells: 36
COMPLETED_READY: 33
RUNNING: 3
locally analyzable completed cells: 33

Cells not included in the current quantitative snapshot:

sorting_algorithms: beta0p005_const_pv0_twr0p5 is still running
sorting_algorithms: beta0p01_lin100_pv0_twr0p5 is still running
base_conversion: beta0_const_pv0 is still running

Current Quantitative Snapshot

From data/final_snapshot.csv:

analyzable completed runs: 33
mean sampled test final-answer accuracy: 0.9948
mean router-sampled probe accuracy: 0.6581
mean posterior-mu probe accuracy: 0.8625
mean baseline-normalized reconstruction loss: 0.7004
mean strategy-match rate: 0.9931
mean unique-strategy rate: 0.9149

Per-loss-setting means:

Loss setting	Final accuracy	Router probe	Posterior probe	Norm recon	Strategy match	Unique strategy
`beta1_const_pv0_unnorm`	`0.9993`	`0.6007`	`0.8284`	`0.0316`	`0.9984`	`0.9245`
`beta0_const_pv0`	`0.9665`	`0.9028`	`0.9995`	`0.0000`	`0.9562`	`0.8419`
`beta0p005_const_pv0`	`1.0000`	`0.6107`	`0.8312`	`1.0402`	`1.0000`	`0.9240`
`beta0p01_lin100_pv0`	`1.0000`	`0.6034`	`0.8146`	`1.0405`	`1.0000`	`0.9242`
`beta0p005_const_pv0_twr0p5`	`1.0000`	`0.6300`	`0.8828`	`1.0439`	`1.0000`	`0.9342`
`beta0p01_lin100_pv0_twr0p5`	`1.0000`	`0.6331`	`0.8414`	`1.0437`	`1.0000`	`0.9348`

Top And Bottom Current Runs By Router Probe

Top current runs:

Task	Loss setting	Final accuracy	Router probe	Posterior probe	Norm recon	Strategy match
`multidigit_addition`	`beta0_const_pv0`	`0.9075`	`0.9630`	`1.0000`	`0.0000`	`0.8991`
`list_summation`	`beta0_const_pv0`	`0.9898`	`0.9239`	`1.0000`	`0.0000`	`0.9884`
`linear_equation_solving`	`beta0_const_pv0`	`0.9451`	`0.9226`	`1.0000`	`0.0000`	`0.9405`
`grid_pathfinding`	`beta0_const_pv0`	`0.9942`	`0.8796`	`1.0000`	`0.0000`	`0.9664`
`sorting_algorithms`	`beta0_const_pv0`	`0.9960`	`0.8250`	`0.9973`	`0.0001`	`0.9867`
`multidigit_addition`	`beta1_const_pv0_unnorm`	`0.9989`	`0.7843`	`0.9976`	`0.0232`	`0.9986`

Bottom current runs:

Task	Loss setting	Final accuracy	Router probe	Posterior probe	Norm recon	Strategy match
`sorting_algorithms`	`beta1_const_pv0_unnorm`	`0.9997`	`0.4235`	`0.8093`	`0.0238`	`0.9954`
`sorting_algorithms`	`beta0p005_const_pv0`	`1.0000`	`0.4428`	`0.7584`	`1.0364`	`1.0000`
`sorting_algorithms`	`beta0p01_lin100_pv0`	`1.0000`	`0.4448`	`0.7621`	`1.0356`	`0.9998`
`linear_equation_solving`	`beta1_const_pv0_unnorm`	`0.9997`	`0.5274`	`0.8672`	`0.0414`	`0.9997`
`linear_equation_solving`	`beta0p01_lin100_pv0`	`1.0000`	`0.5329`	`0.7979`	`0.9713`	`1.0000`
`linear_equation_solving`	`beta0p005_const_pv0`	`1.0000`	`0.5383`	`0.8561`	`0.9701`	`1.0000`

Observations By Design Question

ELBO anchor versus low-KL settings inside `continuous-v2`

The completed continuous-v2 rows do not reproduce the continuous-v1 ranking. Inside this partial dim=64 snapshot, the zero-beta anchor remains the strongest completed setting on router probe (0.9028) and posterior probe (0.9995), but it also has much lower final-answer accuracy (0.9665) and unique-strategy rate (0.8419) than the other completed settings. The ELBO anchor improves markedly relative to its dim=8 predecessor and now sits near the low-beta family on router probe (0.6007) and posterior probe (0.8284).

Matched TWR versus no-TWR inside `continuous-v2`

The completed constant-beta matched TWR deltas are mixed. grid_pathfinding and linear_equation_solving still show positive router-probe and posterior-probe deltas from adding TWR, while base_conversion, list_summation, and multidigit_addition stay near zero or regress slightly on router probe. The completed linear-warmup matched TWR deltas are smaller overall: positive on grid_pathfinding, linear_equation_solving, and multidigit_addition, negative on list_summation and base_conversion.

Router-side versus posterior-side behavior

Posterior-side decodability remains less fragile than router-side decodability. Even when router probe falls into the 0.42-0.54 range, the corresponding posterior probe often stays higher, for example sorting_algorithms / beta1_const_pv0_unnorm (0.4235 router, 0.8093 posterior) and linear_equation_solving / beta0p01_lin100_pv0 (0.5329 router, 0.7979 posterior). The beta0_const_pv0 rows are the main exception, where router and posterior both saturate near 1.

Effect of increasing `continuous_latent_dim` from `8` to `64`

The matched dim=64 - dim=8 comparison is not uniformly positive:

beta1_const_pv0_unnorm improves on average by +0.1581 router probe and +0.2744 posterior probe, with slightly lower normalized reconstruction.
beta0_const_pv0 is nearly unchanged on probe metrics on the matched five completed tasks, but its mean final-answer accuracy is lower by -0.0258.
beta0p005_const_pv0 and beta0p01_lin100_pv0 regress slightly on router probe on average and worsen strongly on normalized reconstruction (+0.4566 and +0.6984 respectively).
Both completed TWR settings regress more clearly relative to dim=8: beta0p005_const_pv0_twr0p5 averages -0.1658 router probe and +1.0351 normalized reconstruction, while beta0p01_lin100_pv0_twr0p5 averages -0.1197 router probe and +0.9553 normalized reconstruction.

The largest positive router-probe delta in the matched table is multidigit_addition / beta1_const_pv0_unnorm at +0.2781. The strongest normalized-reconstruction improvement for dim=64 is base_conversion / beta1_const_pv0_unnorm at -0.0340, while most non-ELBO matched rows move in the opposite direction.

Baseline-normalized reconstruction loss in unnormed is incorrect

When the --baseline-normalized-reconstruction flag is False as in beta1_const_pv0_unnorm, the stored loss/val/baseline_normalized_reconstruction_loss is not directly comparable to the normalized settings. Low values in that row should therefore be treated as descriptive of the stored artifact, not as a fair cross-setting normalized-reconstruction comparison.

Figures

Collection status by task and loss setting. Caption: Current partial-collection status over the 6 x 6 task-by-loss-setting design. The current snapshot contains 33 completed cells and 3 running cells.

Final metric heatmaps by task and loss setting. Caption: Completed-cell heatmaps for final answer accuracy, router probe, posterior probe, strategy-match rate, unique-strategy rate, and baseline-normalized reconstruction.

Router probe summary by loss setting. Caption: Mean final router-probe accuracy for each loss setting, aggregated over the currently completed tasks.

Alignment dynamics by task and loss setting. Caption: Validation final-answer accuracy, router-sampled probe accuracy, and posterior-mu probe accuracy over training.

Objective dynamics by task and loss setting. Caption: Baseline-normalized reconstruction, token-weighted reconstruction, KL prior, and total objective over training.

Final normalized reconstruction versus router probe. Caption: Final baseline-normalized reconstruction loss versus final router-sampled probe accuracy across the completed cells.

Final router probe versus posterior probe. Caption: Final router-sampled probe accuracy versus final posterior-mu probe accuracy across the completed cells.

Matched with-TWR versus without-TWR final metrics within the current collection. Caption: Within-collection matched bars for router probe, posterior probe, final accuracy, and normalized reconstruction.

Matched router-probe dynamics for no-TWR versus +TWR settings. Caption: Validation router-probe accuracy over training for the matched constant-beta=0.005 and linear-warmup beta=0.01 comparisons.

Matched latent-dimension deltas by task and loss setting. Caption: Heatmaps of dim=64 - dim=8 deltas for router probe, posterior probe, final accuracy, and baseline-normalized reconstruction on the currently matched cells.

Matched dim-8 versus dim-64 means by loss setting. Caption: Mean matched-cell values by loss setting for dim=8 and dim=64, comparing router probe, posterior probe, baseline-normalized reconstruction, and final accuracy.

Matched dim-8 versus dim-64 final-metric scatter plots. Caption: One point per matched completed cell, showing dim=8 on the x-axis and dim=64 on the y-axis for the main final metrics.

Matched router-probe dynamics by task and loss setting for dim-8 versus dim-64. Caption: Task-by-setting router-probe trajectories for the matched continuous-v1 and continuous-v2 cells.