Apr24 Best Loss Dynamics

Note

This is an observational deep-dive for the Apr24 loss setting with the highest mean matched strategy router score across tasks and adaptation modes. The selected setting is dist_plus_tokcc_a0p5_b0p01_d50_lin50, which combines distribution-normalized reconstruction with normalized concentration-corrected token surprisal-share weighting at alpha 0.5, beta 0.01, and a delayed-50 linear beta schedule.

Selection

The selected loss setting has mean analysis/router_probe_times_strategy_match_rate = 0.8532 across its 18 complete task-by-adaptation runs. This is the top row in the aggregate loss-setting summary.

Adaptation mode	Matched strategy router score	Final answer accuracy	Router probe	Strategy match rate	Posterior-mu probe	Baseline-normalized reconstruction
`full`	0.9021	0.9901	0.9200	0.9807	0.9999	0.0268
`lora`	0.8927	0.9951	0.9032	0.9883	0.9976	0.0416
`reinit`	0.7647	0.8778	0.9203	0.8299	0.9965	0.0453

By task, the strongest mean matched strategy router score is on grid_pathfinding and the weakest means are on sorting_algorithms, linear_equation_solving, and base_conversion.

Task	Matched strategy router score	Final answer accuracy	Router probe	Strategy match rate	Baseline-normalized reconstruction
`grid_pathfinding`	0.9126	1.0000	0.9167	0.9955	0.0399
`list_summation`	0.8890	0.9502	0.9387	0.9475	0.0315
`multidigit_addition`	0.8549	0.9251	0.9290	0.9225	0.0319
`base_conversion`	0.8244	0.9407	0.8835	0.9337	0.0622
`linear_equation_solving`	0.8229	0.9105	0.9170	0.8952	0.0294
`sorting_algorithms`	0.8152	0.9995	0.9021	0.9035	0.0325

Validation Dynamics

The validation curves show full and lora reaching high matched strategy router score much earlier than reinit. The reinit runs have high final router-probe decodability in this loss setting, but their matched score is lower because the sampled generations match the intended strategy less often.

Router-Sampled Z-Space

The router-sampled z-space frames plot PCA projections of sampled prior z values, colored by inferred generation strategy. Columns are selected training checkpoints and rows are adaptation modes within each task.

Router-sampled z-space animated training dynamics for the best Apr24 loss setting

Posterior-Mu Z-Space

The posterior-mu z-space frames plot PCA projections of posterior mean vectors, colored by ground-truth strategy. Posterior probe accuracy is near ceiling for almost every cell, so these plots mainly show the geometry and timing of the posterior clustering rather than whether posterior strategy labels are decodable at all.

Posterior-mu z-space animated training dynamics for the best Apr24 loss setting

Smoothed posterior-mu z-space animated training dynamics for the best Apr24 loss setting

Run-Level Notes

The highest-scoring individual cells are list_summation/full at 0.9406, linear_equation_solving/full at 0.9363, grid_pathfinding/reinit at 0.9263, and grid_pathfinding/full at 0.9219. The lowest-scoring cells are all reinit except for base_conversion/full: linear_equation_solving/reinit at 0.6278, sorting_algorithms/reinit at 0.7010, base_conversion/reinit at 0.7377, and multidigit_addition/reinit at 0.7415.

The data backing this page are in path:experiments/synthetic_sequences/disentangled/analysis/aggregates/continuous_cvae-Apr24-reconstruction-stack_20260424/data/dist_plus_tokcc_a0p5_b0p01_d50_lin50_final_metrics_by_task_adaptation.csv and path:experiments/synthetic_sequences/disentangled/analysis/aggregates/continuous_cvae-Apr24-reconstruction-stack_20260424/data/dist_plus_tokcc_a0p5_b0p01_d50_lin50_task_adaptation_checkpoints.csv. The generation script is path:experiments/synthetic_sequences/disentangled/analysis/run_continuous_cvae_apr24_best_loss_deepdive.py.