b0 const Deep-Dive Results
b0 const Deep-Dive Results
This note isolates the within-setting analysis for beta0_const_pv0. The current deep dive uses the five completed b0 const cells: list_summation, sorting_algorithms, grid_pathfinding, linear_equation_solving, and multidigit_addition. base_conversion / beta0_const_pv0 is still running and is omitted. The flattened frame grid uses the nearest available checkpoints to 2%, 10%, 20%, 50%, and 100% training progress from the repo-side b0_const_selected_router_frame_progress.csv artifact.
Snapshot
- completed tasks in this deep dive:
5 - mean final test final-answer accuracy:
0.9665 - mean final test router-sampled probe accuracy:
0.9028 - mean final test posterior-mu probe accuracy:
0.9995 - mean final baseline-normalized reconstruction loss:
0.000021 - mean final validation KL:
2364851.1
Latent-Space Structure Over Training
This note uses the same task-by-task router-sampled latent view as the continuous-v1 deep dive, but now for the higher-dimensional dim=64 zero-beta regime. The completed b0 const rows still dominate the collection on router-side decodability. The current deep-dive figures should therefore be read mainly as a within-setting view of how that strong dim=64 zero-beta regime develops across training, not as evidence that higher latent dimension is uniformly beneficial elsewhere in the collection.
Caption: Router-sampled latent geometry over training for the completed b0 const runs.
Caption: Selected checkpoints from the same router-sampled latent trajectory view.
Probe, Reconstruction, And KL Dynamics
Across the five completed b0 const tasks, router probe remains very high (0.8796 to 0.9630) and posterior probe is effectively saturated. The same completed rows also keep near-zero normalized reconstruction while ending with very large KL values. Relative to the matched continuous-v1 deep dive, the main observational difference is not higher decodability but lower final-answer accuracy, driven especially by the multidigit_addition and linear_equation_solving rows.
Caption: Validation router-probe accuracy, posterior-probe accuracy, baseline-normalized reconstruction loss, and KL over training for the completed b0 const tasks.
