b0.005 const + twr0.5 Deep-Dive Results
b0.005 const + twr0.5 Deep-Dive Results
This note isolates the within-setting analysis for beta0p005_const_pv0_twr0p5, which remains the second-best setting by mean final router-sampled probe accuracy. The refreshed deep dive now uses all six completed cells: list_summation, sorting_algorithms, grid_pathfinding, linear_equation_solving, base_conversion, and multidigit_addition. The flattened frame grid targets 2%, 10%, 20%, 50%, and the nearest available late-training checkpoint for each task.
Snapshot
- completed tasks in this deep dive:
6 - mean final test final-answer accuracy:
0.9999 - mean final test router-sampled probe accuracy:
0.7315 - mean final test posterior-mu probe accuracy:
0.9160 - mean final baseline-normalized reconstruction loss:
0.181935 - mean final validation KL:
1.7
Latent-Space Structure Over Training
The fixed-PCA router-sampled latent projections show weaker and more task-variable separation than in the b0 const note. linear_equation_solving develops the clearest late-training banded structure, and grid_pathfinding becomes visibly stratified after early training with a cleaner split by the later checkpoints. base_conversion also develops separated vertical bands late in training. list_summation moves in the opposite visual direction: a broad early cloud contracts into a tighter cluster by the final checkpoint. sorting_algorithms remains largely diffuse and overlapping throughout the trajectory, while multidigit_addition finishes with a smaller but still visibly structured late-training cloud.
Caption: Fixed-PCA projection of router-sampled latent vectors over training for the completed b0.005 const + twr0.5 runs. Columns are tasks, points are colored by inferred strategy label within task, and each frame uses the same task-specific projection basis and axis limits.
Caption: Selected checkpoints from the same fixed-PCA latent trajectory view. Rows are tasks and columns are the nearest available checkpoints to 2%, 10%, 20%, 50%, and the latest available late-training frame for each task.
Probe, Reconstruction, And KL Dynamics
Across the six completed b0.005 const + twr0.5 tasks, final test final-answer accuracy ranges from 0.9996 to 1.0000, final test router-sampled probe accuracy ranges from 0.4103 on sorting_algorithms to 0.8282 on linear_equation_solving, and final test posterior-mu probe accuracy ranges from 0.5199 to 1.0000. Baseline-normalized reconstruction spans 0.0005 to 1.0478 across tasks, while final validation KL stays in a narrow low-KL regime around 1.4-1.9. Relative to the first validation evaluation, the setting shows mixed task-level router-probe changes together with much smaller end-of-training KL values than b0 const.
Caption: Validation router-probe accuracy, posterior-probe accuracy, baseline-normalized reconstruction loss, and KL over training for the completed b0.005 const + twr0.5 tasks. The rows stay in a much lower-KL regime than b0 const, together with weaker and more task-variable router-side separation.
