beta=0.01 Full Schedule Deep-Dive Results
beta=0.01 Full Schedule Deep-Dive Results
This note isolates the within-full comparison of beta0p01_d50_lin50 and beta0p01_cyclin25_w12p5. The trajectory figure uses the shared 51-checkpoint validation history from the repo-side aggregate, while the snapshot bullets below keep the final test means from the collection summary.
Snapshot
- compared cells:
12(6 tasks x 2 schedules x full) - mean final test final-answer accuracy: delayed linear
0.9845; cyclical0.9908 - mean final test router-sampled probe accuracy: delayed linear
0.9156; cyclical0.8828 - mean final test posterior-mu probe accuracy: delayed linear
0.9998; cyclical0.9999 - mean final test
router probe x final answer accuracy: delayed linear0.9013; cyclical0.8745
Task-Wise Validation Dynamics
The full beta=0.01 comparison is not a single-direction win. Delayed linear finishes with the largest final validation router-probe gains on linear_equation_solving (+0.1817) and list_summation (+0.0373), while cyclical keeps small final router-probe leads on sorting_algorithms (-0.0104), base_conversion (-0.0109), and multidigit_addition (-0.0026).
The early and mid training checkpoints show a second, cleaner pattern: delayed linear carries much larger KL almost everywhere before the schedule turns over. At the quarter checkpoint, delayed-minus-cyclical KL is already +71.89 on base_conversion, +64.34 on multidigit_addition, and +63.32 on linear_equation_solving.43
| task_name | delta_delayed_minus_cyclical_val_accuracy | delta_delayed_minus_cyclical_val_router_probe | delta_delayed_minus_cyclical_val_norm_recon | delta_delayed_minus_cyclical_val_kl |
|---|---|---|---|---|
| list_summation | -0.0193 | 0.0373 | 0.0008 | -0.0679 |
| sorting_algorithms | 0.0000 | -0.0104 | -0.1400 | 0.9979 |
| grid_pathfinding | 0.0000 | 0.0185 | 0.0350 | 0.0487 |
| linear_equation_solving | 0.0038 | 0.1817 | -0.0233 | 0.6626 |
| base_conversion | 0.0016 | -0.0109 | 0.0013 | 0.2944 |
| multidigit_addition | -0.0029 | -0.0026 | 0.0414 | 0.1077 |
Caption: Validation accuracy, router probe, strategy match rate, matched-strategy router score, normalized reconstruction, and KL over training for the beta0p01_d50_lin50 versus beta0p01_cyclin25_w12p5 comparison inside full adaptation.
Latent-Space Structure Over Training
The latent-geometry views below use a fixed PCA basis per task-and-schedule cell over the full saved frame stream. Router-sampled panels use generated outputs from continuous_val_router_sampled_z_frames.jsonl; posterior-side panels use the available posterior-mean stream, continuous_val_posterior_mu_frames.jsonl, rather than a newly sampled posterior-z stream.
Caption: Router-sampled latent geometry over training for beta0p01_d50_lin50 and beta0p01_cyclin25_w12p5 under full adaptation, color-coded by inferred strategy.
Caption: Selected router-sampled latent checkpoints from the same comparison.
Caption: Posterior-mu latent geometry over training for beta0p01_d50_lin50 and beta0p01_cyclin25_w12p5 under full adaptation, color-coded by supervised strategy label.
Caption: Smoothed posterior-mu latent geometry for the same comparison, using linear interpolation of projected point positions between saved eval frames.
Caption: Early-emphasis posterior-mu latent checkpoints from the same comparison: approximately 2%, 6%, 10%, 20%, 50%, and 100% training progress.
