continuous_cvae-Apr22-beta-schedule-followup_20260422 Results

Back to design

Collection State

• planned design: 6 tasks x 6 beta schedules x 3 adaptation modes x 1 seed = 108 jobs
• completed runs: 108/108
• locally analyzable runs: 108/108
• best overall settings by mean router probe x final answer accuracy: beta0p1_d50_lin50, then beta0p01_cyclin25_w12p5
• highest raw router-probe setting: beta0p01_d50_lin50

Caption: Every Apr22 task-by-setting-by-mode cell is present exactly once in the discovered aggregate.

Global Schedule Ranking

Using the agreed full-grid mean over all 108 runs, the six settings sort as follows:

Observed ranking facts:

• beta0p1_d50_lin50 ranks first on the agreed composite metric even though beta0p01_d50_lin50 has the strongest raw router probe.
• beta0p01_cyclin25_w12p5 ranks second overall because it stays near the top on both accuracy and router probe.
• beta0p1_cyclin25_w12p5 and beta1_d50_lin50 retain high accuracy, but both sit well below the three leading schedules on router probe.
• beta0_unnorm is not competitive as a global winner once the full-grid accuracy drop is included.

Caption: Mean final-answer accuracy, router probe, posterior probe, and router probe x final answer accuracy for each setting, split by adaptation mode.

Delayed-Linear Versus Cyclical Comparison

The matched delayed-linear versus cyclical comparisons separate cleanly by beta level.

Observed pairwise patterns:

• At beta=0.1, delayed linear is stronger than cyclical on router probe in all three modes and also stronger on the agreed composite score in all three modes.
• At beta=0.01, delayed linear still has the stronger router probe in all three modes, but cyclical overtakes it on the composite score in reinit because the delayed-linear reinit accuracy penalty is much larger.
• The beta=0.01 full and lora rows still favor delayed linear on the composite score.

Caption: The beta-matched delayed-linear and cyclical comparisons broken out by adaptation mode for final accuracy, router probe, posterior probe, and router probe x final answer accuracy.

Adaptation-Mode Interaction And Failure Concentration

The leading schedules are not identical across modes:

Observed mode-level patterns:

• full and lora both rank beta0p01_d50_lin50 first on the composite score.
• reinit ranks beta0p1_d50_lin50 first instead, with beta0p01_cyclin25_w12p5 second.
• The main collapse cluster is beta0_unnorm / reinit, where final-answer accuracy ranges from 0.0520 to 0.5551 on the bottom runs even though router probe stays high.
• Posterior probe remains close to ceiling for most settings; the most visible posterior-probe weakening happens in the higher-beta cyclical and beta1_d50_lin50 rows.

Caption: Final-answer accuracy, router probe, posterior probe, and final-eval normalized reconstruction by task, setting, and adaptation mode.

Caption: One point per run, showing where high router probe and high accuracy coincide and where they separate, especially in reinit.

Dynamics Follow-Up

The repo-side aggregate now includes explicit training-dynamics views plus two focused deep-dive notes:

• beta0p01-full-schedule-deepdive-results
• full-vs-reinit-dynamics-deepdive-results

Both deep-dive notes now also include latent-space structure views over training, with animated GIFs and static checkpoint panels for router-sampled z and the available posterior-mu stream.

Observed dynamics facts:

• full and lora start near ceiling on mean validation accuracy, while reinit starts near 0.14 to 0.16 and then recovers at very different rates by loss setting.
• The final full-minus-reinit validation-accuracy gaps range from 0.0060 for beta1_d50_lin50 to 0.4100 for beta0_unnorm.
• The matched final full-minus-reinit validation router-probe gaps stay much smaller, ranging from -0.0451 to +0.0161.

Caption: Mean validation accuracy, router probe, posterior probe, strategy match rate, and matched-strategy router score over training, aggregated over tasks and split by adaptation mode.

Caption: Mean validation accuracy, router probe, posterior probe, strategy match rate, matched-strategy router score, normalized reconstruction, and KL over training for the full Apr22 sweep.

Top And Bottom Runs By The Agreed Composite Score

Top runs:

Bottom runs:

Runtime Signal

The runtime section has moved to profile, including the previous secondary runtime summary and a section-level profiling breakdown from the per-run profiling_summary.json artifacts.