Schedule And Token-Alpha Deep Dive

Back to results

This deep dive records the paired contrasts for beta schedule and token surprisal-share weighting.

The largest positive mean paired contrast is tok_a0p5_minus_dist_d50, with mean delta +0.0347 and median delta +0.0334 over 18 matched pairs. The corresponding constant-beta contrast, tok_a0p5_minus_dist_const, has mean delta +0.0283 and median delta +0.0121.

The alpha=1.0 token weighting also has positive mean paired contrasts relative to distribution normalization: tok_a1_minus_dist_d50 has mean delta +0.0281 and median delta +0.0245; tok_a1_minus_dist_const has mean delta +0.0125 and median delta -0.0013.

The strongest schedule contrast is schedule_three_way_d50_minus_const, with mean delta +0.0331 and median delta +0.0449. Other delayed-50 schedule contrasts are smaller but positive for dist_plus_tokcc_a1, example_inv_surp, dist_plus_tokcc_a0p5, and dist_norm. Raw baseline schedule contrasts are near zero: schedule_raw_lin100_minus_const has mean delta +0.0014, and schedule_raw_d50_minus_const has mean delta -0.0023.

Loss-setting means place the delayed-50 token-weighted variants at the top of the collection: dist_plus_tokcc_a0p5_b0p01_d50_lin50 has mean matched score 0.8532, and dist_plus_tokcc_a1_b0p01_d50_lin50 has mean 0.8465.

Open empirical checks:

• Break each contrast down by family and adaptation mode to identify whether the mean contrast is broad or driven by a subset of cells.
• Compare alpha=0.5 against alpha=1.0 inside the same family/adaptation/schedule cells; the aggregate means favor alpha=0.5, but one paired median is negative under delayed-50.
• Inspect training dynamics around the first nonzero-beta steps for delayed schedules.