synthetic-pretrained-disentangled-from-entangled-v3 Analysis

Agent-drafted interpretation, pending human review

The completed v3 cells sharpen the directional conclusion that was only tentative in v2: lower constant beta looks stronger than the high-beta warmup controls in this entangled-initialized regime.

The clearest evidence comes from grid_pathfinding, linear_equation_solving, and multidigit_addition, where beta0p01_const_pv0 produces some of the strongest router and posterior probe outcomes in the analyzed grid. base_conversion remains a special case where the warmup baselines stay strong, and list_summation continues to prefer lower-beta constant settings over the full-warmup alternatives. Taken together, the family pattern now looks less like “warmup solves the KL issue” and more like “stronger sustained reconstruction pressure is the more reliable lever.”

The matched mild-PV comparison is not encouraging in this snapshot. Adding pv0p01 on top of beta0p05_lin100 does not yield a broad router-side gain and is actively harmful on multidigit_addition, where final accuracy drops to 0.8077 and total loss becomes very large. That weakens the idea that mild posterior-variance control is an easy add-on once KL timing is favorable.

The remaining caution is missingness: six cells failed, including two low-beta base_conversion runs and most of the sorting_algorithms grid. So v3 should still be read as a partial collection, not a final family verdict. Even with that caution, the main next-step implication is clear: lower constant beta deserves follow-up before further investment in full-training warmup or mild PV regularization.

Built with LogoFlowershow