synthetic-pretrained-disentangled-from-entangled-v2 Analysis
synthetic-pretrained-disentangled-from-entangled-v2 Analysis
The v2 snapshot is directionally useful but not canonical because the local run-directory naming collision breaks trust in part of the completed grid. The main value of this note is therefore to record the pattern that survived the artifact issue and to explain why it motivated v3.
Within the analyzable cells, lower effective KL pressure still looks like the main promising lever. base_conversion and linear_equation_solving show their strongest router or posterior outcomes under either lower constant beta or warmup settings, and multidigit_addition shows a strong result for the bundled beta0p05_lin30_pv0p01 setting. By contrast, grid_pathfinding and list_summation remain comparatively flat across the surviving settings, which weakens any claim that a single loss change solved the router-side gap globally.
The posterior-variance comparison is especially hard to interpret here because the only PV-enabled setting is bundled with both a lower final beta and a warmup schedule. That makes the apparent multidigit_addition gain suggestive rather than diagnostic. The strongest firm conclusion is narrower: reducing or delaying KL pressure looks more promising than keeping the original constant beta=0.1 objective.
Because v2 cannot safely serve as the family reference point, it should be treated as a superseded scouting collection. Its main contribution is to justify the narrower v3 menu, where lower constant beta, longer warmup, and a matched mild-PV comparison are separated more cleanly.