ss-disentangled-direct-pretrained-pv-rmkl-sweep-v1 Analysis
ss-disentangled-direct-pretrained-pv-rmkl-sweep-v1 Analysis
This completed sweep strengthens a narrow negative conclusion: posterior-variance and router-marginal-KL regularization are mechanically effective, but they are not sufficient to make the direct-pretrained latent line up cleanly with reasoning strategies.
The clearest result is the RMKL intervention. Turning on router_marginal_kl_to_prior_weight=0.1 collapses the router-marginal-KL diagnostic from thousands to essentially zero while leaving answer accuracy unchanged. But router probe does not improve. The PV intervention shows the same pattern: posterior variance drops sharply, yet router probe stays broadly flat. So the large unregularized diagnostics were real, but controlling them is not by itself enough to solve the representation problem.
The completed base_conversion cells remove one of the main ambiguities from the earlier partial readout: they do not reveal a hidden broad gain from these regularizers. The best current reading is therefore narrower and more pessimistic than the motivating hypothesis. These auxiliary terms may be useful for shaping distributions, but they are not the main semantic lever in the direct-pretrained regime.