synthetic-pretrained-disentangled-from-entangled-v1 Analysis
synthetic-pretrained-disentangled-from-entangled-v1 Analysis
Main Pattern
This collection still shows a separation between task success and latent-strategy alignment, but it is less uniformly weak than the direct-pretrained counterpart.
Across the 5 completed runs, sampled-test accuracy is saturated at 1.0, mean router probe accuracy is 0.500, and mean posterior-mu probe accuracy is 0.649. Relative to ss-disentangled-direct-pretrained-v1, the strongest shift is on the posterior side: on the shared 5 completed tasks, the from-entangled collection improves mean posterior probe by about +0.172, while mean router probe improves by a smaller +0.072.
The improvement is not uniform. base_conversion is the clearest positive case, with router probe rising to 0.769 and posterior probe reaching 1.000. linear_equation_solving also shows a large posterior-side gain, while list_summation, grid_pathfinding, and multidigit_addition do not show a comparable router-side improvement. The cautious summary is therefore not that entangled initialization solves the representation problem, but that it helps some tasks substantially, especially on the posterior side, while leaving the router-side gap partly intact.
As in the direct-pretrained collection, the active objective is still baseline-normalized reconstruction plus 0.1 * KL prior. Posterior variance and router marginal KL remain diagnostics only because their weights are zero, so those quantities can grow even while the optimized objective behaves cleanly. The normalized reconstruction metric is also harder to compare across collections here, because the baseline is inherited from a stronger entangled source model.
The most grounded reading is that entangled initialization can preserve or expose more strategy-relevant structure than direct-pretrained disentangling alone, but that benefit shows up more reliably in posterior probes than in router probes. This collection provides evidence that initialization matters, but not yet evidence that the strategy-coded latent problem is solved.