ss-disentangled-direct-pretrained-v1 Results

Collection State

  • planned design: 6 tasks x 1 setting x 1 seed = 6 jobs
  • analyzable completed runs: 5
  • failed runs: 1
  • excluded task: sorting_algorithms after CUDA OOM

Final Snapshot

  • sampled final-answer accuracy across completed tasks: 0.9994-1.0
  • router probe accuracy across completed tasks: 0.36-0.49
  • posterior probe accuracy across completed tasks: 0.39-0.54

The dominant observational pattern is a wide gap between near-saturated task accuracy and only moderate probe quality.

Figures

Validation accuracy and alignment dynamics by task. Caption: Validation final-answer accuracy stays very high while router-sampled and posterior-mean probe accuracy remain materially lower across the five completed tasks.

Optimized objective terms by task. Caption: The active objective terms, normalized reconstruction and KL-to-prior, remain low and broadly well behaved.

Unweighted diagnostics by task. Caption: Posterior-variance and router-marginal-KL diagnostics are much larger in scale than the optimized objective terms even though they are logged only as diagnostics here.

Standardized comparison of objective versus unweighted diagnostics. Caption: Standardized trajectories highlight that objective terms can stabilize even while unweighted diagnostics remain large or worsen.

Final accuracy versus router probe across tasks. Caption: Final answer accuracy is nearly saturated on all completed tasks, but router probe accuracy remains only moderate and task-dependent.

Built with LogoFlowershow