continuous-cvae-Mar20 Results

Collection State

• complete runs with run_manifest.json and metrics.json: 94/96
• missing settings:
  • base_conversion + full + beta0p5_base
  • base_conversion + reinit + beta0p5_base

Global Metrics Snapshot

Across the 94 completed runs:

• sampled_test_final/overall_final_answer_accuracy: 0.9094 to 1.0 (mean 0.9870)
• sampled_test_final/parse_success_rate: 0.9917 to 1.0 (mean 0.9995)
• sampled_test_final/sampled_z/linear_probe_accuracy: 0.3660 to 0.5237 (mean 0.4452)
• continuous_diag_test_final/posterior_mu/linear_probe_accuracy: 0.3737 to 0.9991 (mean 0.7038)
• sampled_test_final/unique_valid_strategy_rate: 0.6572 to 0.9980 (mean 0.9056)
• sampled_test_final/invalid_strategy_rate: 0.0 to 0.0556 (mean 0.0077)

By adaptation mode, full has higher means than reinit on sampled-z probe, posterior probe, final-answer accuracy, and invalid-strategy rate.

Figures

Caption: Most task-by-adaptation cells contain all 8 expected settings; the only missing pair is concentrated in base_conversion.

Caption: Final accuracy remains high while sampled-z probe varies much more strongly, showing that answer quality and latent-strategy decodability are only modestly coupled.

Caption: Posterior probes are often materially higher than sampled-z probes for the same run family.

Caption: Better sampled-z probe accuracy tends to coincide with fewer invalid strategy attributions, but the relationship is not strict.

Caption: Probe quality is strongly task-dependent; no single setting dominates all tasks.

Caption: Several settings produce much stronger posterior-side separability than sampled-z separability.

Caption: grid_pathfinding remains the hardest task for clean strategy attribution, while linear_equation_solving and multidigit_addition are stronger.

Caption: In top runs, sampled-z probe quality often peaks early and softens later in training.

Caption: Posterior-mu probe quality typically improves over training even when sampled-z probe does not.

Caption: Answer accuracy continues improving late in training even in runs where sampled-z probe flattens or declines.