Continuous CVAE Apr24 Reconstruction Stack: Results

Back to design · Interpretation draft

Collection State

The aggregate now includes all 234 planned jobs. The analysis scripts still tolerate missing runs by design: they read the available completed manifests and regenerate the same figure set from whatever subset is present.

The completed grid has 13 loss/schedule settings across 3 synthetic families and 3 adaptation modes, with 18 complete runs per setting.

Primary metric in this report is analysis/router_probe_times_strategy_match_rate, plotted in figure labels as matched strategy router score. This is the Apr24 analogue of the Apr22 "match-strategy router score": it multiplies router probe decodability by the fraction of generations matching the intended latent strategy.

Loss Setting Summary

The ranked summary below orders loss settings separately within each adaptation mode by mean matched strategy router score and overlays one point per task, so the figure shows both the within-row winners and how stable those rankings are across tasks. Normalized reconstruction variants occupy the top ten mean matched strategy router scores. The three raw ELBO baselines occupy the bottom three rows while retaining the highest final answer accuracy.

The lora panel below is the compact paper-candidate view: bars show mean strategy-match rate and router probe accuracy across tasks, and points show per-task values. The raw ELBO-style baselines have near-ceiling strategy-match rate but lower router probe accuracy.

Companion versions for full, reinit, and task-faceted panels are available in the local figures/ directory and in the repo-side aggregate figure directory.

Reconstruction-Family And Beta-Schedule Heatmaps

Matched strategy router score varies by reconstruction family and beta schedule. The top loss settings remain normalized variants across reconstruction families, while raw baselines are low in every observed raw-baseline schedule cell.

Final answer accuracy shows the complementary pattern: raw baselines remain near ceiling, while normalized variants are lower and more family dependent.

Paired Contrasts

The strongest positive paired mean contrast is token surprisal-share weighting with alpha=0.5 added to distribution normalization under the delayed-50 linear schedule. The next largest contrast is the delayed-50 schedule for the equal three-way combination relative to the corresponding constant-beta setting.

Reconstruction Ratio Frontier

The normalized variants cluster at substantially lower baseline-normalized reconstruction loss than raw baselines. The raw baselines cluster around baseline-normalized reconstruction near 1.07 and matched strategy router score near 0.44.

The companion views use the logged test reconstruction components directly on the x-axis rather than the final baseline-normalized reconstruction ratio. They show example inverse-surprisal reconstruction loss and normalized concentration-corrected token reconstruction loss, each with linear and log-x scaling.

Adaptation Modes

Paired adaptation-mode deltas show that reinit is lower than full and lora on matched strategy router score and final answer accuracy, while the router probe alone is not consistently lower for reinit.

Dynamics And Objective Scale

Validation dynamics preserve the Apr22-style view with final answer accuracy, router probe, matched strategy router score, posterior probe, strategy match rate, reconstruction ratios, KL, and beta over training.

The objective-component audit records the scale of each reconstruction component and KL term over training for the loss settings that expose those logged metrics.

Top And Bottom Runs

The top six runs by matched strategy router score are all normalized variants. They include both full and lora adaptation, and two task families appear in the top five.

The bottom six runs by matched strategy router score are all raw baselines.

Deep-Dive Result Notes

• Reconstruction frontier deep dive
• Schedule and token-alpha deep dive
• Adaptation-mode deep dive
• Raw-baseline latent deep dive
• Best-loss dynamics deep dive
• Best-loss attention-pattern deep dive