Paper-Selected Reconstruction-Stack Losses: Design
Paper-Selected Reconstruction-Stack Losses: Design
Purpose
This design note defines a preliminary paper-facing subset of reconstruction-stack settings for final figures. The subset should be easy to explain, should include credible baselines, and should make the central method claim testable rather than hiding known controls.
The target aggregate metric is analysis/router_probe_times_strategy_match_rate, shown in figures as matched strategy router score.
Current Evidence Snapshot
Available random-init evidence comes from the completed Apr24 and Apr26 reconstruction-stack collections. In the unified Apr24+Apr26 ranking, the top rows are:
| Rank | Collection | Loss setting | Matched strategy router score | Final answer accuracy |
|---|---|---|---|---|
| 1 | Apr26 | dist_plus_tokcc_a0p5_b0p1_d50_lin50 | 0.8556 | 0.9736 |
| 2 | Apr26 | exinv_plus_tokcc_a0p5_b0p01_d50_lin50 | 0.8533 | 0.9587 |
| 3 | Apr24 | dist_plus_tokcc_a0p5_b0p01_d50_lin50 | 0.8532 | 0.9543 |
| 4 | Apr26 | tokcc_a0p25_b0p01_const | 0.8512 | 0.9793 |
| 5 | Apr26 | tokcc_a0p25_b0p01_d50_lin50 | 0.8482 | 0.9626 |
| 8 | Apr26 | raw_b0p01_d50_lin50 | 0.8429 | 0.9830 |
Available from-entangled qwen25 0.5B evidence has the same Apr24-style loss menu. Its strongest rows are:
| Rank | Loss setting | Matched strategy router score | Final answer accuracy |
|---|---|---|---|
| 1 | dist_plus_tokcc_a0p5_b0p01_d50_lin50 | 0.8508 | 1.0000 |
| 2 | example_inv_surp_b0p01_d50_lin50 | 0.8439 | 0.9991 |
| 3 | dist_norm_b0p01_d50_lin50 | 0.8431 | 0.9989 |
| 4 | dist_plus_tokcc_a1_b0p01_d50_lin50 | 0.8317 | 0.9986 |
| 11 | raw_b1_d50_lin50 | 0.4693 | 0.9652 |
| 12 | raw_b1_lin100 | 0.4402 | 0.9999 |
| 13 | raw_b1_const | 0.4357 | 0.9991 |
Recommendation
Define the preliminary main method as:
dist_plus_tokcc_a0p5_b0p01_d50_lin50
Paper-facing name: distribution-normalized reconstruction plus concentration-corrected token reconstruction, with token alpha 0.5, beta 0.01, and a 50 percent delay plus 50 percent linear warmup.
Why this should be the first main-method candidate:
- It is among the strongest random-init settings: Apr24
0.8532, essentially tied with the Apr26 top rows. - It is the strongest currently available qwen25 0.5B from-entangled setting:
0.8508. - It has a clean decomposition: a broad distribution-level normalization term plus a token-level concentration-corrected term.
- It is already present in both random-init and qwen25 0.5B result sets, and the qwen25 1.5B collection is designed with the same setting.
Important caveat:
- Apr26
raw_b0p01_d50_lin50reaches0.8429on random-init. This means the paper should not frame the result as "all raw reconstruction fails." A more defensible claim is that standard ELBO-style raw beta-1 baselines fail, while low-beta delayed raw is a strong control that must be tracked separately.
Recommended Selection Rule
Use a two-tier rule:
- Main paper panel: include settings available across random-init and qwen25 from-entangled runs that share the Apr24 reconstruction-stack menu and the delayed-linear schedule.
- Control panel or appendix panel: include low-beta raw and cc-TWR-only follow-up controls from Apr26, marking missing pretrained or future cells explicitly.
This rule keeps the main figure simple while preventing the low-beta raw result from being swept away.
Preliminary Included Settings
Main Method And Ablations
| Role | Loss setting | Current source coverage | Status |
|---|---|---|---|
| Main method | dist_plus_tokcc_a0p5_b0p01_d50_lin50 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include |
| Token-alpha variant | dist_plus_tokcc_a1_b0p01_d50_lin50 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include as ablation |
| Distribution-only ablation | dist_norm_b0p01_d50_lin50 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include as ablation |
| Example-normalized ablation | example_inv_surp_b0p01_d50_lin50 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include as ablation |
| Three-way ablation | three_way_equal_b0p01_d50_lin50 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include as secondary ablation |
Baselines And Strong Controls
| Role | Loss setting | Current source coverage | Status |
|---|---|---|---|
| Standard cVAE / ELBO baseline | raw_b1_const | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include |
| Raw schedule baseline | raw_b1_lin100 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include |
| Raw delayed-linear baseline | raw_b1_d50_lin50 | Apr24 random-init; qwen25 0.5B; qwen25 1.5B planned/running | include |
| Extreme low-beta raw control | raw_b0p01_const | Apr26 random-init only | include as random-init control |
| Strong low-beta raw control | raw_b0p01_d50_lin50 | Apr26 random-init only | include as caveat/control, not as a failure baseline |
| Missing beta-VAE raw control | raw_b0p1_const | not currently in these collections | mark missing/future |
Follow-Up Method Variants
| Role | Loss setting | Current source coverage | Status |
|---|---|---|---|
| Exinv plus cc-TWR variant | exinv_plus_tokcc_a0p5_b0p01_d50_lin50 | Apr26 random-init only | include as promising follow-up |
| cc-TWR-only alpha 0.25 constant | tokcc_a0p25_b0p01_const | Apr26 random-init only | include as promising follow-up |
| cc-TWR-only alpha 0.25 delayed-linear | tokcc_a0p25_b0p01_d50_lin50 | Apr26 random-init only | include as promising follow-up |
| cc-TWR-only alpha 0.5 constant | tokcc_a0p5_b0p01_const | Apr26 random-init only | include as weaker follow-up |
| cc-TWR-only alpha 0.5 delayed-linear | tokcc_a0p5_b0p01_d50_lin50 | Apr26 random-init only | include as weaker follow-up |
Candidate Settings From User Proposal
| Candidate | Current status |
|---|---|
raw_b1_const | available in Apr24 random-init and qwen25 0.5B |
raw_b0p1_const | missing from current selected collections |
raw_b0p01_const | available in Apr26 random-init |
raw_b1_lin100 | available in Apr24 random-init and qwen25 0.5B |
raw_b1_d50_lin50 | available in Apr24 random-init and qwen25 0.5B |
tokcc_a0p25_b1_lin100 | missing; future setting |
tokcc_a0p25_b1_d50_lin50 | missing; future setting |
tokcc_a0p25_b0p01_const | available in Apr26 random-init |
tokcc_a0p25_b0p01_lin100 | missing; future setting |
dist_plus_tokcc_a0p5_b0p01_const | available in Apr24 random-init and qwen25 0.5B |
dist_plus_tokcc_a0p5_b0p01_lin100 | missing; future setting |
dist_norm_b0p01_const | available in Apr24 random-init and qwen25 0.5B |
Proposed Paper Figure Set
F1. Main matched-score bar plot. Rows are selected loss settings, columns or facets are regimes: random-init, qwen25 0.5B, and qwen25 1.5B when available. Metric is matched strategy router score.
F2. Baseline-vs-method decomposition. Show raw beta-1 baselines, dist-norm-only ablation, and dist+cc-TWR main method. This is the cleanest figure for the paper's main claim.
F3. Low-beta raw caveat panel. Show Apr26 raw_b0p01_const and raw_b0p01_d50_lin50 beside the main method and standard raw baselines. This protects against overclaiming that raw reconstruction always fails.
F4. Follow-up variants panel. Show exinv+cc-TWR and cc-TWR-only alpha variants as secondary evidence, not as the headline method unless the paper shifts to a broader "cc-TWR family" claim.
Conversation Structure
I suggest structuring the discussion in five passes:
- Claim scope. Decide whether the paper's method claim is about one setting (
dist_plus_tokcc_a0p5_b0p01_d50_lin50) or a family (tokcc/dist+tokccobjectives). - Fair baselines. Decide which raw settings are baselines, which are schedule/beta controls, and whether low-beta delayed raw should be in the main figure or a caveat figure.
- Coverage matrix. Decide which cells must exist across random-init, qwen25 0.5B, and qwen25 1.5B before a setting can appear in the main paper figure.
- Figure layout. Decide whether final figures are grouped by regime, by loss setting, or by claim: baseline failure, method success, and control caveats.
- Future-run gap list. Decide which missing settings are worth running versus which should remain explicitly marked as absent.
Open Decisions
- Should the main method be a single setting,
dist_plus_tokcc_a0p5_b0p01_d50_lin50, or a broader cc-TWR family? - Should Apr26
raw_b0p01_d50_lin50appear in the main paper figure, or only in a caveat/control panel? - Should final paper figures require coverage in both random-init and qwen25 from-entangled settings, or may Apr26-only follow-up variants appear in the main figure?
- Is qwen25 1.5B required for the final main claim, or is it a scale-up validation figure?
- Should missing candidate settings such as
raw_b0p1_const,tokcc_a0p25_b1_*, anddist_plus_tokcc_a0p5_b0p01_lin100be run, or documented as intentionally omitted?
Preliminary Recommendation
Use dist_plus_tokcc_a0p5_b0p01_d50_lin50 as the paper's main method for now. Present raw beta-1 settings as standard cVAE baselines, include distribution-only and example-normalized rows as ablations, and keep Apr26 low-beta raw delayed-linear as a strong control/caveat. Do not make the paper's main claim "raw reconstruction fails"; make it "standard raw beta-1 CVAE baselines fail to expose router-side strategy structure, while the selected normalized-plus-token reconstruction objective consistently improves matched strategy router score in the current random-init and qwen25 0.5B evidence."
