exp2_discrete_cvae-Apr3-7loss_20260403 Results

Collection State

• original planned grid: 108 runs
• extended grid adds three new loss settings
• observed completed runs in the analyzed dataset: 144

Matched Contrasts

Token-weighted reconstruction contrast

• matched-pair delta mean on alignment (twr0 -> twr2): -0.0059
• 95% bootstrap CI: [−0.1067, +0.1096]
• treatment-setting mean accuracy: 0.9949

Caption: The matched TWR contrast shows a wide alignment interval spanning zero while accuracy remains above the gate.

JSD contrast

• matched-pair delta mean on alignment (twr2_jsd0 -> twr2_jsd0p5): +0.0208
• 95% bootstrap CI: [−0.1022, +0.1412]
• treatment-setting mean accuracy: 0.9948

Caption: The JSD contrast is directionally positive on alignment, but the interval still spans zero.

Normalization contrast

• matched-pair delta mean on alignment (base -> norm): +0.1298
• 95% bootstrap CI: [+0.0387, +0.2375]
• treatment-setting mean accuracy: 0.9933

Caption: The normalization contrast produces the clearest positive shift in alignment while preserving answer accuracy.

Combination And Winner Views

• best alignment mean in the predefined ladder: 0.4405 at beta0p1_norm_twr2_jsd0p5
• highest overall tested mean alignment in the expanded set: 0.4821 at beta0p1_norm_twr3_jsd0
• per-task winners under the accuracy gate:
  • beta0p1_norm_twr3_jsd0: 3/6 tasks
  • beta0p1_norm_twr2_jsd0p5: 1/6 task
  • beta0p1_norm_twr2_jsd0: 1/6 task
  • beta0p1_norm_twr0_jsd0: 1/6 task

Caption: Within the predefined ladder, beta0p1_norm_twr2_jsd0p5 is the best-performing combination.

Caption: No single setting wins every task; the strongest default is still task-conditional.

Caption: Several projected next settings remain attractive, but the expanded dataset already validates beta0p1_norm_twr3_jsd0 as a strong tested default.

Caption: The reconstruction-versus-alignment tradeoff remains task- and mode-dependent rather than collapsing to a single scalar ranking.