Losses

Notes

• Current reconstruction loss stack
• Planned implementation: reconstruction-loss token aggregation mode
• Current implementation: empirical z-free-relative reconstruction rescaling
• Current implementation: example-level inverse-surprisal weighting
• Current implementation: token-level surprisal-share weighting
• Current implementation: normalized concentration-corrected token-weighted reconstruction
• Current implementation: normalized inverse-surprisal token-weighted reconstruction ablation

Scope

This cluster documents the current reconstruction-loss design in code: how the loss terms are composed today, which methodology variants are implemented, and which proposed variants are not yet part of the implementation.