synthetic-pretrained-disentangled-from-entangled-v4 Design

Summary

synthetic-pretrained-disentangled-from-entangled-v4 is a focused follow-up to synthetic-pretrained-disentangled-from-entangled-v3. It keeps the same entangled-initialized Qwen2.5-0.5B setup, the same 6 synthetic tasks, and the same continuous-latent lora training recipe, but shifts the loss study further into the low-KL regime.

Planned design:

• 6 tasks x 4 loss settings x 1 seed = 24 jobs
• initialization from synthetic-pretrained-entangled-v1
• backbone Qwen2.5-0.5B
• continuous latent, dim 8, last-token pooling

Motivation

The v3 results strengthened the case that lower effective KL pressure is one of the most promising current levers for improving router-side strategy alignment. In particular:

• the strongest completed constant setting was beta0p01_const_pv0
• several warmup settings remained task-dependent rather than consistently better
• the matched pv=0.01 comparison did not look like a clear win

That motivates a narrower follow-up centered on four higher-value tests:

1. What happens when KL pressure is removed entirely?
2. Is a very small constant beta (0.005) better than beta=0?
3. Does warming to beta=0.01 over the full training run outperform the low constant-beta settings?
4. If beta=0.01 with full warmup is favorable, does a very mild posterior-variance penalty (0.005) help?

Loss Settings

The collection uses four settings:

1. beta0_const_pv0 Zero-beta anchor: beta=0.0, constant schedule, no posterior-variance penalty.

2. beta0p005_const_pv0 Very small constant beta: beta=0.005, no posterior-variance penalty.

3. beta0p01_lin100_pv0 Full-training warmup to beta=0.01, no posterior-variance penalty.

4. beta0p01_lin100_pv0p005 Matched mild-PV comparison: same beta=0.01 full warmup with posterior_variance_weight=0.005.

All settings keep:

• normalize_reconstruction_loss = true
• router_marginal_kl_to_prior_weight = 0.0
• router_support_weight = 0.0
• token_weighted_reconstruction_weight = 0.0
• inter_latent_divergence_weight = 0.0

Design Intent

This collection is meant to answer four concrete questions:

1. Does removing KL entirely at beta=0.0 improve router-side strategy alignment relative to a very small nonzero beta?
2. Is beta=0.005 a better constant setting than the zero-beta limit?
3. Does a full-training warmup to beta=0.01 outperform the zero- and near-zero constant-beta settings?
4. Does a very mild posterior-variance penalty help once beta timing is already favorable at beta=0.01?

Current Known Risks

The current partial snapshot already shows two operational risks that matter for interpretation:

• sorting_algorithms remains memory-fragile, with three failed OOM cells so far
• the two base_conversion low-beta constant runs are still running, so the zero- and near-zero constant-beta comparison is not yet complete for that task