Qwen25 Multi-Task Paper-Method Grid: Design

Back to hub

Motivation

This report is a compact model-size and method check for the multi-task pretrained-from-entangled setting. It takes the strongest paper-facing reconstruction families from the Apr27 paper-selected analysis, evaluates them on Qwen2.5 0.5B and Qwen2.5 1.5B entangled multi-task source models at z64, then integrates targeted TOK-CC/dist-weight, token-inverse-surprisal, and raw beta baseline follow-ups.

Research Questions

1. Which reconstruction setting gives the strongest router-side strategy structure in the completed Qwen2.5 0.5B multi-task runs?
2. Does the multi-task gap between all-task and within-task router decodability persist under this pretrained-from-entangled setup?
3. How do the z8 and z64 b0p01 TOK-CC/dist-normalized follow-up rows compare?
4. Which Qwen2.5 1.5B settings are strongest after the paper grid and ratio/tokinv follow-up completed?
5. How does the raw beta=1 constant standard-ELBO baseline compare with annealed and lower-beta raw baselines?
6. Which settings should remain candidates for analogical diagnostics or later replication?

Sweep

• Source model: qwen25_05b, qwen25_15b
• Reconstruction family: raw, dist_norm, tokcc_a0p25, dist_plus_tokcc_a0p25_w0p05_tokcc0p95, dist0p25_tokcc0p75_a0p25, dist0p50_tokcc0p50_a0p25, dist0p75_tokcc0p25_a0p25, tokinv_a0p25, tokinv_a1_eps1em8
• Beta: b0p01, b0p1, b1
• Continuous latent dim: 8, 64
• Fixed: multi_task, LoRA, seed 314159; beta schedule is lin100 except raw constant baseline rows

Metrics

The primary metric is matched router score, computed as the equal-task mean within-task router probe times sampled strategy-match rate. Guardrail metrics are final answer accuracy, strategy-match rate, posterior-mu probe accuracy, all-task router probe accuracy, reconstruction losses, and KL.

Completion Rule

Rows with metrics.json are final-test results. When resubmission creates multiple run directories for the same planned cell, the analysis uses the metrics-bearing run directory and keeps any stale failed directory out of the final table.