Qwen25 Multi-Task Paper-Method Grid: Analysis

Back to hub

Agent-drafted interpretation, pending human review

The completed Qwen2.5 0.5B rows support the same qualitative multi-task distinction as the Apr27 report: the model can preserve high sampled behavior and high task-local strategy decodability while failing to align all task-prefixed strategies into one globally linear router coordinate system. The best completed row is dist_plus_tokcc_a0p25_w0p75_tokcc0p25_b0p01_lin100 at z64, with within-task matched router score 0.8502, but even strong rows should still be discussed with the all-task versus within-task probe distinction rather than as globally solved latent strategy alignment.

The TOK-CC/dist-weight follow-up is now part of the evidence base rather than a pending side experiment. Its most useful role is a capacity and objective-shape comparison: z8 and z64 share six b0p01 cells, while z64 also connects back to the original paper grid.

The raw beta baseline rows add the standard ELBO reference point (raw_b1_const) and schedule/beta controls for both model sizes. The completed Qwen2.5 1.5B rows strengthen the method-family comparison: the strongest 1.5B results now come from TOK-CC/dist mixtures and TOK-CC-style rows rather than the raw reconstruction baseline. The token-inverse-surprisal a1_eps1e-8 row remains more diagnostic than competitive in this snapshot.

The completed analogical-steering evaluator adds a separate cross-input strategy-stability diagnostic. It should not replace the router-probe ranking: answer correctness is near saturated across the analogical summaries, while strategy-set overlap has a much wider spread by loss setting and task. The top analogical rows are therefore promising semantic-stability candidates, but this metric should be reported alongside, not collapsed into, matched router score.

Next Checks

  • Inspect a small sample of analogical records.jsonl outputs for the highest- and lowest-set-overlap rows before turning the aggregate overlap rates into a paper-facing semantic-stability claim.
  • Keep the 25:75, 50:50, 75:25, and token-inverse-surprisal variants in the comparison tables so the follow-up objective sweep remains visible.
  • Use the within-task router probe as the ranking metric, and keep the all-task probe beside it as a global-coordinate diagnostic in paper-facing multi-task figures.
Built with LogoFlowershow