Paper-Grid Multi-Task Report

Back to hub · single-task results · single-task dynamics

This note records the multi_task slice of the same analyzed collections covered in results: the Apr27 paper basic-grid base collection, the Apr30 extension runs appended into that collection, the May01 token-inverse stress / dist-TOK-CC follow-up collection, and the May02 0.75 distribution-normalized / 0.25 TOK-CC follow-up collection. It is multi-task-only: figures mirror the single-task result and dynamics views but use only task_name == "multi_task".

Collection State

The planned multi-task grid has 72 cells across the Apr27, Apr30, May01, and May02 analyzed collection coverage: twelve reconstruction families, three beta settings, two beta schedules, LoRA adaptation, and seed 314159. Final-test artifacts are complete for 72 cells. There are no failed final-test cells in the current aggregate. Raw validation dynamics are available for 72 settings with 51 to 102 checkpoints per setting.

item	count
planned multi_task cells	72
complete final-test artifacts	72
failed cells	0
settings with validation dynamics	72
validation checkpoint rows	3978
final checkpoints within 0.01 of best validation matched score	8
settings improving by more than 0.02 over last 10 checkpoints	15

Aggregate Metrics

Rows are complete multi_task final-test cells ranked by analysis/within_task_router_probe_times_strategy_match_rate_mean. The best complete setting is tokcc_a0p25_b0p01_lin100 with within-task matched router score 0.8316. The within matched column uses the equal-effective-task mean of within-task router-probe times strategy-match, so it scores strategy separation inside each task rather than requiring one global task-prefixed strategy geometry.

loss setting	within matched	analogical	answer acc	all-task router	within-task router	strategy match	posterior	base-norm recon	KL
tokcc_a0p25_b0p01_lin100	0.8316	0.8203	0.9778	0.3940	0.8936	0.9342	0.6831	7.3994	3.4344
dist0p25_tokcc0p75_a0p25_b0p01_lin100	0.8302	0.8135	0.9802	0.3549	0.8709	0.9486	0.6718	0.0810	3.3786
dist_plus_tokcc_a0p25_b0p01_lin100	0.8301	0.8096	0.9872	0.2676	0.8578	0.9619	0.6358	0.1248	3.4037
dist0p05_tokcc0p95_a0p25_b0p01_lin100	0.8265	0.8057	0.9754	0.2650	0.8774	0.9361	0.5571	0.0678	3.1581
dist0p10_tokcc0p90_a0p25_b0p01_lin100	0.8252	0.7998	0.9792	0.2944	0.8634	0.9483	0.7054	0.0806	3.5642
dist0p05_tokcc0p95_a0p25_b0p01_const	0.8201	0.7812	0.9746	0.2329	0.8775	0.9305	0.7052	0.0839	3.2231
example_inv_surp_b0p01_lin100	0.8022	0.7803	0.9895	0.2334	0.8254	0.9691	0.6215	0.0695	3.6603
tokcc_a0p25_b0p1_lin100	0.8007	0.7930	0.9711	0.3593	0.8545	0.9311	0.6205	8.2907	1.8469
dist0p25_tokcc0p75_a0p25_b0p01_const	0.7991	0.7881	0.9769	0.2447	0.8365	0.9490	0.7512	0.0922	3.2986
dist0p75_tokcc0p25_a0p25_b0p01_lin100	0.7854	0.7852	0.9823	0.2326	0.8221	0.9555	0.7310	0.0921	3.6978
dist_norm_b0p01_lin100	0.7837	0.7773	0.9827	0.2777	0.8137	0.9562	0.7073	0.0789	3.8448
dist0p05_tokcc0p95_a0p25_b0p1_lin100	0.7742	0.7490	0.9737	0.1899	0.8153	0.9331	0.6701	0.1907	1.7767
tokcc_a0p25_b0p01_const	0.7719	0.7803	0.9723	0.2809	0.8351	0.9227	0.6229	7.7678	2.7540
dist0p10_tokcc0p90_a0p25_b0p1_lin100	0.7430	0.7686	0.9795	0.1853	0.7812	0.9503	0.6808	0.1654	1.6801
dist0p05_tokcc0p95_a0p25_b0p1_const	0.7375	0.8057	0.9751	0.1860	0.7793	0.9432	0.6501	0.1778	1.8387

Lowest-ranked complete settings:

loss setting	within matched	analogical	answer acc	all-task router	within-task router	strategy match	posterior	base-norm recon	KL
dist_plus_tokcc_a0p25_b1_const	0.4203	0.3379	0.9892	0.0942	0.4285	0.9783	0.2980	1.1265	0.0004
raw_b1_lin100	0.4207	0.3271	0.9864	0.0935	0.4308	0.9755	0.4847	1.1385	0.0004
dist_norm_b1_lin100	0.4212	0.3389	0.9907	0.0936	0.4300	0.9793	0.4431	1.1326	0.0003
raw_b0p01_lin100	0.4214	0.3379	0.9897	0.0974	0.4295	0.9779	0.5211	1.1298	0.0007
dist0p05_tokcc0p95_a0p25_b1_const	0.4215	0.3262	0.9898	0.0959	0.4331	0.9709	0.2599	1.1459	0.0010
example_inv_surp_b1_const	0.4215	0.3203	0.9881	0.0835	0.4296	0.9757	0.5622	1.1991	0.0003
dist0p25_tokcc0p75_a0p25_b1_const	0.4219	0.3252	0.9879	0.0962	0.4328	0.9677	0.2824	1.1450	0.0010
tokinv_a0p25_b1_lin100	0.4221	0.3447	0.9910	0.0964	0.4290	0.9822	0.3109	5.8944	0.0007

The aggregate-metric subset bar plot includes the baseline, distribution-normalized reconstruction, pure TOK-CC, the selected 0.25 distribution-normalized / 0.75 TOK-CC and 0.75 distribution-normalized / 0.25 TOK-CC ratios, and the token-inverse-surprisal stress setting. In that first plot, fidelity-alignment is the product of strategy-match rate and within-task latent probe; the all-task latent probe is omitted.

Within-Task Strategy Decodability

The metric bars distinguish the global all-task latent strategy probe from the within-task latent strategy probe. The all-task probe asks whether one global linear classifier decodes the full task-prefixed strategy label set from router-sampled z; the within-task probe averages six per-task strategy probes with equal task weight.

loss setting	within matched	all-task router	within-task router	within-task gain	all-task posterior	within-task posterior	tasks
tokcc_a0p25_b0p01_lin100	0.8316	0.3940	0.8936	0.4996	0.6831	0.9985	6.0000
dist0p25_tokcc0p75_a0p25_b0p01_lin100	0.8302	0.3549	0.8709	0.5160	0.6718	0.9889	6.0000
dist_plus_tokcc_a0p25_b0p01_lin100	0.8301	0.2676	0.8578	0.5902	0.6358	0.9925	6.0000
dist0p05_tokcc0p95_a0p25_b0p01_lin100	0.8265	0.2650	0.8774	0.6124	0.5571	0.9920	6.0000
dist0p10_tokcc0p90_a0p25_b0p01_lin100	0.8252	0.2944	0.8634	0.5689	0.7054	0.9906	6.0000
dist0p05_tokcc0p95_a0p25_b0p01_const	0.8201	0.2329	0.8775	0.6446	0.7052	0.9924	6.0000
example_inv_surp_b0p01_lin100	0.8022	0.2334	0.8254	0.5920	0.6215	0.9463	6.0000
tokcc_a0p25_b0p1_lin100	0.8007	0.3593	0.8545	0.4952	0.6205	0.9911	6.0000
dist0p25_tokcc0p75_a0p25_b0p01_const	0.7991	0.2447	0.8365	0.5918	0.7512	0.9897	6.0000
dist0p75_tokcc0p25_a0p25_b0p01_lin100	0.7854	0.2326	0.8221	0.5895	0.7310	0.9783	6.0000
dist_norm_b0p01_lin100	0.7837	0.2777	0.8137	0.5360	0.7073	0.9782	6.0000
dist0p05_tokcc0p95_a0p25_b0p1_lin100	0.7742	0.1899	0.8153	0.6253	0.6701	0.9418	6.0000
tokcc_a0p25_b0p01_const	0.7719	0.2809	0.8351	0.5542	0.6229	0.9861	6.0000
dist0p10_tokcc0p90_a0p25_b0p1_lin100	0.7430	0.1853	0.7812	0.5958	0.6808	0.9582	6.0000
dist0p05_tokcc0p95_a0p25_b0p1_const	0.7375	0.1860	0.7793	0.5932	0.6501	0.9797	6.0000

Within-task versus global strategy probe

Within-task router-probe gain by family, beta, and schedule

Within-task router probe by effective task

Family, Beta, And Schedule Views

The heatmaps are direct multi_task values, not means over datasets. All heatmap cells are populated in the current aggregate.

Multi-task within-task matched score by method, beta, and schedule

Multi-task final answer accuracy by method, beta, and schedule

Analogical Consistency

Analogical consistency uses analogical_steering/pair_set_overlap_rate, the rate at which paired generations that reuse the same latent sample have overlapping inferred strategy sets. Analogical summaries are currently available for all 72 complete multi-task cells.

loss setting	set overlap	overlap if nonempty	unique agreement	within matched
dist0p10_tokcc0p90_a0p25_b0p01_const	0.8223	0.9345	0.9320	0.7119
tokcc_a0p25_b0p01_lin100	0.8203	0.9344	0.9323	0.8316
dist0p25_tokcc0p75_a0p25_b0p01_lin100	0.8135	0.8986	0.8993	0.8302
dist_plus_tokcc_a0p25_b0p01_lin100	0.8096	0.8600	0.8511	0.8301
dist0p05_tokcc0p95_a0p25_b0p01_lin100	0.8057	0.9076	0.9026	0.8265
dist0p05_tokcc0p95_a0p25_b0p1_const	0.8057	0.8938	0.8919	0.7375
dist0p10_tokcc0p90_a0p25_b0p01_lin100	0.7998	0.8961	0.8916	0.8252
tokcc_a0p25_b0p1_lin100	0.7930	0.9124	0.9062	0.8007
dist0p25_tokcc0p75_a0p25_b0p01_const	0.7881	0.8897	0.8855	0.7991
dist0p75_tokcc0p25_a0p25_b0p01_lin100	0.7852	0.8645	0.8635	0.7854

Correlations below use all complete multi-task cells.

comparison metric	n	Pearson r
within-task matched-strategy router score	72.0000	0.9717
all-task router probe	72.0000	0.8408
within-task router probe	72.0000	0.9660
strategy-match rate	72.0000	-0.5373
final answer accuracy	72.0000	-0.2899
baseline-normalized reconstruction	72.0000	-0.3542

Multi-task analogical overlap by method, beta, and schedule

Multi-task analogical overlap versus within-task matched score

Multi-task analogical agreement decomposition

Reconstruction Frontier

These frontier plots use final-test reconstruction measures against within-task matched router score. Point size is intentionally fixed.

Multi-task baseline-normalized reconstruction versus within-task matched score

Multi-task distribution-normalized reconstruction versus within-task matched score

Multi-task example-level reconstruction versus within-task matched score

Multi-task token-level reconstruction versus within-task matched score

Training Dynamics

The dynamics panels use raw multi_task validation checkpoint streams for all settings.

The router probe baseline uses the class distribution in each saved router-sampled frame. Uniform chance is 1/K over observed non-unknown labels; the majority dummy baseline is the largest observed class fraction. Normalized excess is (probe - baseline) / (1 - baseline).

source	mean classes	uniform chance	majority chance	excess over uniform	excess over majority
posterior_mu	20.0000	0.0500	0.0813	0.4887	0.4713
router_sampled_z	19.9962	0.0500	0.0874	0.1283	0.0924

Multi-task router probe against dummy baselines

Multi-task router probe normalized excess

Multi-task training sufficiency diagnostic

Best-Setting Dynamics

The best complete multi-task final-test setting by within-task matched router score is tokcc_a0p25_b0p01_lin100. The plots below show the validation trajectory and latent geometry for that setting only.

Best multi-task router-sampled z-space static frames

Best multi-task router-sampled z-space animation

Best multi-task posterior-mu z-space static frames

Best multi-task posterior-mu z-space animation

Best multi-task smoothed posterior-mu z-space animation

Interpretation Note

The complete final-test rows support statements about all 72 multi_task cells in the planned grid.

Repo-Side Artifacts

Multi-task aggregate: path:experiments/synthetic_sequences/disentangled/analysis/aggregates/exp2_continuous_cvae-Apr27-paper-basic-grid_20260429/multitask
Report: path:experiments/synthetic_sequences/disentangled/analysis/aggregates/exp2_continuous_cvae-Apr27-paper-basic-grid_20260429/multitask/report.md
Findings JSON: path:experiments/synthetic_sequences/disentangled/analysis/aggregates/exp2_continuous_cvae-Apr27-paper-basic-grid_20260429/multitask/findings.json
Manifest JSON: path:experiments/synthetic_sequences/disentangled/analysis/aggregates/exp2_continuous_cvae-Apr27-paper-basic-grid_20260429/multitask/aggregate_manifest.json
Generation script: path:experiments/synthetic_sequences/disentangled/analysis/run_continuous_cvae_apr27_multitask_dynamics_20260429.py