Additional Model Families Beyond Qwen

This report records the extension of the paper's pretrained multi-task experiments beyond Qwen in one place. SmolLM2-360M and OLMo 2 1B completed successfully and reproduce the paper's qualitative objective separation. Pythia-410M did not learn the source benchmark adequately and is retained as a diagnostic. Both Gemma sources learned the benchmark, and all twelve Gemma factorization roles completed after the six OOM retries were reduced to evaluation batch size 16. Gemma's scheduled-validation and analogical results are usable, but its large final-test generation pass still exhibits a model-specific parse collapse. Distinct per-chunk sampling seeds did not resolve that discontinuity.

• design
• Model compatibility validation
• results
• analysis
• Pythia diagnostic

Artifacts

• Analysis script: path:experiments/synthetic_sequences/disentangled/analysis/run_cross_family_small_20260725.py
• Reproducible analysis: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/
• Machine-readable findings: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/findings.json
• Task-level endpoints: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/data/task_endpoint_metrics.csv
• All-model endpoints: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/data/additional_model_endpoints.csv
• All-model task endpoints: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/data/additional_model_task_endpoints.csv
• Gemma validation and analogical endpoints: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/data/gemma_validation_endpoints.csv
• Gemma task endpoints: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/data/gemma_validation_task_endpoints.csv
• Gemma analysis script: path:experiments/synthetic_sequences/disentangled/analysis/run_gemma_followup_results_20260727.py
• Runtime comparison: path:experiments/synthetic_sequences/disentangled/analysis/collections/cross-family-small_20260724/data/runtime_comparison.csv
• Pythia three-epoch analysis: path:experiments/synthetic_sequences/disentangled/analysis/collections/pythia-410m-3epoch-followup_20260725/
• Pythia three-epoch findings: path:experiments/synthetic_sequences/disentangled/analysis/collections/pythia-410m-3epoch-followup_20260725/findings.json

Context

• Qwen2.5 paper-method reference

Runs

• Initial entangled collection: synthetic-pretrained-entangled-v2-multi-task-cross-family-small (2 jobs)
• Initial factorization collection: synthetic-pretrained-disentangled-from-entangled-v2-paper-selected-multi-task-z64-cross-family-small (12 jobs)
• Pythia three-epoch entangled collection: synthetic-pretrained-entangled-v2-multi-task-pythia-410m-3epoch-followup (1 job)
• Pythia three-epoch factorization collection: synthetic-pretrained-disentangled-from-entangled-v2-paper-selected-multi-task-z64-pythia-410m-3epoch-followup (6 jobs)
• Usable SmolLM2 arm: 7/7 jobs complete
• Pythia three-epoch source job: 2163970, completed in 4:17:15
• Pythia three-epoch factorization jobs: 2163971–2163975 completed; 2163976 failed
• OLMo 2 / Gemma 3 entangled collection: synthetic-pretrained-entangled-v2-multi-task-olmo2-1b-gemma3-270m (2 jobs)
• OLMo 2 / Gemma 3 factorization collection: synthetic-pretrained-disentangled-from-entangled-v2-paper-selected-multi-task-z64-olmo2-1b-gemma3-270m (12 jobs)
• OLMo 2 entangled SFT: 2166337; dependent factorizations: 2166339–2166344
• Gemma 3 entangled SFT: 2166338; dependent factorizations: 2166345–2166350
• OLMo 2 1B: SFT and all six factorizations completed
• Gemma 3 270M: SFT and all six factorizations completed; retries 2171730–2171732 completed in 4:29:53–4:48:26
• Gemma 3 1B PT entangled collection: synthetic-pretrained-entangled-v2-multi-task-gemma3-1b-pt (1 job)
• Gemma 3 1B PT factorization collection: synthetic-pretrained-disentangled-from-entangled-v2-paper-selected-multi-task-z64-gemma3-1b-pt (6 jobs)
• Gemma 3 1B PT entangled SFT: 2167077; dependent factorizations: 2167078–2167083
• Gemma 3 1B PT: SFT and all six factorizations completed; retries 2171733–2171735 completed in 6:31:56–7:04:52
• Retry configuration: evaluation batch size 16, pinned snapshot commit 5bb11913, submission group gemma-eval-b16-rng-fix-v2
• Logical-job state at 2026-07-27 15:16 EDT: 42 total; 40 completed, 2 failed, 0 running, and 0 pending

Active Follow-Up Models

• allenai/OLMo-2-0425-1B
• google/gemma-3-270m
• google/gemma-3-1b-pt