Synthetic Sequences Tasks And Data

This setting uses deterministic symbolic tasks where one input can admit multiple valid solution strategies. The tasks are designed so strategy affects the intermediate trace, while the final answer remains checkable by task-specific validators.

Task Family

The current task suite contains six task families:

list_summation: left-to-right, right-to-left, pairwise
sorting_algorithms: bubble-sort, selection-sort, insertion-sort, merge-sort, heap-sort
grid_pathfinding: right-first, down-first, alternating
linear_equation_solving: subtract-then-divide, divide-then-subtract, inverse-ops
base_conversion: repeated-division, via-binary, decomposition
multidigit_addition: right-to-left-carry, left-to-right-partials, rounding-decomposition

Each task defines a fixed strategy set with deterministic strategy attribution over generated traces. The repo-side task contract is documented in path:experiments/synthetic_sequences/tasks/README.md.

Canonical Sequence Format

All tasks use the same framed text format:

<bos> <x> ... </x> <y> ... </y> <eos>

When a latent strategy token is made explicit, the format extends to:

<bos> <x> ... </x> <z> <z_i> </z> <y> ... </y> <eos>

This shared format lets the same training and evaluation pipeline operate across all tasks.

Validation And Strategy Attribution

Each task owns semantic generation, parsing, and validation. The important property is that the task can deterministically answer three questions about a generated solution:

did parsing succeed?
is the output structurally valid?
which strategy or strategies match the produced trace?

This makes the setting useful for disentanglement work because latent behavior can be measured against exact strategy labels instead of subjective annotations.

Dataset Contract

The data pipeline exports per-task train, validation, and test splits under .data/synthetic_sequences. Parquet rows include tokenized (x, y) sequences, structural masks, strategy metadata, and validation fields. The export contract is documented in path:experiments/synthetic_sequences/data/README.md.

Why This Setting Matters

Synthetic sequences provide a controlled environment where the project can separate three questions that are usually entangled in open-ended language tasks:

whether a model solves the task at all,
whether it preserves strategy information internally,
and whether an explicit latent variable can recover or control that strategy structure.