Capacity-constrained semantic strategy factorizations

• 1. Purpose
• 2. Setup
• 3. Semantic hierarchies as task-dependent output partitions
• 4. Two notions of strategy recovery
• 5. Hard attribute factorizations
• 6. Relabeling obstruction
• 7. Capacity-constrained tree cuts
• 8. Top-level branches are encoded before substrategy refinements
• 9. Capacity path and “more branches are encoded”
• 10. Why an $H(Z\mid X)$ penalty alone is not enough
• 11. From weak recovery to strong recovery: semantic synchronization
• 12. Related-task graph
• 13. Alignment penalty for weakly recovered factorizations
• 14. Lexicographic and regularized objectives
• 15. Complexity as an alignment mechanism
• 16. Summary of the proposed theory
  • 16.1. Local branch discovery
  • 16.2. Hierarchical capacity path
  • 16.3. Global semantic alignment
• 17. Failure modes and limitations
• 18. Relation to existing literature
• 19. Main takeaway

---

1. Purpose

The baseline theory shows that, for a fixed conditional law $p(y\mid x)$ and a feasible latent factorization

$$p(y\mid x)=\sum_z r(z\mid x)g(y\mid x,z),$$

the max-info criterion satisfies

$$I(Y;Z\mid X) = H(Y\mid X)-H(Y\mid X,Z) = \mathbb E_X\operatorname{JSD}_{r(\cdot\mid X)}\{g(\cdot\mid X,z)\}_z.$$

Thus max-info minimizes residual generator entropy. In highly expressive classes, this becomes degenerate: if deterministic feasible generators exist, max-info selects every deterministic simulator, and the latent $Z$ need not have stable semantics across inputs.

This note develops a constrained theory meant to model multi-strategy reasoning. The intended setting is:

• $x$ is a task, e.g. a theorem statement;
• $y$ is a solution, e.g. a proof;
• $y$ may be generated by one of several abstract strategies, e.g. direct proof, contradiction, induction, construction;
• the same abstract strategy has different task-specific realizations for different $x$.

Semantic consistency means that the latent index $z$ names the same abstract branch of the task-conditioned output space across inputs.

The theory below separates three levels:

1. capacity-constrained branch discovery: max-info recovers taskwise output branches;
2. capacity path: increasing $|\mathcal Z|$ recovers deeper tree refinements;
3. semantic synchronization: an additional cross-input simplicity/equivariance penalty aligns the branch labels globally.

---

2. Setup

Let $X,Y$ be finite, with fixed input law $\mu$ on $\mathcal X$ and fixed conditional law $p(y\mid x)$ on $\mathcal Y$.

All entropies and mutual informations are computed under the relevant induced joint law.

For a set $A\subseteq \mathcal Y$, write

$$p(A\mid x):=\sum_{y\in A}p(y\mid x).$$

If $p(A\mid x)>0$, define the conditional law

$$p(y\mid x,A) := \frac{p(y\mid x)\mathbf 1\{y\in A\}}{p(A\mid x)}.$$

---

3. Semantic hierarchies as task-dependent output partitions

Intuitively, a reasoning process can be viewed as a branching process represented as tree where each branch corresponds a strategic choice at a given level of hierarchy. We formalize this as a tree where each node of the tree is an input-dependent subset of outputs. The root node reflects all possible outputs, and each branch divides the output space into subsets reflecting different strategic choices.

The tree nodes are semantic objects. Their realizations $B_u(x)$ may vary with $x$.

For example, the root is “all proofs of theorem $x$.” Its depth-$1$ children may be “direct proofs,” “proofs by contradiction,” “induction proofs,” etc. For different theorems, the actual proofs in those sets are different, but the node name is globally meaningful.

No Markov condition such as $Y\perp X\mid u$ is imposed. The branch-conditioned law $p(y\mid x,Y\in B_u(x))$ may depend arbitrarily on $x$.

Intuitively, a cut chooses the resolution at which we describe the output. A shallow cut records a coarse strategy; a deeper cut records a more refined substrategy.

Examples and Intuition

For example, suppose for a theorem statement $x$, proofs are organized as

$$\text{all proofs} \to \begin{cases} \text{direct proofs},\\ \text{contradiction proofs},\\ \text{induction proofs}. \end{cases}$$

and induction proofs further split as

$$\text{induction proofs} \to \begin{cases} \text{weak induction},\\ \text{strong induction}. \end{cases}$$

Then a top-level cut is

$$C = \{\text{direct},\text{contradiction},\text{induction}\}.$$

This cut records only the high-level proof strategy.

A mixed-resolution cut is

$$C = \{\text{direct},\text{contradiction},\text{weak induction},\text{strong induction}\}.$$

This cut keeps direct and contradiction coarse, but refines induction into its two subtypes.

The set

$$\{\text{induction}\}$$

is not a cut, because direct and contradiction proofs are not covered.

The set

$$\{\text{induction},\text{strong induction}\}$$

is also not a cut, because a strong-induction proof lies on a path passing through both selected nodes. It would be assigned two branch labels.

The "exactly once" condition rules out both failures: every output must receive one and only one branch label.

Then

$$p(y\mid x)=\sum_{u\in C}r_C(u\mid x)g_C(y\mid x,u).$$

Thus every semantic cut gives a feasible factorization.

Preview: Why cuts matter for capacity-constrained max-info

If the latent space has size $m$, then $Z$ can represent at most an $m$-cell cut. Maximizing

$$I(Y;Z\mid X)$$

over cut-induced factorizations asks which $m$-branch resolution captures the most information about $Y$ beyond $X$.

Since $Z=\phi_{C_X}(X,Y)$ is deterministic given $(X,Y)$,

$$I(Y;Z\mid X)=H(Z\mid X).$$

Thus, among admissible cuts of size $m$, max-info chooses the cut with largest conditional branch entropy.

This formalizes the idea that limited-capacity $Z$ should encode the most important branching structure first, provided the strategy tree and admissibility assumptions rule out arbitrary non-semantic partitions.

---

4. Two notions of strategy recovery

Let $S$ denote a target strategy variable with $K$ states. In the tree model, $S$ may be the top-level depth-$1$ branch label.

A learned latent $Z$ may recover $S$ in two different senses.

This means each individual task has been decomposed into the right branches, but the latent labels may be inconsistent across tasks.

Strong recovery is the semantic target: the same latent label means the same abstract strategy across inputs.

---

5. Hard attribute factorizations

The tree model is a special case of hard output attributes.

Intuition: concentrated $q(z|x,y)$

This definition relates to our discussion about potentially incorporating a term in the objective that pressures the latent variable to be concentrated given $(x,y)$, with the intuition that a strategy $Z$ is an attribute of $Y$, and in particular that a strategy is determined after observing its instantiation.

^hard-attribute-identity

\begin{proof} Feasibility follows from the law of total probability:

$$p(y\mid x) = \sum_z \mathbb P(\phi(x,Y)=z\mid X=x) p(y\mid x,\phi(x,Y)=z).$$

Because $Z=\phi(X,Y)$ is deterministic given $(X,Y)$, $H(Z\mid X,Y)=0$. Therefore

$$I(Y;Z\mid X)=H(Z\mid X)-H(Z\mid X,Y)=H(Z\mid X).$$

\end{proof}

Thus, for hard branch factorizations, max-info becomes a conditional entropy maximization problem.

^hard-attribute-capacity

---

6. Relabeling obstruction

The following result states that the max-info objective is invariant to $x$-dependent relabeling, obstructing the recovery of a globally-aligned strategy structure.

^x-dependent-relabeling-invariance

\begin{proof} For each fixed $x$, $\phi^\pi(x,Y)$ is a bijective relabeling of $\phi(x,Y)$. Therefore

$$H(\phi^\pi(x,Y)\mid X=x)=H(\phi(x,Y)\mid X=x).$$

Averaging over $X$ and using #^hard-attribute-identity gives the claim. \end{proof}

Consequently, max-info can at best recover branches up to $x$-dependent permutation unless the model class or objective contains an additional cross-input alignment principle.

---

7. Capacity-constrained tree cuts

We now specialize hard attributes to tree cuts. The simplified viewpoint is:

1. a cut $C$ chooses a semantic resolution;
2. the cut induces the hard attribute $A_C$;
3. max-info equals $H(A_C\mid X)$;
4. refining a node adds a local entropy gain;
5. capacity-constrained max-info chooses the cut with maximal total gain.

When comparing a semantic cut to a learned latent variable, the cut should be viewed as an unordered partition. The same cut can be encoded by many latent variables, including variables that relabel the cut cells differently for different inputs.

Equivalently, $\operatorname{Split}(C)$ is the set of branch decisions exposed before the tree reaches the resolution $C$. If $u\in C$, then $u$ is not split; the cut stops at $u$. If descendants of $u$ lie in $C$, then the cut has refined below $u$.

Thus $G(u)$ is the expected information gained by asking: conditional on $Y$ lying in branch $u$, which child of $u$ contains $Y$? It is large when branch $u$ is reached often and its children are balanced.

\begin{proof} Condition on $X=x$. The branch label $A_C$ records which cell of the cut $C$ contains $Y$. The refined label $A_{C^{+u}}$ agrees with $A_C$ on every branch except $u$. On the event $Y\in B_u(x)$, it replaces the coarse label $u$ by the child of $u$ containing $Y$.

Since $C^{+u}$ refines $C$, the coarse label $A_C$ is a deterministic function of the refined label $A_{C^{+u}}$. Hence, by the conditional chain rule for entropy,

$$H(A_{C^{+u}}\mid X=x) = H(A_{C^{+u}}, A_C \mid X=x) = H(A_C\mid X=x) + H(A_{C^{+u}}\mid X=x,A_C).$$

Rearranging terms gives

$$H(A_{C^{+u}}\mid X=x) - H(A_C\mid X=x) = H(A_{C^{+u}}\mid X=x,A_C).$$

We now compute $H(A_{C^{+u}}\mid X=x,A_C) \equiv \sum_{w \in C} \mathbb P(A_C = w) \, H(A_{C^{+u}}\mid X=x,A_C=w)$. The conditional entropy term $H(A_{C^{+u}}\mid X=x,A_C=w)$ is zero unless $A_C=u$. Indeed, if $A_C=w$ for some $w\in C\setminus\{u\}$, then the split did not affect that branch and $A_{C^{+u}}=A_C=w$ deterministically.

On the event $A_C=u$, equivalently $Y\in B_u(x)$, the refined label $A_{C^{+u}}$ is the child $v\in\operatorname{ch}(u)$ such that $Y\in B_v(x)$. Therefore its conditional law is

$$\mathbb P(A_{C^{+u}}=v\mid X=x,A_C=u) = \frac{p(B_v(x)\mid x)}{p(B_u(x)\mid x)} = \alpha_{u,v}(x).$$

Thus

$$H(A_{C^{+u}}\mid X=x,A_C=u) = H(\alpha_u(x)),$$

where $\alpha_u(x):=(\alpha_{u,v}(x))_{v\in\operatorname{ch}(u)}$.

Combining the preceding observations,

$$H(A_{C^{+u}}\mid X=x,A_C) = p(B_u(x)\mid x)\,H(\alpha_u(x)).$$

Hence

$$H(A_{C^{+u}}\mid X=x) - H(A_C\mid X=x) = p(B_u(x)\mid x)\,H(\alpha_u(x)).$$

Averaging over $x\sim\mu$ gives

$$H(A_{C^{+u}}\mid X)-H(A_C\mid X) = \mathbb E_X\left[ p(B_u(X)\mid X)H(\alpha_u(X)) \right].$$

By definition, the right-hand side is $G(u)$. Therefore

$$H(A_{C^{+u}}\mid X)-H(A_C\mid X)=G(u).$$

\end{proof} The following result states that the information encoded by a tree-cut attribute $A_C$ about the output $Y$ beyond the input $X$ is equal to the sum of the entropy gains from all branch decisions exposed above the cut.

^cut-entropy-split-gains

\begin{proof} The identity $I(Y;A_C\mid X)=H(A_C\mid X)$ follows from #^hard-attribute-identity, since $A_C=a_C(X,Y)$ is deterministic given $(X,Y)$.

It remains to show

$$H(A_C\mid X)=\sum_{u\in\operatorname{Split}(C)}G(u).$$

Let $\operatorname{Split}(C)$ be the set of internal nodes lying strictly above the cut. Starting from the root cut $C_0=\{\rho\}$, choose any ordering

$$u_1,\dots,u_k$$

of $\operatorname{Split}(C)$ such that ancestors appear before descendants. Such an ordering exists because $\operatorname{Split}(C)$ is ancestry-closed. Define cuts recursively by

$$C_i:=C_{i-1}^{+u_i} = (C_{i-1}\setminus\{u_i\})\cup\operatorname{ch}(u_i).$$

Then $C_k=C$.

By the entropy-gain lemma for splitting a node,

$$H(A_{C_i}\mid X)-H(A_{C_{i-1}}\mid X)=G(u_i)$$

for each $i=1,\dots,k$. Summing over $i$ gives

$$H(A_C\mid X)-H(A_{\{\rho\}}\mid X) = \sum_{i=1}^k G(u_i).$$

The root cut has only one possible branch label, so $H(A_{\{\rho\}}\mid X)=0$. Therefore

$$H(A_C\mid X)=\sum_{u\in\operatorname{Split}(C)}G(u).$$

Combining this with $I(Y;A_C\mid X)=H(A_C\mid X)$ gives

$$I(Y;A_C\mid X) = H(A_C\mid X) = \sum_{u\in\operatorname{Split}(C)}G(u).$$

\end{proof}

The preceding decomposition turns the max-info objective for a cut into a sum of local split gains. The next result adds the capacity constraint: if $Z$ may represent at most $m$ branch labels, then max-info selects the cut whose exposed branch decisions have the largest total entropy gain subject to the $m$-cell budget. Equivalently, capacity-constrained max-info over tree cuts is a tree-structured, precedence-constrained gain maximization problem.

\begin{proof} Recall that $A_C$ is a deterministic property of $(X,Y)$ and represents the branch of the tree $\mathcal{T}_X$ containing $Y$ at resolution $C$, hence $I(Y; A_C | X) = H(A_C|X)$ by #^hard-attribute-identity. Immediately from #^cut-entropy-split-gains we have that for each cut $C$, the mutual information between $Y$ and the latent variable $A_C$ is

$$I(Y; A_C | X) = H(A_C | X) = \sum_{u \in \operatorname{Split}(C)} G(u).$$

Therefore

$$\max_{C\in\mathfrak C_m}I(Y;A_C\mid X) = \max_{C\in\mathfrak C_m}\sum_{u\in\operatorname{Split}(C)}G(u).$$

If $C^\star$ maximizes the right-hand side, then it also maximizes $I(Y;A_C\mid X)$ over cuts with $|C|\le m$. The branch-induced factorization associated with $C^\star$ is feasible because the sets $\{B_u(x):u\in C^\star\}$ partition $\operatorname{supp}p(\cdot\mid x)$:

$$p(y\mid x)=\sum_{u\in C^\star}p(B_u(x)\mid x)\,p(y\mid x,Y\in B_u(x)).$$

It uses $|C^\star|\le m$ latent states and has latent variable $A_{C^\star}$. Thus it is max-info among tree-cut factorizations with at most $m$ states, up to relabeling of the latent states. \end{proof}

The theorem can also be written in precedence-constrained form. Let $E$ range over ancestor-closed sets of internal nodes, where ancestor-closed means $u\in E$ implies every proper ancestor of $u$ lies in $E$. Such an $E$ is exactly the same object as $\operatorname{Split}(C)$ for some cut $C$. Therefore the preceding theorem is equivalent to

This is the same gain maximization problem, but cuts are the primary object and $E$ is only a shorthand for the nodes split by the cut. The result follows by the following simple calculation on the size of $|C|$ in terms of $\operatorname{Split}(C)$.

^cut-size-split-nodes \begin{proof} Start from the root cut, which has one cell. Splitting an internal node $u$ replaces one cell by $d(u)$ children, increasing the number of cut cells by $d(u)-1$. To obtain $C$, one performs exactly the splits in $\operatorname{Split}(C)$. \end{proof}

This theorem gives a precise interpretation of capacity-constrained max-info in a branching model:

Max-info chooses the branch decisions with largest total entropy gain, subject to the rule that a substrategy can be encoded only after its ancestors have been encoded.

---

8. Top-level branches are encoded before substrategy refinements

The preceding results show that, in the tree-cut model, max-info chooses a cut by maximizing total split gain subject to a latent-state budget. We now isolate the simplest consequence of this framework: if the root split has $K$ children, then capacity $K$ is exactly enough to expose the top-level strategy branches, but not enough to refine any one of them further.

This formalizes the sense in which top-level branches are encoded before substrategy refinements. The point is not that max-info intrinsically understands semantic depth. Rather, within the admissible class of tree cuts, any non-root refinement requires first exposing the root split and then spending at least one additional latent state. Therefore, with only $K$ available latent states, the only nontrivial cut that can expose all top-level branches is the depth-$1$ cut.

Let the root have $K$ children, interpreted as the top-level strategies:

$$\operatorname{ch}(\rho)=\{s_1,\dots,s_K\}.$$

Let $C_1:=\operatorname{ch}(\rho)$ be the depth-$1$ cut, and let $S:=A_{C_1} $ be the corresponding top-level strategy attribute. Thus $S$ records which child of the root contains the output $Y$.

Informally, the theorem below says:

If $H(S\mid X)>0$, so the top-level strategy is not deterministic from $X$, then among all tree-cut factorizations using at most $K$ latent states, max-info selects the top-level strategy cut $C_1$. Any substrategy refinement would require at least $K+1$ states.

Equivalently, at capacity $K$, max-info recovers the top-level partition

$$\{B_s(x):s\in\operatorname{ch}(\rho)\}$$

for each input $x$. However, the latent labels may still be permuted differently for different inputs, so this is weak recovery rather than global semantic alignment.

^top-level-recovery

\begin{proof} A cut with at most $K$ cells is either the root cut $\{\rho\}$ or the depth-$1$ cut $\operatorname{ch}(\rho)$, except for refinements that would exceed the budget. Indeed, splitting the root creates $K$ cells. Any further split of a non-root internal node replaces one cell by at least two children, increasing the number of cells to at least $K+1$.

The root cut has entropy $0$. The depth-$1$ cut has entropy $H(S\mid X)>0$. Therefore the depth-$1$ cut is the unique entropy-maximizing cut under the capacity $|\mathcal Z|\le K$. By #^hard-attribute-identity, it is also the unique max-info partition. Latent labels may still be permuted separately for each $x$ by #^x-dependent-relabeling-invariance. \end{proof}

^substrategies-after-top-level

\begin{proof} Any non-root split requires first splitting the root, producing $K$ cells, and then replacing at least one root child by at least two descendants. This produces at least $K+1$ cells. \end{proof}

If one wants nontrivial behavior for capacities $m<K$, one should include a coarser semantic tree above the $K$ named strategies, e.g. first split “constructive vs. nonconstructive,” then split “nonconstructive” into contradiction, contrapositive, etc. The same theorem then applies to that coarser hierarchy.

---

9. Capacity path and “more branches are encoded”

The preceding theorem identifies the first strategy level. The next result describes what happens as the capacity increases.

^capacity-path

\begin{proof} The first statement is exactly #^tree-cut-knapsack. If $\operatorname{Split}(C_m^*)\subseteq\operatorname{Split}(C_{m'}^*)$, then the larger-capacity cut performs all splits of the smaller-capacity cut and possibly more. Hence $C_{m'}^*$ refines $C_m^*$. \end{proof}

A more explicit sufficient condition is available for binary trees.

^binary-ordered-gains

\begin{proof} Because ancestor gains are larger than descendant gains, the set of the $m-1$ largest gains is ancestor-closed: if it contains a node, it contains every ancestor. Since all split costs are equal to one, maximizing total gain under $|\operatorname{Split}(C)|\le m-1$ is achieved by choosing the $m-1$ largest gains. The capacity-constrained tree-cut theorem then gives the result. \end{proof}

This formalizes the intuition that a capacity-limited latent variable encodes high-level branches first when high-level splits have larger entropy gain than lower-level refinements and the admissible class respects the hierarchy.

---

10. Why an $H(Z\mid X)$ penalty alone is not enough

One might try to use

$$I(Y;Z\mid X)-\lambda H(Z\mid X)$$

to prevent degeneracy. For hard attributes, this penalty does not create a new semantic preference.

^entropy-penalty-hard

\begin{proof} By #^hard-attribute-identity, $I(Y;Z\mid X)=H(Z\mid X)$. Substitution gives the result. \end{proof}

Thus, in the hard-branch regime, the useful capacity controls are cardinality, tree-cut complexity, description length, or an explicit cross-input alignment penalty, not a bare $H(Z\mid X)$ penalty.

---

11. From weak recovery to strong recovery: semantic synchronization

The tree-cut results recover the right branches locally, but only up to $x$-dependent permutation. To recover a globally meaningful latent label, we need a cross-input alignment principle.

The central idea is:

Max-info discovers local branches. Simplicity/equivariance synchronizes their labels across tasks.

12. Related-task graph

Let $G_X=(\mathcal X,E)$ be a graph connecting related inputs. An edge $(x,x')$ means that branch realizations at $x$ and $x'$ can be compared by some low-complexity transformation.

For top-level strategies $s,t\in[K]$, assume a local semantic matching cost

$$d_{x,x'}(s,t)\ge 0.$$

Intuitively, $d_{x,x'}(s,t)$ is small if the strategy-$s$ component at task $x$ can be transformed simply into the strategy-$t$ component at task $x'$.

A flexible way to define such a cost is

$$d_{x,x'}(s,t) := \inf_{T\in\mathcal T_{x,x'}} \Bigl\{ \operatorname{Comp}(T) + \eta D\bigl(T_\#p(\cdot\mid x,S=s),\ p(\cdot\mid x',S=t)\bigr) \Bigr\},$$

where:

• $\mathcal T_{x,x'}$ is a class of simple transformations between output spaces or proof representations;
• $\operatorname{Comp}(T)$ is a complexity cost;
• $D$ is a divergence or discrepancy;
• $T_\#$ is pushforward.

Known group equivariance is the special case where $\mathcal T_{x,x'}$ contains a known transformation. For mathematical reasoning, the transformations need not be explicitly known; they can be treated as low-complexity transports implicitly favored by a shared generator.

This says that matching the same abstract strategy across nearby inputs is strictly simpler than matching different strategies.

13. Alignment penalty for weakly recovered factorizations

Suppose a weakly recovered factorization is indexed by local permutations $\pi_x$, so that

$$g(\cdot\mid x,z)=p(\cdot\mid x,S=\pi_x(z)).$$

Define

$$\mathcal A(\pi) := \sum_{(x,x')\in E}\sum_{z\in[K]} d_{x,x'}\bigl(\pi_x(z),\pi_{x'}(z)\bigr).$$

^semantic-synchronization

\begin{proof} If $\pi_x\equiv\pi$ is constant over $x$, then for every edge $(x,x')$ and every $z$,

$$\pi_x(z)=\pi_{x'}(z),$$

so each summand is diagonal and $\mathcal A(\pi)=0$.

Conversely, if $\mathcal A(\pi)=0$, then every summand is zero. By the diagonal alignment gap,

$$\pi_x(z)=\pi_{x'}(z)$$

for every edge $(x,x')$ and every $z$. Since $G_X$ is connected, this equality propagates along paths, so $\pi_x(z)$ is independent of $x$ for every $z$. Hence $\pi_x\equiv\pi$ globally. \end{proof}

This is a permutation-synchronization theorem: local branch identities are aligned by enforcing cycle-consistent, low-cost correspondences across tasks.

---

14. Lexicographic and regularized objectives

The preceding theorem can be implemented as a lexicographic selection rule:

$$\text{first maximize } I(Y;Z\mid X), \qquad \text{then minimize } \mathcal A(g).$$

Equivalently, one may use a regularized objective

$$I(Y;Z\mid X)-\lambda \mathcal A(g), \qquad \lambda>0,$$

provided $\mathcal A(g)\ge 0$ and some globally aligned weak maximizer has $\mathcal A(g)=0$.

^regularized-strong-recovery

\begin{proof} By #^top-level-recovery, the max-info value in the top-level tree-cut class is achieved exactly by weakly recovered top-level factorizations, up to relabeling and null states. Among these, #^semantic-synchronization says that $\mathcal A$ is minimized exactly by globally aligned permutations, and that aligned solutions have penalty $0$.

Any non-max-info solution has strictly smaller $I(Y;Z\mid X)$ and nonnegative penalty, so it cannot beat a globally aligned max-info solution with penalty $0$. Any weak but non-globally-aligned max-info solution has the same information but strictly positive penalty, so it is worse for every $\lambda>0$. Therefore every regularized maximizer is globally aligned. \end{proof}

---

15. Complexity as an alignment mechanism

The alignment penalty can be interpreted as generator complexity.

Suppose the aligned generator can be written as

$$g(y\mid x,z)=G_\theta(y\mid x,e_z),$$

where $G_\theta$ is a shared interpreter and $e_z$ is a strategy embedding.

A globally aligned factorization has description length roughly

$$C_{\mathrm{aligned}} \approx C(G_\theta)+\sum_z C(e_z).$$

An $x$-dependent relabeling requires an additional input-dependent permutation table:

$$g(y\mid x,z)=G_\theta(y\mid x,e_{\pi_x(z)}),$$

with complexity

$$C_{\mathrm{misaligned}} \approx C(G_\theta)+\sum_z C(e_z)+C(\{\pi_x\}_x).$$

^complexity-selects-alignment

\begin{proof} The max-info step restricts to the weakly recovered solution set. On that set, minimizing $C(g^\pi)$ is equivalent to minimizing $\Omega(\pi)$, whose zero set consists exactly of globally constant permutations. \end{proof}

This formalizes the idea that a simpler generator applies a given $z$ “the same way” across inputs, while inconsistent semantics require extra machinery to implement task-dependent relabelings.

---

16. Summary of the proposed theory

The extension decomposes strategy recovery into three logically separate mechanisms.

16.1. Local branch discovery

Under a hard semantic hierarchy, every tree cut induces a feasible factorization. For such factorizations,

$$I(Y;Z\mid X)=H(Z\mid X).$$

Thus capacity-constrained max-info selects the branch partition with maximal conditional branch entropy.

16.2. Hierarchical capacity path

For a semantic tree, each internal node $u$ has a split gain

$$G(u)= \mathbb E_X\left[ p(B_u(X)\mid X) H\bigl((\alpha_{u,v}(X))_{v\in\operatorname{ch}(u)}\bigr) \right],$$

where $\alpha_{u,v}(x)=p(B_v(x)\mid x)/p(B_u(x)\mid x)$ on positive-mass branches. For every cut $C$,

$$I(Y;A_C\mid X)=\sum_{u\in\operatorname{Split}(C)}G(u).$$

Thus max-info with $m$ latent states chooses the cut $C$ with $|C|\le m$ whose split nodes have maximal total gain. This gives a precise sense in which increasing capacity encodes more branches.

16.3. Global semantic alignment

Max-info alone is invariant under $x$-dependent relabeling. Therefore it gives at most weak recovery. To obtain strong recovery, add an alignment or complexity penalty. If same-strategy components are easier to align across related tasks than different-strategy components, then minimizing the alignment penalty synchronizes the local labels and recovers strategies up to a global permutation.

---

17. Failure modes and limitations

1. Capacity alone is insufficient without an admissible hierarchy.
   If arbitrary partitions of $\mathcal Y$ are allowed, max-info may choose mass-balanced or low-level attributes rather than semantic strategies.

2. Top-level branches are not automatically information-dominant.
   A stylistic or local attribute can have higher conditional entropy than a reasoning strategy. The hierarchy restriction or an entropy-gap condition is doing real work.

3. Weak recovery is the most max-info can provide by itself.
   The objective is invariant under $x$-dependent relabeling, so global semantics cannot be identified from max-info alone.

4. Disconnected task graph gives only componentwise alignment.
   If $G_X$ has multiple connected components, synchronization recovers one global permutation per component.

5. Indistinguishable strategies are not identifiable.
   If $d_{x,x'}(s,t)$ has no diagonal gap, or if two strategies induce identical branch-conditioned distributions across all relevant tasks, no objective can distinguish them without additional side information.

6. Zero-mass branches are locally unidentifiable.
   If $p(S=s\mid x)=0$, then the realization of strategy $s$ at input $x$ cannot be inferred from $p(\cdot\mid x)$.

---

18. Relation to existing literature

1. Max-info and Jensen–Shannon divergence.
   The baseline identity

   $$I(Y;Z\mid X)=\mathbb E_X\operatorname{JSD}_{r(\cdot\mid X)}\{g(\cdot\mid X,z)\}_z$$

   uses the weighted generalized Jensen–Shannon divergence introduced by Lin.
   URL: https://doi.org/10.1109/18.61115

2. Information bottleneck and hard clustering.
   The hard-attribute restriction resembles a conditional version of deterministic information bottleneck / hard clustering: one searches for a limited discrete representation preserving information about a target variable.
   Tishby–Pereira–Bialek: https://www.princeton.edu/~wbialek/our_papers/tishby+al_99.pdf
   Strouse–Schwab deterministic IB: https://doi.org/10.1162/NECO_a_00810

3. Tree cuts and hierarchical clustering.
   The split-gain decomposition is the entropy-chain-rule analogue of optimizing a hierarchical decision tree cut under a leaf budget.

4. Permutation synchronization.
   The semantic synchronization step is formally related to group and permutation synchronization: recovering globally consistent labels from pairwise relative correspondences.
   Singer angular synchronization: https://pmc.ncbi.nlm.nih.gov/articles/PMC3003935/
   Pachauri–Kondor–Singh permutation synchronization: https://papers.nips.cc/paper/4987-solving-the-multi-way-matching-problem-by-permutation-synchronization
   Shen–Huang–Srebro–Sanghavi map synchronization: https://papers.nips.cc/paper/6128-normalized-spectral-map-synchronization

---

19. Main takeaway

A precise nondegenerate theory should not claim that max-info alone recovers semantic strategies. The sharper statement is:

$$\boxed{ \text{max-info + branch capacity} \Rightarrow \text{local branch recovery up to }x\text{-dependent permutation} }$$

and

$$\boxed{ \text{local branch recovery + simplicity/equivariance synchronization} \Rightarrow \text{global semantic recovery up to one permutation.} }$$

This cleanly separates the roles of information, capacity, hierarchy, and semantic consistency.