Exploration Efficiency Kernel Calibration

This note gives dimension-aware calibration rules for the kernel-reweighted router sampler in Exploration efficiency methods. It is agent-drafted and should be treated as a proposal until reviewed against follow-up decoding runs.

The main correction is that $\sigma$ and $\beta$ should not be fixed constants across latent dimensions. In the current implementation, the kernel is applied in whitened router coordinates $u \sim \mathcal N(0,I_d)$, then mapped back to the original latent by $z=\mu_x+\sigma_x \odot u$. Therefore the right scale for $\sigma$ is the geometry of iid Gaussian samples in $d$ whitened dimensions, not the per-coordinate scale of the original router distribution.

Setup

Let $U,V \sim \mathcal N(0,I_d)$ be independent whitened router samples, let $h$ be the number of previous samples at the step being calibrated, and let

$$D_h(u) = \sum_{i=1}^{h}\exp\left(-\frac{\|u-u_i\|^2}{2\sigma^2}\right).$$

The rejection sampler uses

$$a(u) = \epsilon + (1-\epsilon)\exp(-\beta D_h(u)).$$

The calibration problem is to choose $\sigma$ so $D_h$ is neither numerically zero nor globally large, then choose $\beta$ so the resulting acceptance probabilities have a desired amount of downweighting without excessive rejection or fallback.

RBF Activity Criterion

For the actual RBF kernel, the most direct scale diagnostic is the expected pairwise kernel overlap:

$$\mathbb E\left[\exp\left(-\frac{\|U-V\|^2}{2\sigma^2}\right)\right] = \left(\frac{\sigma^2}{\sigma^2+2}\right)^{d/2}.$$

At a history length $h$, the expected kernel density is

$$\mathbb E[D_h(U)] = h\left(\frac{\sigma^2}{\sigma^2+2}\right)^{d/2}.$$

Given a target expected density $C$ at the chosen horizon $h$, solve for

$$q = \left(\frac{C}{h}\right)^{2/d}, \qquad \sigma = \sqrt{\frac{2q}{1-q}}.$$

This is the most useful first calibration rule for the current sampler. For an exploratory rejection sampler, a reasonable first target is often $C \in [0.1,1.0]$ at the final evaluation horizon, then refine by empirical acceptance diagnostics.

For the z64 exploration-efficiency sweep with $d=64$ and $h=31$, this gives:

The failed sweep used $\sigma \in \{0.5,0.75,1.0\}$, for which the maximum expected final-step density is only $1.7\times 10^{-14}$. That makes $\beta$ almost irrelevant because $a(u)\approx 1$ for every proposal.

Hard-Ball Affected-Mass Criterion

If the question is literally "what fraction of proposals fall within radius $\sigma$ of any previous sample?", use the pairwise distance distribution

$$\frac{\|U-V\|^2}{2} \sim \chi^2_d.$$

The probability that a new proposal falls within a hard ball of radius $\sigma$ around one prior sample is

$$p_\mathrm{pair}(\sigma; d) = F_{\chi^2_d}\left(\frac{\sigma^2}{2}\right).$$

The probability that it falls within radius $\sigma$ of at least one of $h$ independent history samples is approximately

$$p_\mathrm{any}(\sigma; d,h) = 1-\left(1-p_\mathrm{pair}(\sigma;d)\right)^h.$$

Given a target affected mass $M$, solve

$$\sigma = \sqrt{2F_{\chi^2_d}^{-1}\left(1-(1-M)^{1/h}\right)}.$$

This criterion answers the volume question directly, but it should not be confused with the softer RBF-density criterion. In high dimension it produces much larger radii. For $d=64,h=31$, affecting even 1% of proposals by a hard ball requires $\sigma\approx 8.04$, and affecting 10% requires $\sigma\approx 8.67$.

Beta Calibration

After choosing $\sigma$, choose $\beta$ using an acceptance target at a reference density $D^\star$. Rearranging the sampler's acceptance function gives

$$\beta = -\frac{\log\left(\frac{a^\star-\epsilon}{1-\epsilon}\right)}{D^\star},$$

where $a^\star$ is the desired acceptance probability at density $D^\star$.

For $\epsilon=0.02$, examples are:

This shows why $\beta=5$ can be either inert or very aggressive depending on $\sigma$. With $\sigma=0.75$, $D_h$ is effectively zero, so $\beta=5$ does nothing. With $\sigma=4.25$, the CPU sweep estimates median acceptance around 0.10 at $\beta=5$, which would be far more aggressive.

Lower target acceptance means stronger repulsion. Higher target acceptance means gentler repulsion.

CPU-Only Calibration Protocol

Before running GPU decoding, run a calibration pass that uses only sampled or saved whitened $u$ vectors:

1. Fix the coordinate system and effective dimension. For the current implementation this is full whitened latent space, so $d=64$ for z64 runs. If future work uses PCA, angular features, or a strategy-relevant subspace, use that effective dimension instead.
2. Fix the evaluation horizon $h$. For a 32-sample exploration budget, use $h=31$ for final-step calibration and optionally inspect earlier steps.
3. Choose an RBF-density target $C$, such as $0.1$, $0.5$, or $1.0$, and compute the corresponding $\sigma$ analytically.
4. Compute empirical $D_h$ quantiles from stored or simulated iid whitened samples for the candidate $\sigma$ values.
5. Choose $\beta$ by targeting median or upper-quantile acceptance, then verify expected rejection attempts, fallback probability, and the fraction of proposals with acceptance below thresholds such as 0.95 and 0.8.
6. Only after this CPU pass, run a small decoding pilot and evaluate external strategy coverage as in Exploration efficiency coverage metrics.

For the z64 Qwen2.5 sweep, the CPU-only calibration artifacts are in path:experiments/synthetic_sequences/disentangled/analysis/aggregates/synthetic-pretrained-exploration-efficiency-from-entangled-v2-z64-qwen25-global-token-kernel-calibration_20260604, with the repo-side report at path:experiments/synthetic_sequences/disentangled/analysis/aggregates/synthetic-pretrained-exploration-efficiency-from-entangled-v2-z64-qwen25-global-token-kernel-calibration_20260604/report.md.

Recommended Pilot Grid For z64

The calibration sweep suggests this paired pilot grid before any broad GPU evaluation. The first six settings target moderate RBF activity. The final six settings target larger hard-ball affected mass; their $\beta$ values are much smaller because the implemented sampler still uses RBF density, which is already large at these radii.

This grid is not a claim that these settings improve task-local strategy coverage. It is a calibrated way to test the intended mechanism: unlike the original grid, these settings should actually change the proposal distribution in 64D whitened space while keeping rejection behavior measurable.

Caveats

The RBF calibration only ensures that the sampler is active in whitened Euclidean geometry. It does not ensure that whitened Euclidean distance is aligned with strategy identity. The failed sweep also showed that same-strategy and different-strategy sample pairs have very similar full-space distances, so follow-up work should also test angular repulsion, lower-dimensional strategy-relevant subspaces, or candidate reranking methods if calibrated RBF repulsion still fails to improve coverage.