Literature: RLVR in LLMs
Literature: RLVR in LLMs
Papers on reinforcement learning with verifiable rewards (RLVR) for language models, with particular attention to the limits of passive on-policy exploration. The failure of passive on-policy exploration to discover new reasoning strategies is one of the motivations of this project.
Scope includes:
- empirical studies of what RLVR does and does not teach relative to the base model (e.g. "Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?");
- exploration methods for RLVR, including representation-based and off-policy approaches (e.g. "Representation-Based Exploration for Language Models: From Test-Time to Post-Training");
- related analyses of RLVR training dynamics, reward shaping, and strategy diversity collapse.
Papers
Capability Boundaries, Support, And Sharpening
- Yue et al. (2025) - Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- Wu et al. (2025) - The Invisible Leash? Why RLVR May or May Not Escape Its Origin
- Huang et al. (2024) - Self-Improvement in Language Models: The Sharpening Mechanism
- Chen et al. (2025) - The Coverage Principle: How Pre-Training Enables Post-Training
- Zhang et al. (2025) - On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
These papers are the clearest place to start if the question is whether RLVR really expands reasoning support, or mostly reweights what the base model already knows. Taken together, they suggest a shared distinction between support coverage, sampling efficiency, and genuine capability expansion.
Exploration And Curriculum Methods
- Tuyls et al. (2025) - Representation-Based Exploration for Language Models: From Test-Time to Post-Training
- Qu et al. (2026) - POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
- Sundaram et al. (2026) - Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
- Hu et al. (2025) - BRORL: Scaling Reinforcement Learning via Broadened Exploration
These papers all attack the "hard problems get zero useful signal" bottleneck, but in different ways: representation-space novelty bonuses, oracle-guided exploration, self-generated stepping-stone curricula, or much broader rollout search.
Credit Assignment, Group Weighting, And Training Efficiency
- Hubotter et al. (2026) - Reinforcement Learning via Self-Distillation
- Wei and Huang (2026) - MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting
- Yang et al. (2026) - Your Group-Relative Advantage Is Biased
These papers are useful for separating "better optimization" from "broader discovery." They often improve learning dynamics, but they do not by themselves settle whether the model is accessing new reasoning strategies or just exploiting already-sampled ones more effectively.
Broader Failure Taxonomy
This survey is not RLVR-specific, but it is a good reference point for situating exploration failure, support collapse, robustness gaps, and abstraction failures inside the larger literature on LLM reasoning breakdowns.
