Reasoning

Reading list and comments/short reviews around reasoning models, training techniques, and related papers

TODO

• build tag system e.g. Policies {DPO, GRPO, ...} with a view to building reading list, integrate with Zotero/Obsidian
• use new approach for formatting table and generating summary: test time and cost first

---

Papers

Priority

• s1: Simple test-time scaling
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
• Competitive Programming with Large Reasoning Models - OpenAI, "Our findings show that although specialized pipelines such as o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model surpasses those results without relying on hand-crafted inference heuristics. Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces rating on par with elite human competitors. Overall, these results indicate that scaling general-purpose reinforcement learning, rather than relying on domain-specific techniques, offers a robust path toward state-of-the-art AI in reasoning domains, such as competitive programming."
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
• CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction - DeepSeek
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale - ByteDance/Tsinghua
• VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks - ByteDance
• GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning - Alibaba, simplifies relative to GRPO, fewer models needed during training also
• Self-Steering Language Models - program synthesis related
• Skywork-OR1 (Open Reasoner 1) - read the nice Notion technical blog via this Github
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs - ByteDance, seems to be like a Code Action training but need to read in detail
• ToRL: Scaling Tool-Integrated RL - TODO: create a Tool Use RL general topic
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? - Tsinghua/SJTU, critical analysis of RLVR reinforcement learning with verifiable rewards
• Generative AI Act II: Test Time Scaling Drives Cognition Engineering - 76 pages, really nice overview of all the related topics, has code and mini tutorials also
• TTRL: Test-Time Reinforcement Learning
• Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation - read this with the Tsinghua/SJTU paper above; seems using a strong base model and some DPO works to boost reasoning performance
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example
• WebThinker: Empowering Large Reasoning Models with Deep Research Capability - nice Deep Research implementation, has Github https://github.com/RUC-NLPIR/WebThinker
• 100 Days After DeepSeek-R1: A on Replication Studies and More Directions for Reasoning Language Models - nice review
• RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning - reasoning + agents, has Github and code etc.
• Absolute Zero: Reinforced Self-play Reasoning with Zero Data - has a cool project page also https://andrewzh112.github.io/absolute-zero-reasoner/
• X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains - Microsoft Research, they train entirely on text and find that reasoning transfers to multimodal (image) performance
• Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers - jointly train reasoner and verifier
• Reinforcement Learning: A Comprehensive Overview - entire book by Kevin P Murphy (probabilistic machine learning author); not entirely about reasoning models but has nice sections on RL+LLMs and related subjects
• Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space - seems to be training-free, need to re-read to be clear
• lmgame-Bench: How Good are LLMs at Playing Games? - not entirely about reasoning, but all the best performing models are reasoning models (o3 etc.) and I really like the paper, especially the correlation studies with tasks in Table/Figure 3, and the general area
• DisCO: Reinforcing Large Reasoning Models with Discriminative Constrained Optimization - looks like interesting approach/alternative to GRPO, need to implement and test
• Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles - cool, training with logic puzzles, shows generalization to coding and mathematics; has a benchmark also
• DeepTheorem: Advancing LLM Reasoning for Theorem Proving Through Natural Language and Reinforcement Learning - Tencent, nice approach and methods
• Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for Frozen LLMs - looks good, approach works with frozen models but I didn't understand training on first reading so need to restudy paper
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models - Nvidia/Nemotron team, model is here https://huggingface.co/nvidia/Nemotron-Research-Reasoning-Qwen-1.5B
• REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards - nice Python library to check out, Github is here https://github.com/open-thought/reasoning-gym
• Spurious Rewards: Rethinking Training Signals in RLVR - blog for their paper; "We show that you can do RLVR on Qwen2.5-Math models with completely random or incorrect rewards, and still get massive math benchmark gains."
• OpenThoughts: Data Recipes for Reasoning Models - cool project and organization, they have a project page https://www.openthoughts.ai/
• Play to Generalize: Learning to Reason Through Game Play - nice, they find that just by training on video games you get out of domain performance improvements (i.e. on mathematics); it would be cool to see more of the results about "game feature vs. out of domain improvement" (at the end of the paper - basically it seems that if you train on games that have a "spatial" component, the mathematics improves on "geometry" tasks for example)
• Reinforcement Pre-Training - from the abstract: "Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers."
• CoRT: Code-integrated Reasoning within Thinking - USTC/Qwen team
• Resa: Transparent Reasoning Models via SAEs
• ComfyUI-R1: Exploring Reasoning Models for Workflow Generation - Alibaba; really cool
• A taxonomy for next-generation reasoning models - Where we've been and where we're going with RLVR - Nathan Lambert blog; really good overview as of June 2025
• Scaling Speculative Decoding with Lookahead Reasoning - Hao lab UCSD; technical, need to re-read details carefully, code is available. " Our key insight is that reasoning models generate step-by-step, and each step needs only to be semantically correct, not exact token matching. In Lookahead Reasoning, a lightweight draft model proposes several future steps; the target model expands each proposal in one batched pass, and a verifier keeps semantically correct steps while letting the target regenerate any that fail. Token-level SD still operates within each reasoning step, so the two layers (i.e. standard speculative decoding) of parallelism multiply."
• Thought Anchors: Which LLM Reasoning Steps Matter? - Interesting "interpretability" paper; Neel Nanda coauthor. Links to a nice interactive tool also https://www.thought-anchors.com/
• WebSailor: Navigating Super-human Reasoning for Web Agent - awesome, Tongyi Lab Alibaba; "Proprietary agentic systems like DeepResearch have demonstrated superhuman capabilities on extremely complex information-seeking benchmarks such as BrowseComp, a feat previously unattainable. We posit that their success hinges on a sophisticated reasoning pattern absent in open-source models: the ability to systematically reduce extreme uncertainty when navigating vast information landscapes. Based on this insight, we introduce WebSailor, a complete post-training methodology designed to instill this crucial capability. Our approach involves generating novel, high-uncertainty tasks through structured sampling and information obfuscation, RFT cold start, and an efficient agentic RL training algorithm, Duplicating Sampling Policy Optimization (DUPO)."
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities - Gemini 2.5 technical report
• Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination - nice study attemping to clear up some of the confusing results from April-June where people were training models on incorrect or low quality reasoning datasets etc.
• Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs - great review
• Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training - Nvidia, nice technical report on Nemotron-Research-Reasoning-Qwen-1.5B, they talk about improvements to GRPO (basically seems they validate using DAPO and some implementation changes etc.)
• Group Sequence Policy Optimization - Qwen team, describes Qwen3 training also
• Test-Time Scaling with Reflective Generative Model - looks good but I didn't understand all the implementation details yet; "MetaStone‑S1 is trained based on our proposed reflective generative form, which combines “Long-CoT Reinforcement Learning” and “Process Reward Learning” into a unified training form. This form enables a single model to simultaneously achieve deep reasoning and high-quality reasoning trajectory selection. By sharing the backbone network between the PRMs and policy models, MetaStone‑S1 significantly reduces the inference cost of PRMs by 99%, resulting in faster and higher-quality responses."; Github is https://github.com/MetaStone-AI/MetaStone-S1
• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
• Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty - MIT; RLCR (reinforcement learning with calibration rewards), they add calibration term and help model verbalize its confidence (claims it generalizes to out of domain tasks also, need to reread that part)
• Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens - critical review
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models - the technical report for GLM-4.5 models by Z.ai
• MolmoAct: Action Reasoning Models that can Reason in Space - Allen Institute AI2; really cool and they open source everything (I can't find the training code yet though, maybe ask authors) - there is a blog here also that gives a nice overview https://allenai.org/blog/molmoact
• Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning - Alibaba, nice review; "This paper systematically reviews widely adopted RL techniques through rigorous reproductions and isolated evaluations within a unified open-source framework. We analyze the internal mechanisms, applicable scenarios, and core principles of each technique through fine-grained experiments, including datasets of varying difficulty, model sizes, and architectures. Based on these insights, we present clear guidelines for selecting RL techniques tailored to specific setups, and provide a reliable roadmap for practitioners navigating the RL for the LLM domain."
• Agentic Reinforced Policy Optimization - awesome, multi-turn reasoning with tool use etc., lots of interesting sections
• Intern-S1: A Scientific Multimodal Foundation Model
• rStar2-Agent: Agentic Reasoning Technical Report - Microsoft Research; really cool and has Github (it's the updated version of rStar) https://github.com/microsoft/rStar
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
• Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers - looks like a nice approach; Github is here https://github.com/camel-ai/loong
• The Landscape of Agentic Reinforcement Learning for LLMs: A Survey - awesome review
• SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning - has great Notion blog also https://simpletir.notion.site/report
• A Survey of Reinforcement Learning for Large Reasoning Models - awesome 100+ pages, good overview of all the major topics
• The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs - basically seems to be optimistic about continued scaling of LLMs (test-time/reasoning in general)
• Inpainting-Guided Policy Optimization for Diffusion Large Language Models - I found this useful for understanding some of the advantages of diffusion LLMs for reasoning tasks and technical issues that it can fix
• Single-stream Policy Optimization - Tencent; found the approach quite convincing and talking about some of the limitations of group-based methods (like GRPO) for e.g. training reasoning models. They have a blog also https://zhongwenxu.notion.site/Single-stream-Policy-Optimization-26a1c4e140e380d78d51fa4567727f50
• FlowRL: Matching Reward Distributions for LLM Reasoning - really cool; "Recent advanced reasoning models adopt reward-maximizing methods (e.g. PPO and GRPO), which tend to over-optimize dominant reward signals while neglecting less frequent but valid reasoning paths, thus reducing diversity. In contrast, we transform scalar rewards into a normalized target distribution using a learnable partition function, and then minimize the reverse KL divergence between the policy and the target distribution."
• FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questions - a nice and detailed evaluation paper, as of September 2025, with some interesting comments about performance (hallucination of tools etc.); good for quickly seeing and getting a feel for where we are presently
• Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards - really interesting; need to read all the appendix stuff to understand why/if(?) this approach is specific to mathematical reasoning, as I didn't understand this aspect on first reading
• RLP: Reinforcement as a Pretraining Objective
• RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
• Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail At It - U Washington, Allen Institute; cool area - personalization of reasoning
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• LaSeR: Reinforcement Learning with Last-Token Self-Rewarding - Renmin, Tencent; nice - "...we propose LaSeR (Reinforcement Learning with Last-Token Self-Rewarding), an algorithm that simply augments the original RLVR loss with a MSE loss that aligns the last-token self-rewarding scores with verifier-based reasoning rewards, jointly optimizing the reasoning and self-rewarding capabilities of LLMs. The optimized self-rewarding scores can be utilized in both training and testing to enhance model performance."
• Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AI - nice review; changing approach to reasoning/tool use etc. They have a nice Github here https://github.com/ADaM-BJTU/model-native-agentic-ai
• Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model - Inclusion AI; very cool technical report with nice details
• Scaling Latent Reasoning via Looped Language Models - ByteDance, MILA, lots of other institutions also
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning - " Towards self-verifiable mathematical reasoning, we investigate how to train an accurate and faithful LLM-based verifier for theorem proving. We then train a proof generator using the verifier as the reward model, and incentivize the generator to identify and resolve as many issues as possible in their own proofs before finalizing them. To maintain the generation-verification gap as the generator becomes stronger, we propose to scale verification compute to automatically label new hard-to-verify proofs, creating training data to further improve the verifier. Our resulting model, DeepSeekMath-V2, demonstrates strong theorem-proving capabilities, achieving gold-level scores on IMO 2025 and CMO 2024 and a near-perfect 118/120 on Putnam 2024 with scaled test-time compute."
• Step-Audio-R1 Technical Report - Cool area, audio reasoning
• DynaAct: Large Language Model Reasoning with Dynamic Action Spaces - Nice ideas IMHO; NeurIPS 2025
• RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments - Recent from AI2/University of Washington, seems code is public (check the RLVE-Gym).
• Every Activation Boosted: Scaling General Reasoner to 1 Trillion Open Language Foundation - Ling 2.0 Technical Report; nice details
• DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models - nice study
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices - Qwen team; really nice experimental study etc.
• QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management - Qwen team; nice technical report
• JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
• INTELLECT-3: Technical Report
• Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning
• A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models - TODO: create separate section on reward models
• PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning - StepFun, Tsinghua, Peking
• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models - DeepSeek; "Most notably, while the memory module is expected to aid knowledge retrieval (e.g., MMLU +3.4; CMMLU +4.0), we observe even larger gains in general reasoning (e.g., BBH +5.0; ARC-Challenge +3.7) and code/math domains~(HumanEval +3.0; MATH +2.4). Mechanistic analyses reveal that Engram relieves the backbone's early layers from static reconstruction, effectively deepening the network for complex reasoning.", Github page is https://github.com/deepseek-ai/Engram
• RoboBrain 2.5: Depth in Sight, Time in Mind - BAAI; really nice reasoning robotics - project page is https://superrobobrain.github.io/
• Agentic Reasoning for Large Language Models - really nice review
• Innovator-VL: A Multimodal Large Language Model for Scientific Discovery - project page is https://innovatorlm.github.io/Innovator-VL/
• Kimi K2.5: Visual Agentic Intelligence - Kimi K2.5 technical report; figures 11 and 12 in appendix on the visual reasoning and understanding 40 hour game playthrough are cool
• Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters - StepFun, technical report
• Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL
• Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments - Meta; GAIA 2 benchmark
• CL-bench: A Benchmark for Context Learning - "... we introduce CL-bench, a real-world benchmark consisting of 500 complex contexts, 1,899 tasks, and 31,607 verification rubrics, all crafted by experienced domain experts. Each task is designed such that the new content required to resolve it is contained within the corresponding context. Resolving tasks in CL-bench requires models to learn from the context, ranging from new domain-specific knowledge, rule systems, and complex procedures to laws derived from empirical data, all of which are absent from pre-training."
• A Very Big Video Reasoning Suite - nice resource; also cool project page https://video-reason.com/
• daVinci-Env: Open SWE Environment Synthesis at Scale - not directly reasoning related but really nice infrastructure/organization for generating environments
• Demystifing Video Reasoning - "Reasoning in video generation models happens along diffusion steps, not frames. We discover Chain-of-Steps - a new understanding of how video models think."; cool and nice project page also https://www.wruisi.com/demystifying_video_reasoning/
• MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verification
• LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning - "We introduce LongCat-Flash-Prover, a flagship 560-billion-parameter open-source Mixture-of- Experts (MoE) model that advances Native Formal Reasoning in Lean4 through agentic tool-integrated reasoning (TIR)."
• Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale
• The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook - nice general review with many relevant sections
• Solving Olympiad via Reinforcement Learning on Simulators - nice project page also https://sim2reason.github.io/
• Recursive Multi-Agent Systems - "Scaling Agent Collaboration through Latent-space Recursion" nice project page also https://recursivemas.github.io/
• GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents - tech report, lots of multimodal reasoning
• MolmoAct2: Action Reasoning Models for Real-world Deployment - AI2; project page with videos https://allenai.org/blog/molmoact2
• Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling - project page https://simplified-reasoning.github.io/SU-01/
• Generative Recursive Reasoning
• Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
• VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models - "Extending our previous 1.5B work, these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios. This perspective suggests that compact models are not merely deployment-efficient substitutes, but a complementary path toward frontier-level performance in parameter-dense capability regimes."
• Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning - Nvidia; nice technical report
• Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
• AREX: Towards a Recursively Self-Improving Agent for Deep Research
• Kimi K3: Open Frontier Intelligence - awesome
• GPT-Red: Automated Red Teaming via Self-Play at Scale - OpenAI technical report
• Intern-S2-Preview: Scientific Agentic Foundation Model
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
• DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression - DeepSeek technical report
• MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
• A team of Gemini 4 Argon agents beats Minecraft
• h1: A Taste of the Bitter Lesson in Automated Theorem Proving
• Axion 1.0: Open Agent Foundation Models, Simplified

Topic - General or unsorted

• Technical Report: Enhancing LLM Reasoning with Reward-guided Tree Search
• Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
• Improving Retrieval Augmented Language Model with Self-Reasoning
• Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models
• Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning
• Advancing LLM Reasoning Generalists with Preference Trees
• Test-time Computing: from System-1 Thinking to System-2 Thinking
• Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
• BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning
• Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
• Search-o1: Agentic Search-Enhanced Large Reasoning Models
• CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis
• REL: Working out is all you need
• Unconstrained Model Merging for Enhanced LLM Reasoning
• MALT: Improving Reasoning with Multi-Agent LLM Training
• Reverse Thinking Makes LLMs Stronger Reasoners
• Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning
• Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus
• Evolving Deeper LLM Thinking
• Reasoning Language Models: A Blueprint
• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning
• Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
• Generative Verifiers: Reward Modeling as Next-Token Prediction
• Generative Reward Models
• Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
• Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
• Table as Thought: Exploring Structured Thoughts in LLM Reasoning
• B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners
• Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
• GuardReasoner: Towards Reasoning-based LLM Safeguards
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
• QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Search
• LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information
• LLM+AL: Bridging Large Language Models and Action Languages for Complex Reasoning about Actions
• Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification
• ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
• Process Reinforcement through Implicit Rewards - see also their HuggingFace blog
• Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning
• BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation
• TypedThinker: Typed Thinking Improves Large Language Model Reasoning
• ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
• Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning
• CoT-Valve: Length-Compressible Chain-of-Thought Tuning
• LLM Pretraining with Continuous Concepts - follow-up to Coconut paper (Training Large Language Model to Reason in a Continuous Latent Space, see below)
• LIMO: Less is More for Reasoning
• Diverse Inference and Verification for Advanced Reasoning
• CRANE: Reasoning with constrained LLM generation
• AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence
• OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning
• Thinking Preference Optimization
• S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning
• Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
• Reasoning with Reinforced Functional Token Tuning
• SIFT: Grounding LLM Reasoning in Contexts via Stickers
• How to Get Your LLM to Generate Challenging Problems for Evaluation - maybe useful for making reasoning datasets
• Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
• General Reasoning Requires Learning to Reason from the Get-go
• CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
• START: Self-taught Reasoner with Tools
• Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
• An Empirical Study on Eliciting and Improving R1-like Reasoning Models
• Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
• ϕ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitation
• Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't
• Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
• Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs -TODO: make a "code enhances reasoning" tag
• CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance
• I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
• SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
• A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methods
• Planning In Natural Language Improves LLM Search For Code Generation
• ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
• MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search
• Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
• CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model
• Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains
• Z1: Efficient Test-time Scaling with Code
• Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? - critical of reasoning results, similar to SimpleBench
• Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL
• JudgeLRM: Large Reasoning Models as a Judge
• When More is Less: Understanding Chain-of-Thought Length in LLMs
• OpenCodeReasoning: Advancing Data Distillation for Competitive Coding - mainly a dataset but interesting paper also
• Bag of Tricks for Inference-time Computation of LLM Reasoning - looks nice, practical stuff
• GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
• Sample, Don't Search: Rethinking Test-Time Alignment for Language Models
• Rethinking Reflection in Pre-Training
• Generative Evaluation of Complex Reasoning in Large Language Models - synthetic benchmark/evaluation
• MastermindEval: A Simple But Scalable Reasoning Benchmark - benchmark
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility - critical of some benchmarks, evals and methodology
• Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation - ICLR 2025
• Seed-Thinking-v1.5: Advancing Superb Reasoning Models with Reinforcement Learning - ByteDance, paper is available via this Github
• TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning
• Complex LLM Planning via Automated Heuristics Discovery - code enhanced reasoning related, generates heuristics as Python functions
• Reasoning Models Can Be Effective Without Thinking - basically skip the thinking step in a thinking model
• RARE: Retrieval-Augmented Reasoning Modeling - no code/examples yet in preprint
• Efficient Reasoning Models: A Survey
• A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
• Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
• SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
• Language Models can Self-Improve at State-Value Estimation for Better Search
• SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks - not directly about a reasoning model but nice RL and task area, collaborating with reasoning system etc.
• Learning to Reason under Off-Policy Guidance
• FlowReasoner: Reinforcing Query-Level Meta-Agents
• ToolRL: Reward is All Tool Learning Needs - not directly reasoning model, but GRPO for tool use; general "tool use RL" area interesting
• OTC: Optimal Tool Calls via Reinforcement Learning - see above, "tool use RL"
• Learning Adaptive Parallel Reasoning with Language Models
• Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning
• Tina: Tiny Reasoning Models via LoRA
• Process Reward Models That Think
• Offline Reinforcement Learning for LLM Multi-Step Reasoning
• Zero-shot Robotic Manipulation with Language-guided Instruction and Formal Task Planning - not directly about reasoning models, it's symbolic reasoning for robotics and the "reasoning plan" general area; TODO: search for reasoning models in robotics and create Topic for it
• SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning
• Toward Evaluative Thinking: Meta Policy Optimization with Evolving Reward Models
• Phi-4-reasoning Technical Report - Microsoft
• DeepCritic: Deliberate Critique with Large Language Models
• reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs
• Toward Evaluative Thinking: Meta Policy Optimization with Evolving Reward Models - nice area, but code not available yet it seems
• Llama-Nemotron: Efficient Reasoning Models - Nvidia, goes into technical details and implementation
• Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL
• ZeroSearch: Incentivize the Search Capability of LLMs without Searching
• Scalable Chain of Thoughts via Elastic Reasoning
• LIMR: Less is More for RL Scaling
• The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models
• Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning
• MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining - Xiaomi, claims 7B performance >= to o1-mini
• Learning from Peers in Reasoning Models
• AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scale
• Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures - DeepSeek, not directly about reasoning but nice technical comments
• Kalman Filter Enhanced GRPO for Reinforcement Learning-Based Language Model Reasoning
• The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think
• ImagineBench: Evaluating Reinforcement Learning with Large Language Model Rollouts - nice approach, added since may be useful for reasoning models and RL
• How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study
• Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models
• AdaptThink: Reasoning Models Can Learn When to Think
• Optimizing Anytime Reasoning via Budget Relative Policy Optimization
• Reward Reasoning Model
• General-Reasoner: Advancing LLM Reasoning Across All Domains
• Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning
• Training Step-Level Reasoning Verifiers with Formal Verification Tools
• The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning - they claim a training-free approach that uses inference-time modifications to logits that boosts "reasoning" dataset performance, amongst other things
• QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient
• ARM: Adaptive Reasoning Model
• ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning
• SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond
• rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset - Microsoft, but code/dataset isn't available yet
• RL Tango: Reinforcing Generator and Verifier Together for Language Reasoning
• Skywork Open Reasoner 1 Technical Report
• Are Reasoning Models More Prone to Hallucination?
• LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling
• R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing - Tsinghua, cool test-time scaling approach using SLM and larger LLM
• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models - theoretical paper, need to reread carefully
• Pitfalls of Rule- and Model-based Verifiers -- A Case Study on Mathematical Reasoning - they study the pros and cons of different verifiers when doing RL; the focus in on mathematical dataset/examples but it seems broadly applicable to RLVR
• System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts - CoCoNut paper related, see also review paper Reasoning Beyond Language linked below
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
• AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
• Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models
• R1-Code-Interpreter: Training LLMs to Reason with Code via Supervised and Reinforcement Learning
• RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling
• Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation
• MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
• AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy - Nvidia; nice improvements since 1.0 model (still 7B based on Qwen 2.5)
• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
• ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs - ByteDance; "... we propose ProtoReasoning, a framework that enhances the reasoning ability of LLMs by leveraging scalable and verifiable prototypical representations (Prolog for logical reasoning, PDDL for planning) ... Significantly, our ablation studies confirm that learning in prototype space also demonstrates enhanced generalization to structurally similar problems compared to training solely on natural language representations, validating our hypothesis that reasoning prototypes serve as the foundation for generalizable reasoning in large language models."
• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective - links to nice Github pages also https://guru-reasoning.github.io/ and https://github.com/LLM360/Reasoning360
• Reward-SQL: Boosting Text-to-SQL via Stepwise Reasoning and Process-Supervised Rewards
• ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs
• Robust Reward Modeling via Causal Rubrics - Google DeepMind; "We introduce Crome (Causally Robust Reward Modeling), a novel framework grounded in an explicit causal model designed to mitigate reward hacking.", has some results on reasoning benchmarks that seem good
• OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
• Reinforcement Learning Teachers of Test Time Scaling - Sakana AI; Github https://github.com/SakanaAI/RLT and blog https://sakana.ai/rlt/
• Fractional Reasoning via Latent Steering Vectors Improves Inference Time Compute
• SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning
• Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning - "Math reasoning has become the poster child of progress in large language models (LLMs), with new models rapidly surpassing human-level performance on benchmarks like MATH and AIME. But as math leaderboards improve week by week, it is worth asking: do these gains reflect broader problem-solving ability or just narrow overfitting? To answer this question, we evaluate over 20 open-weight reasoning-tuned models across a broad suite of tasks, including math, scientific QA, agent planning, coding, and standard instruction-following. We surprisingly find that most models that succeed in math fail to transfer their gains to other domains."
• Fast and Simplex: 2-Simplicial Attention in Triton - Meta; this is mainly an architecture/technical paper, but the claims are related specifically to improving performance on reasoning tasks - not clear to me yet, need to take notes carefully
• Pre-Trained Policy Discriminators are General Reward Models - not strictly about reasoning models but applicable and relevant
• R1-RE: Cross-Domain Relationship Extraction with RLVR - nice, classic NLP task; claim 7B model reaches 4o performance
• Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation - Microsoft, this is the technical paper for the Phi 4 mini flash reasoning model (3.8B) with some nice architecture ideas. The blogpost is also here: Reasoning reimagined: Introducing Phi-4-mini-flash-reasoning
• First Return, Entropy-Eliciting Explore - ByteDance
• Test-Time Scaling with Reflective Generative Model
• REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once
• Logit Arithmetic Elicits Long Reasoning Capabilities Without Training - reminds me a bit of The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning (see link above)
• Beyond Context Limits: Subconscious Threads for Long-Horizon Reasoning
• MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning - has nice explanation of data filtering and processing also
• Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning
• LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization
• Re:Form -- Reducing Human Priors in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny - formal methods
• KV Cache Steering for Inducing Reasoning in Small Language Models - it is a one-shot method, seems cool: " Our approach leverages GPT-4o-generated reasoning traces to construct steering vectors that shift model behavior toward more explicit, multi-step reasoning without fine-tuning or prompt modifications."; examples on the Github https://github.com/MaxBelitsky/cache-steering
• UloRL:An Ultra-Long Output Reinforcement Learning Approach for Advancing Large Language Models' Reasoning Abilities - Tencent, nice results/performance; code is available
• Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Following
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
• R-Zero: Self-Evolving Reasoning LLM from Zero Data
• Learning to Reason for Factuality - FAIR/Meta
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks
• Train Long, Think Short: Curriculum Learning for Efficient Reasoning
• Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models
• SSRL: Self-Search Reinforcement Learning - cool, haven't read references but can't remember seeing this idea before; basically using the LLM as a web simulator for search/tool use during reasoning etc.
• NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model - "We introduce Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer language model designed to increase throughput for reasoning workloads while achieving state-of-the-art accuracy compared to similarly-sized models. Nemotron-Nano-9B-v2 builds on the Nemotron-H architecture, in which the majority of the self-attention layers in the common Transformer architecture are replaced with Mamba-2 layers, to achieve improved inference speed when generating the long thinking traces needed for reasoning."
• Deep Think with Confidence
• Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
• TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling - ByteDance
• ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models
• StepWiser: Stepwise Generative Judges for Wiser Reasoning - FAIR Meta
• Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models
• Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training
• InternBootcamp Technical Report: Boosting LLM Reasoning with Verifiable Task Scaling
• Bootstrapping Task Spaces for Self-Improvement - Meta Superintelligence; some nice results across multi-turn tool use, machine learning (MLE benchmark), and mathematics
• Reverse-Engineered Reasoning for Open-Ended Generation - work backwards from "good" samples to try to produce the reasoning; they use on e.g. artistic domain (writing for ex.) where "forward" approach to reasoning might not work as well
• Scaling up Multi-Turn Off-Policy RL and Multi-Agent Tree Search for LLM Step-Provers - I like the general RL approach; indeed the abstract says "While demonstrated in the domain of formal mathematics, the RL and inference techniques presented in this work are of broader interest and may be applied to other domains requiring long-horizon multi-turn reasoning and complex search"
• Parallel-R1: Towards Parallel Thinking via Reinforcement Learning
• The Majority is not always right: RL training for solution aggregation - FAIR Meta; "In this work, we propose to learn aggregation as an explicit reasoning skill: given a set of candidate solutions, we train an aggregator model to review, reconcile, and synthesize a final, correct answer using reinforcement learning from verifiable rewards." - this part is interesting: "Furthermore, it generalizes effectively to solutions from differing models, including stronger ones than contained in the training data, all while requiring substantially fewer tokens than majority voting with larger numbers of solutions."
• On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification - not a reasoning-specific article but has lots of relevant comments and results
• THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning - code results also, not just mathematical reasoning
• Compute as Teacher: Turning Inference Compute Into Reference-Free Supervision - a different approach to using test time computation
• What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
• SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines - seems to be mainly biology focused (SMILES, protein properties etc.)
• Quantile Advantage Estimation for Entropy-Safe Reasoning
• Variational Reasoning for Language Models - actually OK to follow despite all the theory; I hadn't heard of "importance-weighted autoencoder (IWAE)" before either (page 2) and it's interesting
• Beyond the Exploration-Exploitation Trade-off: A Hidden State Approach for LLM Reasoning in RLVR
• Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Training - nice mechanistic/explainability; "In this work, we use circuit analysis to demonstrate that post-training for complex reasoning sparks the emergence of novel, functionally specialized attention heads."
• Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation - ByteDance Seed; TODO - create a section for "intelligent/informed sample selection" and related approaches to improving reasoning model training
• DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
• ExGRPO: Learning to Reason from Experience
• Large Reasoning Models Learn Better Alignment from Flawed Thinking - Meta Superintelligence Labs; basically seems to be including a few "flawed" (wrong chain of thoughts for ex) samples and promote self-reflection to correct these "bad starts" during completion
• Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data - Nvidia; "We conduct the first systematic study of how reasoning data-varying in scale, diversity, and quality-affects LLM performance when introduced at different stages of training. We find that front-loading reasoning data into pretraining is critical (19% avg gain), establishing foundational capabilities that cannot be fully replicated by later-stage SFT, even with more data."
• In-the-Flow Agentic System Optimization for Effective Planning and Tool Use - website is https://agentflow.stanford.edu/
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning - reasoning dataset generation with automatic validation steps
• Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward
• ACADREASON: Exploring the Limits of Reasoning Models with Academic Research Problems - nice benchmark; "50 expert-annotated academic problems across five high-reasoning domains, including computer science, economics, law, mathematics, and philosophy. All questions are sourced from top-tier publications in recent years and undergo rigorous annotation and quality control to ensure they are both challenging and answerable."
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails - nice "red teaming"/security kind of approach
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning
• LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts
• Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning - Ling team, Ant Group; nice technical paper
• Open-o3 Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence - ByteDance
• Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
• Reasoning with Sampling: Your Base Model is Smarter Than You Think
• DynaAct: Large Language Model Reasoning with Dynamic Action Spaces - NeurIPS 2025
• Critique-RL: Training Language Models for Critiquing through Two-Stage Reinforcement Learning - interesting for developing critique models for reasoning training
• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads
• SPICE: Self-Play In Corpus Environments Improves Reasoning
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists
• Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models
• ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
• Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs - Nvidia; interesting twist on efficient reasoning/model pruning, uses Mamba architectures also.
• P1: Mastering Olympiads with Reinforcement Learning - "... Further equipped with an agentic framework PhysicsMinions, P1-235B-A22B+PhysicsMinions achieves overall No.1 on IPhO 2025, and obtains the highest average score over the 13 physics competitions."
• Real-Time Reasoning Agents in Evolving Environments - Cool study, introducing urgency/time constraints in reasoning tasks
• VeriCoT: Neuro-symbolic Chain-of-Thought Validation via Logical Consistency Checks - neurosymbolic approach to validate reasoning, outside of mathematics/code domain; also TODO make a section for neurosymbolic/DSL stuff
• Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence - intriguing; "In this work, we study how to convert existing pretrained non-recurrent language models into depth-recurrent models. We find that using a curriculum of recurrences to increase the effective depth of the model over the course of training preserves performance while reducing total computational cost. In our experiments, on mathematics, we observe that converting pretrained models to recurrent ones results in better performance at a given compute budget than simply post-training the original non-recurrent language model."
• RLoop: An Self-Improving Framework for Reinforcement Learning with Iterative Policy Initialization
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought - sketchpad type approaches; seen several papers on this before but seems to work OK here?
• Terra Nova: A Comprehensive Challenge Environment for Intelligent Agents - not strictly reasoning, but a nice environment for RL; it's a Civilization V kind of game and you need to reason about strategies also
• Towards Robust Mathematical Reasoning - Google DeepMind; IMO-Bench "... played a crucial role in our historic achievement of the gold-level performance at IMO 2025 with Gemini Deep Think"
• PretrainZero: Reinforcement Active Pretraining
• SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL
• SPARK: Stepwise Process-Aware Rewards for Reference-Free Reinforcement Learning - process reward models
• Long-horizon Reasoning Agent for Olympiad-Level Mathematical Problem Solving
• OPV: Outcome-based Process Verifier for Efficient Long Chain-of-Thought Verification
• V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions
• MMGR: Multi-Modal Generative Reasoning
• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
• TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning
• Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning
• NVIDIA Nemotron 3: Efficient and Open Intelligence
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
• Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewards
• Meta-RL Induces Exploration in Language Agents
• Scaling Open-Ended Reasoning to Predict the Future
• Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space
• From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR
• MarsRL: Advancing Multi-Agent Reasoning System via Reinforcement Learning with Agentic Pipeline Parallelism - Tencent
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models
• Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models - Tencent Youtu Lab; nice performance on reasoning tasks for small model
• Recursive Language Models - not directly reasoning related but they test the approach on several long-context/reasoning datasets and tasks
• Atlas: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning
• GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors
• VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory - cool, nice project page also https://wsakobe.github.io/VLingNav-web/
• Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning
• Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs
• MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching
• PRL: Process Reward Learning Improves LLMs' Reasoning Ability and Broadens the Reasoning Boundary
• A^3-Bench: Benchmarking Memory-Driven Scientific Reasoning via Anchor and Attractor Activation
• Your Group-Relative Advantage Is Biased
• SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature - nice approach to formulating tasks that require reasoning
• Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs
• Reasoning Models Generate Societies of Thought - has some nice things on mechanistic interpretability of reasoning traces also
• rSIM: Incentivizing Reasoning Capabilities of LLMs via Reinforced Strategy Injection - "... this paper proposes a novel reinforced strategy injection mechanism (rSIM), that enables any LLM to become an RLM by employing a small planner to guide the LLM's CoT through the adaptive injection of reasoning strategies. To achieve this, the planner (leader agent) is jointly trained with an LLM (follower agent) using multi-agent RL (MARL), based on a leader-follower framework and straightforward rule-based rewards."
• MemoBrain: Executive Memory as an Agentic Brain for Reasoning
• Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
• DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints - Qwen team; "...we introduce DeepPlanning, a challenging benchmark for practical long-horizon agent planning. It features multi-day travel planning and multi-product shopping tasks that require proactive information acquisition, local constrained reasoning, and global constrained optimization. Evaluations on DeepPlanning show that even frontier agentic LLMs struggle with these problems, highlighting the importance of reliable explicit reasoning patterns and parallel tool use for achieving better effectiveness-efficiency trade-offs. Error analysis further points to promising directions for improving agentic LLMs over long planning horizons. We open-source the code and data to support future research."
• AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning
• Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation
• Reinforcement Learning via Self-Distillation
• Exploring Reasoning Reward Model for Agents
• Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning
• Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
• ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas
• Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learning
• Rethinking the Trust Region in LLM Reinforcement Learning
• Likelihood-Based Reward Designs for General LLM Reasoning
• Reinforced Attention Learning
• ProAct: Agentic Lookahead in Interactive Environments - "... we introduce Grounded LookAhead Distillation (GLAD), where the agent undergoes supervised fine-tuning on trajectories derived from environment-based search. By compressing complex search trees into concise, causal reasoning chains, the agent learns the logic of foresight without the computational overhead of inference-time search. ..."
• LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning
• P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads - cool project page, they have some nice visualizations https://prime-rl.github.io/P1-VL/
• When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning
• Internalizing Meta-Experience into Memory for Guided Reinforcement Learning in Large Language Models
• Effective Reasoning Chains Reduce Intrinsic Dimensionality
• ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
• GeoAgent: Learning to Geolocate Everywhere with Reinforced Geographic Characteristics
• InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problem
• Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality - thinking/inference time computation to enhance recall of facts
• Language-based Trial and Error Falls Behind in the Era of Experience
• Does Your Reasoning Model Implicitly Know When to Stop Thinking?
• From Perception to Action: An Interactive Benchmark for Vision Reasoning
• AI Gamestore: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Games - nice project page also https://aigamestore.org/
• POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
• LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding - CVPR 2026
• PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference
• V_1: Unifying Generation and Self-Verification for Parallel Reasoners
• BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
• Interactive Benchmarks
• Learn Hard Problems During RL with Reference Guided Fine-tuning
• CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation - ByteDance Seed
• Mario: Multimodal Graph Reasoning with Large Language Models - CVPR 2026
• Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
• Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
• GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing - project page is https://grade-bench.github.io/
• Automatic Generation of High-Performance RL Environments
• Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
• TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning
• MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learning
• CCTU: A Benchmark for Tool Use under Complex Constraints
• TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation - CVPR 2026, nice project page https://shuyansy.github.io/terrascope/
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model
• Can LLM Agents Be CFOs? A Benchmark for Resource Allocation in Dynamic Enterprise Environments - interesting long horizon reasoning task
• Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models
• Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe
• Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
• Make Geometry Matter for Spatial Reasoning
• FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization
• MuSEAgent: A Multimodal Reasoning Agent with Stateful Experience
• LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model
• LightThinker++: From Reasoning Compression to Memory Management
• Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
• Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability
• PokeGym: A Visually-Driven Long-Horizon Benchmark for Vision-Language Models
• SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning
• Process Reward Agents for Steering Knowledge-Intensive Reasoning
• Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning - NVIDIA; nice technical report
• SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments
• LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
• PersonaVLM: Long-Term Personalized Multimodal LLMs - multi-turn reasoning for retrieval/relevance of memories in VLM context
• EvoMaster: A Foundational Evolving Agent Framework for Agentic Science at Scale
• Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
• Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks
• Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixes
• PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research
• Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis
• Large Language Models Explore by Latent Distilling
• Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extraction
• HeavySkill: Heavy Thinking as the Inner Skill in Agentic Harness
• RemoteZero: Geospatial Reasoning with Zero Human Annotations - cool kind of self-supervised approaches to geospatial reasoning
• OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents
• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling
• Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
• Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMs
• MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoning - "MAP consists of three stages: (1) Global Exploration, acquiring environment-general priors; (2) Task-Specific Mapping, constructing a structured cognitive map; and (3) Knowledge-Augmented Execution, solving tasks grounded on the map. Experiments show consistent gains across benchmarks and LLMs. On ARC-AGI-3, MAP enables frontier models to surpass near-zero baseline performance in 22 of 25 game environments."
• NanoResearch: Co-Evolving Skills, Memory, and Policy for Personalized Research Automation
• Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR
• CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning
• From Reasoning Chains to Verifiable Subproblems: Curriculum Reinforcement Learning Enables Credit Assignment for LLM Reasoning
• ACC: Compiling Agent Trajectories for Long-Context Training
• SkillOpt: Executive Strategy for Self-Evolving Agent Skills
• Language Models Need Sleep - "Transformer-based large language models are increasingly used for long-horizon tasks; however, their attention mechanism scales poorly with context length. To handle this, we study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights before clearing its key-value cache."
• Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement - routing related maybe; "In this paper, we propose AKBE (Agentic Knowledge Boundary Enhancement), an on-policy method that dynamically probes the model's intrinsic knowledge boundary through dual-path (with-tool and no-tool) rollouts during training. We define the knowledge boundary as the per-instance determination of whether tools are required and the minimum tool calls necessary. By comparing correctness across paths, AKBE categorizes trajectories and constructs targeted supervisory signals that guide efficient tool-use patterns for each question."
• DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes
• QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks
• LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards
• Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses
• OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
• Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning
• AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints
• Latent Reasoning with Normalizing Flows
• SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
• Reinforcement Learning from Rich Feedback with Distributional DAgger
• On the Geometry of On-Policy Distillation
• Rethinking the Divergence Regularization in LLM RL
• Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization
• Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
• MaxProof: Scaling Mathematical Proof with Generative-Verifier RL and Population-Level Test-Time Scaling
• Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
• APPO: Agentic Procedural Policy Optimization
• SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks
• Context-Aware RL for Agentic and Multimodal LLMs
• Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models
• Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark - physics reasoning still difficult June 2026
• NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers? - "Evaluating ten frontier agent configurations under a strict web-search-disabled protocol, we find that the strongest model surpasses SOTA on only 17.8% of tasks under the g>0.1 criterion. Analysis of method pathways reveals that agents succeed primarily through methodological translation, converting scientific tasks into familiar supervised prediction problems, rather than through genuine scientific invention. Failures are dominated by wrong method choice and insufficient compute budget, not by task misunderstanding."
• Information-Aware KV Cache Compression for Long Reasoning
• OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning
• AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning
• CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economies - Sakana AI; nice business simulation and reasoning setup, they have project page also https://pub.sakana.ai/coffeebench/index.html
• Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent
• DOPD: Dual On-policy Distillation
• Graph-Native Reinforcement Learning Enables Traceable Scientific Hypothesis Generation through Conceptual Recombination
• EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
• When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search - "... we introduce DiscoBench, a benchmark for clarification-aware deep search, designed to evaluate whether search agents can proactively identify ambiguity, ask effective clarification questions, and recover correct reasoning paths through user interaction."
• The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning
• EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments - cool study and findings, reinforcement learning etc.
• Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
• Trust Region Policy Distillation
• Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
• Spectral Rewiring for Exploration, Purification, and Model Merging
• Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning
• S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation - nice AI for science model, see project page also https://scienceone-ai.github.io/S1-Omni/
• DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment
• ISO: An RLVR-Native Optimization Stack
• Evoflux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents - SLMs ideas
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
• The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
• Flux-OPD: On-Policy Distillation with Evolving Contexts
• Beacon: Knowing When and How to Perform Agentic Visual Reasoning
• Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering
• Enhancing Rubric-based RL via Self-Distillation
• LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
• When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation
• ChronoVision: Temporal Reasoning via Latent State Reconstruction
• Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models
• Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning - has reasoning for search applications examples, nice paper
• Stealing Reasoning Traces from Proprietary LLM APIs
• Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design
• OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution
• Learning to Orchestrate Agents in Natural Language with the Conductor - Sakana AI, ICLR 2026
• Evo-Bench: Can Language Models Improve Agent Harness?
• Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning
• How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks
• Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements - evolution strategies
• SPADE: Self-Play in Adaptive Synthetic Executable Environments
• SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?
• From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier - AI for Math, Terence Tao
• Evidence-Informed LLM Beliefs for Continual Scientific Discovery
• Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models
• Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress
• On-policy Distillation with Verifiable Reward
• Is Next-Chunk Reasoning RL Really Better than SFT? Revisiting Training Strategies under no-CoT Data
• Prefix Sliding for efficient test-time scaling
• TTPO: Test-Time Policy Optimization
• D 3 MOPD: Adaptive Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation
• Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
• WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation
• On-Policy Self-Distillation without Any Supervision
• Language Models Can Control Their Own Attention
• Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
• Post-Training Language Models for Gold-Medal Performance in Coding Competitions
• Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement
• RISE: Recursive Improvement via Self-Extrapolating Policy Distillation
• MemoryWalker: Stop Training Agents on Contexts They Never Saw
• BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
• PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents
• SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
• Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation
• Expert-Space Exploration in MoE Reinforcement Learning
• When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
• Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents
• An Empirical Study of Harness Design for Coding Agents
• EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
• CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
• Harness-Zero: Harness Distillation via Agent-as-Harness
• Agensh: Scaling Organizational Intelligence to 1,024 Agents
• Agent-Editing World Model: Rethinking World Modeling for LLM Agents
• ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds
• Rufus-Air: An Open LLM Post-Training Recipe - nice end-to-end descriptions and tech report
• SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL
• Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge
• EasyPPO: Stabilizing the Critic Is Key
• Marathoner: Ultra-Long-Horizon Autonomous Intelligence
• QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
• Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents
• Language Models that Play Chess and Explain Their Moves
• OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
• OPD Before RL: Warm-Starting Rubric-Based RL with On-Policy Distillation
• Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective
• Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals
• SatNav: A Scalable Benchmark for Long-Horizon UAV Vision-Language Navigation from Satellite Imagery - NeurIPS 2026 Evaluations and Datasets
• When Do Biological Reasoning Models Use Their Biological Inputs?
• Stateless Language Agents: Scaling Long-Horizon Automated Research - deep research
• PANINI-DR: Controlled Deep Research through Generative Semantic Workspaces
• Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
• Hybrid Reasoning Agents
• Athanor: An Agent-Native System for Autonomous Post-Training Research
• FDE-Gym: The Last Mile of AI Delivery - interesting real-world tasks
• Into the Unknown: Benchmarking AI agents on open scientific research problems
• Evidence-First Agentic Verification
• Predicting Privileged Information Effectiveness for On-Policy Self-Distillation
• Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild? - NeurIPS 2026 evaluation and datasets track; Anthropic
• Harness Scaling for Generalizable Agentic Reinforcement Learning
• Recursive Self-Improvement through Multi-Agent Self-Supervision
• EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement? - project page blog is nice also https://mercury7353.github.io/self-evaluation-bench/blog/when-ai-agents-design-the-test.html
• RSIHub: A Composable Framework for Agent-Evolution Methods - NeurIPS 2026 Meta-Agents Workshop
• InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants
• Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation
• Training Foundation Models with a Single Algorithm
• Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces
• Harness-Model Collaboration for Long-Horizon Continual Learning
• SciGame: Let Agents Play the Scientific Discovery Games in the Real and Counterfactual World
• Follow the Gradient: Tracing Safety Degradation Back to Training Data
• FrontierPhysics: Evaluating Agents for End-to-End Physics Research
• SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding
• You don't need long context
• Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime
• P2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors
• TinyAstral: Transferring Self-Evolving Harnesses to Tiny Generalist Manipulation Agents
• InfraRSI: Harnessing Recursive Self-Improvement for Training Infrastructure
• Evolving Infra: Agent-Adaptive LLM Serving at Scale - MIT Han Lab
• Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
• On the Design of Hyper-Connections: Simplification, Stability, and Scalability

Topic - Audio Reasoning

• Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning

Topic - Routing

• When to Reason: Semantic Router for vLLM
• TCAndon-Router: Adaptive Reasoning Router for Multi-Agent Collaboration
• ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces

Topic - Reasoning in Diffusion Models

• Video Models Reason Early: Exploiting Plan Commitment for Maze Solving

Topic - Parallel Reasoning

Start with the first-linked survey.

• A Survey on Parallel Reasoning
• Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning
• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
• Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing

Topic - Verifier-free RL and approaches without External Rewards

Emerging topic, seen a few papers around this theme recently.

• Learning to Reason without External Rewards - "We propose Intuitor, an RLIF method that uses a model's own confidence, termed self-certainty, as its sole reward signal. Intuitor replaces external rewards in Group Relative Policy Optimization (GRPO) with self-certainty scores, enabling fully unsupervised learning. Experiments demonstrate that Intuitor matches GRPO's performance on mathematical benchmarks while achieving superior generalization to out-of-domain tasks like code generation, without requiring gold solutions or test cases."
• NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning
• Reinforcing General Reasoning without Verifiers
• RLPR: Extrapolating RLVR to General Domains without Verifiers
• The Invisible Leash: Why RLVR May Not Escape Its Origin
• Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process

Topic - Surveys and reviews

• Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
• Reinforcement Learning Enhanced LLMs: A Survey
• From System 1 to System 2: A Survey of Reasoning Large Language Models
• What, How, Where, and How Well? A Survey on Test-Time Scaling in Large Language Models - section 4.1 is on reasoning specifically but general paper is nicely organized and relevant
• Knowledge Augmented Complex Problem Solving with Large Language Models: A Survey - Zhejiang, mainly reasoning related topics
• Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
• Sailing AI by the Stars: A Survey of Learning from Rewards in Post-Training and Test-Time Scaling of Large Language Models - nice related topics in general about learning from rewards, also has nice Github https://github.com/bobxwu/learning-from-rewards-llm-papers
• Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning - CoCoNut type experiments on latent space reasoning etc.
• A Survey on Latent Reasoning - " We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes."
• Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models - associated Github page https://github.com/yuelinan/Awesome-Efficient-R1-style-LRMs
• Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks

Topic - Tool Use/Function Calling

• FunReason: Enhancing Large Language Models' Function Calling via Self-Refinement Multiscale Loss and Automated Data Refinement - general methodology

Topic - Reasoning + Agents

• ATLaS: Agent Tuning via Learning Critical Steps
• Agent models: Internalizing Chain-of-Action Generation into Reasoning models
• The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks
• Distilling LLM Agent into Small Models with Retrieval and Code Tools - nice, not stricty about agents but more about distillation; read this comment on HF for quick summary https://huggingface.co/posts/m-ric/682683815641001; "when trying to distil reasoning capability from a strong LLM ("teacher") into a smaller one ("student"), it's much better to use Agent traces than CoT traces."
• Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction - nice paper
• Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward - Ant Group, no code shared though afaict
• PVPO: Pre-Estimated Value-Based Policy Optimization for Agentic Reasoning
• SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents - "While prior work has proposed training recipes for base or instruction-tuned LLMs, we focus on continual reinforcement learning (RL) of reasoning-optimized models to further enhance agentic skills while preserving reasoning ability. Towards this end, we propose a simple RL recipe with entirely synthetic data, which we apply to various open-source LLMs. Our best variant SFR-DR-20B achieves up to 28.7% on Humanity's Last Exam benchmark."
• Emergent Collusion in Long-Horizon LLM Agent Interaction

Topic - Information retrieval and search

• Learning More Effective Representations for Dense Retrieval through Deliberate Thinking Before Search
• O1 Embedder: Let Retrievers Think Before Action
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Open Deep Search: Democratizing Search with Open-source Reasoning Agents
• Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking - uses BRIGHT benchmark
• DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments - deep research, does have a reasoning type model involved for planning stage
• DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learning - results seem good
• ReasonIR: Training Retrievers for Reasoning Tasks - uses BRIGHT benchmark
• An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents - follow-up to Search-R1 paper
• FREESON: Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS
• ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning
• VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning - multimodal
• From Token to Action: State Machine Reasoning to Mitigate Overthinking in Information Retrieval - uses BRIGHT benchmark, defines a refine and rerank operation that they use to cut down token use compared to CoT
• R2ec : Towards Large Recommender Models with Reasoning - use reasoning for recommendation systems
• Decoupled Planning and Execution: A Hierarchical Reasoning Framework for Deep Search
• ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability - as of August 2025 they claim SOTA on the BRIGHT benchmark
• DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking - nice for reasoning in agents; "In DeepWideSearch, agents must process a large volume of data, each requiring deep reasoning over multi-hop retrieval paths. Specifically, we propose two methods to converse established datasets, resulting in a curated collection of 220 questions spanning 15 diverse domains."
• LimRank: Less is More for Reasoning-Intensive Information Reranking - uses BRIGHT benchmark
• Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities
• LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval
• Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections
• Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems - ACL 2026; BRIGHT Pro follow-up of BRIGHT benchmark
• PLUME: Latent Reasoning Based Universal Multimodal Embedding
• Xetrieval: Mechanistically Explaining Dense Retrieval
• RL-Index: Reinforcement Learning for Retrieval Index Reasoning - BRIGHT benchmark used
• RecGPT-V3 Technical Report - recommendation systems

Topic - X of Thoughts variants

• Path-of-Thoughts: Extracting and Following Paths for Robust Relational Reasoning with Large Language Models
• Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
• AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning

Topic - Efficiency, Decoding Strategies, Implementation Tricks etc.

• Efficiently Serving LLM Reasoning Programs with Certaindex
• Path-Consistency: Prefix Enhancement for Efficient Inference in LLM
• Reward-Guided Speculative Decoding for Efficient LLM Reasoning
• LightThinker: Thinking Step-by-Step Compression
• Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
• A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond - their Github page is nice also: https://github.com/XiaoYee/Awesome_Efficient_LRM_Reasoning
• Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models - Github https://github.com/DevoAllen/Awesome-Reasoning-Economy-Papers
• Efficient Inference for Large Reasoning Models: A Survey - Github https://github.com/yueliu1999/Awesome-Efficient-Inference-for-LRMs
• Hawkeye:Efficient Reasoning with Model Collaboration - uses a Qwen 0.5B SLM to speed up reasoning
• SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
• SeerAttention-R: Sparse Attention Adaptation for Long Reasoning - Microsoft Research; uses https://github.com/tile-ai/tilelang that I need to learn about also O_o
• R-KV: Redundancy-aware KV Cache Compression for Training-Free Reasoning Models Acceleration
• Inference-Time Hyper-Scaling with KV Cache Compression
• Less Is More: Training-Free Sparse Attention with Global Locality for Efficient Reasoning
• Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs - nice review
• Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners - Tencent
• Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
• Arbitrage: Efficient Reasoning via Advantage-Aware Speculation - "Across multiple mathematical reasoning benchmarks, Arbitrage consistently surpasses prior step-level Speculative Decoding baselines, reducing inference latency by up to 2x at matched accuracy."
• Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
• RelayLLM: Efficient Reasoning via Collaborative Decoding

Topic - Ensembling, Boosting, Stacking etc.

• Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning

Topic - Evaluation of reasoning

• Are Your LLMs Capable of Stable Reasoning?
• The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzles
• xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

Topic - Mathematics

• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
• AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling
• LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models - see also the Yannic Kilcher paper review
• MetaLadder: Ascending Mathematical Solution Quality via Analogical-Problem Reasoning Transfer
• Leanabell-Prover: Posttraining Scaling in Formal Reasoning
• FANS -- Formal Answer Selection for Natural Language Math Reasoning Using Lean4
• LeanProgress: Guiding Search for Neural Theorem Proving via Proof Progress Prediction
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning - Lean
• AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset - Nvidia
• Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math - Microsoft technical report
• FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models - benchmark but technical report is nice, also there is a Github at https://spherelab.ai/FormalMATH/
• CombiBench: Benchmarking LLM Capability for Combinatorial Mathematics - finally some combinatorics :) seems like a difficult area, model performance on benchmark is quite low even with Kimina Prover
• MPS-Prover: Advancing Stepwise Theorem Proving by Multi-Perspective Search and Data Curation
• DeepTheorem: Advancing LLM Reasoning for Theorem Proving Through Natural Language and Reinforcement Learning
• Mathesis: Towards Formal Theorem Proving from Natural Languages
• Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving
• Towards Solving More Challenging IMO Problems via Decoupled Reasoning and Proving - Tencent, nice approach
• MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimization
• Seed-Prover: Deep and Broad Reasoning for Automated Theorem Proving - ByteDance; 5/6 problems on IMO 2025
• Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction - really cool; "We introduce Goedel-Prover-V2, a series of open-source language models that set a new state-of-the-art in automated theorem proving. Built on the standard expert iteration and reinforcement learning pipeline, our approach incorporates three key innovations: (1) Scaffolded data synthesis: We generate synthetic tasks of increasing difficulty to train the model to master increasingly complex theorems; (2) Verifier-guided self-correction: We enable the model to iteratively revise its proofs by leveraging feedback from the Lean compiler; (3) Model averaging: We merge model checkpoints to mitigate the decrease in model output diversity in later stages of training."
• StepFun-Formalizer: Unlocking the Autoformalization Potential of LLMs through Knowledge-Reasoning Fusion
• Putnam-AXIOM: A Functional and Static Benchmark
• Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math - "a human-annotated, step-level verification benchmark produced with over 500 hours of human labor"
• Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision
• Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experience
• CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving
• Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematics - not entirely about reasoning model per se, but nice approach also
• Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math
• Reasoning over mathematical objects: on-policy reward modeling and test time aggregation
• MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval - nice mathematics benchmark, ICLR 2026, MIT team; project webpage is https://mathnet.mit.edu/
• Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs
• OProver: A Unified Framework for Agentic Formal Theorem Proving
• An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics

Topic - Coding

• ACECODER: Acing Coder RL via Automated Test-Case Synthesis
• S*: Test Time Scaling for Code Generation
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation - not directly about reasoning but nice approach
• Think Like Human Developers: Harnessing Community Knowledge for Structured Code Reasoning
• LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?

Topic - Biology

• BioProBench: Comprehensive Dataset and Benchmark in Biological Protocol Understanding and Reasoning - really cool area
• BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model

Topic - Other specialized reasoning areas (medical, legal, etc.)

• ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning
• MedS3: Towards Medical Small Language Models with Self-Evolved Slow Thinking
• MedAgentsBench: Benchmarking Thinking Models and Agent Frameworks for Complex Medical Reasoning
• Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning
• m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models
• DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large Language Models - Qwen team
• MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports
• CodeV-R1: Reasoning-Enhanced Verilog Generation - hardware design language HDL
• MedAgentGym: Training LLM Agents for Code-Based Medical Reasoning at Scale
• Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
• ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning
• Enhancing Step-by-Step and Verifiable Medical Reasoning in MLLMs
• An Agentic System for Rare Disease Diagnosis with Traceable Reasoning
• ChemDFM-R: An Chemical Reasoner LLM Enhanced with Atomized Chemical Knowledge
• Reasoning Language Models for Root Cause Analysis in 5G Wireless Networks
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning - this is for science in general; these claims from the abstract: (1) Retrieving task-relevant knowledge from model parameters is a critical bottleneck for LLMs in scientific reasoning; (2) Reasoning models consistently benefit from external knowledge added in-context on top of the reasoning enhancement; (3) Enhancing verbalized reasoning improves LLMs' ability to surface task-relevant knowledge.
• Baichuan-M2: Scaling Medical Capability with Large Verifier System - 32B medical reasoning model, nice training details
• From What to Why: A Multi-Agent System for Evidence-based Chemical Reaction Condition Reasoning
• ELAIPBench: A Benchmark for Expert-Level Artificial Intelligence Paper Understanding
• Chem-R: Learning to Reason as a Chemist
• AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems - very cool domain! "We introduce AstroReason-Bench, a comprehensive benchmark for evaluating agentic planning in Space Planning Problems (SPP), a family of high-stakes problems with heterogeneous objectives, strict physical constraints, and long-horizon decision-making."
• Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Making - "Key capabilities include: (i) proactive information acquisition to resolve ambiguity; (ii) long-horizon reasoning that unifies scattered evidence into coherent diagnoses; and (iii) adaptive hallucination suppression to ensure factual reliability. Empirical evaluations demonstrate that Baichuan-M3 achieves state-of-the-art results on HealthBench, the newly introduced HealthBench-Hallu and ScanBench, significantly outperforming GPT-5.2 in clinical inquiry, advisory and safety."
• MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs
• LawThinker: A Deep Research Legal Agent in Dynamic Environments
• DeonticBench: A Benchmark for Reasoning over Rules - "... we introduce DEONTICBENCH, a benchmark of 6,232 tasks across U.S. federal taxes, airline baggage policies, U.S. immigration administration, and U.S. state housing law. These tasks can be approached in multiple ways, including direct reasoning in language or with the aid of symbolic computation. Besides free-form chain-of-thought reasoning, DEONTICBENCH enables an optional solver-based workflow in which models translate statutes and case facts into executable Prolog, leading to formal problem interpretations and an explicit program trace. We release reference Prolog programs for all instances."
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation
• FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios - nice project page https://ai4manufacturing.github.io/forge-web/
• FinanceComplexQA: Benchmarking Agentic Reasoning on Industrial-grade Financial Documents

Topic - Multimodal

TODO: expand with separate tags

• Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
• https://embodied-reasoner.github.io/
• Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging - cool, claims transfering reasoning ability from LLM to VLM
• Active-O3: Empowering Multimodal Large Language Models with Active Perception via GRPO - Zhejiang and Ant Group, really cool
• Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO
• VideoDeepResearch: Long Video Understanding With Agentic Tool Using - "Our approach relies solely on a text-only large reasoning model (LRM) combined with a modular multi-modal toolkit, including multimodal retrievers and visual perceivers, all of which are readily available in practice. For each LVU task, the system formulates a problem-solving strategy through reasoning, while selectively accessing and utilizing essential video content via tool using."
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning - "Without relying on any human-annotated question-answer pairs, our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks."
• Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs - spatial reasoning
• Scaling RL to Long Videos - "We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 52K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling."
• Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning - cool paper and benchmark
• MindJourney: Test-Time Scaling with World Models for Spatial Reasoning
• M2-Reasoning: Empowering MLLMs with Unified General and Spatial Reasoning - Ant group
• Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning - cool dataset, using a kind of "sketchpad" while reasoning
• ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning - NVIDIA; robotics related also, nice VLA approach
• 3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding - not sure if I understood appendix correctly but it seems the model dynamically moves around the 3D scene to get better perspectives to answer queries; cool approach if so, need to reread the details
• InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation - "Additionally, InstructVLA surpasses baseline VLMs on multimodal tasks and exhibits inference-time scaling by leveraging textual reasoning to boost manipulation performance in both simulated and real-world settings."
• Ovis2.5 Technical Report - Alibaba; nice technical report and training explanation, also model has nice "grounding reference" feature (see in the appendix for examples)
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency - Shanghai AI Lab; see also the Intern-S1 science foundation model (link in essential reading section)
• Self-Rewarding Vision-Language Model via Reasoning Decomposition - Tencent; nice ideas "Vision-SR1 decomposes VLM reasoning into two stages: visual perception and language reasoning. The model is first prompted to produce self-contained visual perceptions that are sufficient to answer the question without referring back the input image. To validate this self-containment, the same VLM model is then re-prompted to perform language reasoning using only the generated perception as input to compute reward. This self-reward is combined with supervision on final outputs, providing a balanced training signal that strengthens both visual perception and language reasoning."
• Planning with Reasoning using Vision Language World Model
• EmbRACE-3K: Embodied Reasoning and Action in Complex Environments
• Reinforced Visual Perception with Tools - Tool use for vision models, reasoning about which tools to use (like depth perception or object detection)
• Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search
• MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlook - nice, it's a writeup/summary of: ICCV 2025 MARS2 Workshop and Challenge - Multimodal Reasoning and Slow Thinking in the Large Model Era: Towards System 2 and Beyond
• Nav-R1: Reasoning and Navigation in Embodied Scenes - cool; nice website also with videos https://aigeeksgroup.github.io/Nav-R1/
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning - I thought section 4 on the Emergent Properties was interesting; "We identify eight distinct types of unseen visual manipulations, with zoom-in operations being the most common. As shown in Figure 2 (lower panel) and Figure 4, these manipulations also include inpainting, multi-box generation, motion forecasting, perspective transformation, and region cropping, among others. These emergent behaviors are not rare: on some benchmarks, unseen manipulations account for up to 10% of all visual operations produced during inference. Importantly, these operations are not arbitrary artifacts but precise and task-effective visual actions that contribute directly to problem solving. For example, when asked “Is the bell pepper red or yellow?”, the model automatically generates a zoomed-in view to better distinguish subtle color differences, which closely mirroring human visual inspection without explicit prompting"
• VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning
• SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning
• PROGRESSLM: Towards Progress Reasoning in Vision-Language Models - "Estimating task progress requires reasoning over long-horizon dynamics rather than recognizing static visual content. While modern Vision-Language Models (VLMs) excel at describing what is visible, it remains unclear whether they can infer how far a task has progressed from partial observations. To this end, we introduce Progress-Bench, a benchmark for systematically evaluating progress reasoning in VLMs."
• ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning - awesome project page also https://3dlg-hcvc.github.io/revsi/
• SketchVLM: Vision language models can annotate images to explain thoughts and guide users
• ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue - drone searching with GIS and reasoning steps
• S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence
• EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos

Topic - Robotics

• RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics
• Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation
• Robix: A Unified Model for Robot Interaction, Reasoning and Planning
• Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoning - nice project page also https://rd-vla.github.io/
• RynnBrain: Open Embodied Foundation Models - DAMO Alibaba

Topic - Adjacent subjects, maybe related

Personal reading notes/papers that gave me some reasoning-related ideas

• Answer Set Networks: Casting Answer Set Programming into Deep Learning
• UQE: A Query Engine for Unstructured Databases
• LTNtorch: PyTorch Implementation of Logic Tensor Networks
• SymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning
• Shifting Long-Context LLMs Research from Input to Output - generating large reasoning chains
• BlendRL: A Framework for Merging Symbolic and Neural Policy Learning - ICLR 2025
• Critical Thinking: Which Kinds of Complexity Govern Optimal Reasoning Length? - more theoretical but maybe interesting
• Pantograph: A Machine-to-Machine Interaction Interface for Advanced Theorem Proving, High Level Reasoning, and Data Extraction in Lean 4 - Lean, mathematics
• Gold-Medal-Level Olympiad Geometry Solving with Efficient Heuristic Auxiliary Constructions
• Sparse Reward Subsystem in Large Language Models
• NanoKnow: How to Know What Your Language Model Knows - might be useful for "reasoning vs remembering?" studies
• The Format Tax - "... structured output requirements - JSON, XML, LaTeX, Markdown - substantially degrade reasoning and writing performance across open-weight models"

---

Datasets

• https://huggingface.co/datasets/open-r1/OpenR1-Math-220k - OpenR1-Math-220k is a large-scale dataset for mathematical reasoning. It consists of 220k math problems with two to four reasoning traces generated by DeepSeek R1 for problems from NuminaMath 1.5. The traces were verified using Math Verify for most samples and Llama-3.3-70B-Instruct as a judge for 12% of the samples, and each problem contains at least one reasoning trace with a correct answer.
• https://huggingface.co/collections/open-r1/reasoning-datasets-67980cac6e816a0eda98c678 - Datasets with reasoning traces for math and code released by the community
• DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning - DeepMath-103K, a new, large-scale dataset comprising approximately 103K mathematical problems, specifically designed to train advanced reasoning models via RL.

---

Benchmarks

• The CLRS-Text Algorithmic Reasoning Language Benchmark
• LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
• BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
• NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
• LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
• PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
• ProcessBench: Identifying Process Errors in Mathematical Reasoning
• ToolComp: A Multi-Tool Reasoning & Process Supervision Benchmark
• PhD Knowledge Not Required: A Reasoning Challenge for Large Language Models
• HARP: A challenging human-annotated math reasoning benchmark
• SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially?
• CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning - Google, ICLR 2025
• QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?
• CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesis
• CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generation
• FINEREASON: Evaluating and Improving LLMs' Deliberate Reasoning through Reflective Puzzle Solving
• LR2Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems
• Nondeterministic Polynomial-time Problem Challenge: An Ever-Scaling Reasoning Benchmark for LLMs - cool idea but no Github/access to benchmark yet?
• CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges
• DeepMath-Creative: A Benchmark for Evaluating Mathematical Creativity of Large Language Models - nice area, more creative reasoning about mathematical problems and deduction
• R2MED: A Benchmark for Reasoning-Driven Medical Retrieval
• UQ: Assessing Language Models on Unsolved Questions - not completely reasoning-based but quite relevant and I like the idea; website is here https://uq.stanford.edu/
• Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math - "We introduce Hard2Verify, a human-annotated, step-level verification benchmark produced with over 500 hours of human labor. Hard2Verify is designed to rigorously assess step-level verifiers at the frontier: Verifiers must provide step-level annotations or identify the first error in responses generated by frontier LLMs for very recent, challenging, and open-ended math questions."
• FrontierCS: Evolving Challenges for Evolving Intelligence - "Combining open-ended design, measurable progress, and expert curation, FrontierCS provides a benchmark at the frontier of computer-science difficulty. Empirically, we find that frontier reasoning models still lag far behind human experts on both the algorithmic and research tracks, that increasing reasoning budgets alone does not close this gap, and that models often over-optimize for generating merely workable code instead of discovering high-quality algorithms and system designs."
• T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning
• OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning - Databricks
• Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation - ByteDance
• TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
• Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
• PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems
• AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
• Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
• MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operations - Alibaba e-commerce; "We introduce MerchantBench, a 365-day order-level simulation grounded in 98,843 real e-commerce product records and equipped with 26 tools for agent interaction. MerchantBench couples promptly observable Upstream Supplier Events with delayed Downstream Order Outcomes, requiring agents to follow individual order lifecycles and revisit earlier decisions."
• StochBench: A Domain-Specific Benchmark for Stochastic Processes in Lean

---

Blogs and articles

• HKUST - Simple Reinforcement Learning for Reasoning
• OpenAI - Deliberative alignment: reasoning enables safer language models
• DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL - main project page is https://agentica-project.com/, HuggingFace page is https://huggingface.co/agentica-org/DeepScaleR-1.5B-Preview
• Open R1: Update #3 - HuggingFace Open-R1 update on Codeforces and IOI 24 Informatics Olympiads, datasets etc.
• Understanding R1-Zero-Like Training: A Critical Perspective
• YouTube - Nathan Lambert - GRPO's new variants and implementation secrets
• Reasoning models don't always say what they think - nice Anthropic blog, basically they take a reasoning model and place hints to correct answer in the input prompts (during eval), then measure whether model mentions it used the hint to reach its final answer etc.
• DeepCoder: A Fully Open-Source 14B Coder at O3-mini Level - Through a joint collaboration between the Agentica team and Together AI, we release DeepCoder-14B-Preview, a code reasoning model finetuned from Deepseek-R1-Distilled-Qwen-14B via distributed RL. It achieves an impressive 60.6% Pass@1 accuracy on LiveCodeBench (+8% improvement), matching the performance of o3-mini-2025-01-031 (Low) and o1-2024-12-17 with just 14B parameters. We’ve open-sourced our dataset, code, training logs, and systems optimizations for everyone to progress on scaling and accelerating intelligence with RL.
• Shunyu Yao - The Second Half - awesome blog on status of reinforcement learning as of April 2025 by OpenAI researcher
• ServiceNow Reseach - PipelineRL - We are excited to open-source PipelineRL, an experimental RL implementation that tackles a fundamental challenge in large-scale Reinforcement Learning with LLMs: the trade-off between inference throughput and on-policy data collection. PipelineRL's key innovation is inflight weight updates during RL training. This allows PipelineRL to achieve constantly high inference throughput and minimize the lag between the weights used for rollouts and the most recently updated model weights. The result: fast and stable RL training for large language models.
• OpenAI's o3: Over-optimization is back and weirder than ever - Nathan Lambert blog
• Reinforcement learning with random rewards actually works with Qwen 2.5 - Nathan Lambert blog, see new papers Topic about "Verifier-free RL and approaches without External Rewards" above in main section
• Crafting a good (reasoning) model - Nathan Lambert talk video/blog notes
• The rise of reasoning machines - Nathan Lambert blog
• POLARIS: A POst-training recipe for scaling reinforcement Learning on Advanced ReasonIng modelS - "Our 4B model achieves an impressive 81.2% Pass@1 accuracy on AIME24 and 79.4% Pass@1 accuracy on AIME25, outperforming state-of-the-art commercial models like Claude-4-Opus, Grok-3-Beta, and o3-mini-high(2025/01/31) via scaling reinforcement learning on open-source data. On AIME25, POLARIS astonishingly achieves comparable performance to Qwen3-235B-A22B while using less than 2% of its parameters and can be deployed on consumer-grade GPUs."
• HuggingFace Spaces - Scaling Test Time Compute with Open Models - from December 2024 but nice presentation and introduction
• Self-Adapting Language Models - "We demonstrate SEAL in two domains: (1) Knowledge Incorporation, where the model integrates new factual information by generating logical implications as synthetic data, and (2) Few-Shot Learning, where the model autonomously selects data augmentations and training hyperparameters to adapt to new abstract reasoning tasks."
• DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL - "We introduce DeepSWE-Preview, a reasoning-enabled coding agent trained from Qwen3-32B with only reinforcement learning (RL). It achieves an impressive 59.0% on SWE-Bench-Verified with test-time scaling, reaching SOTA for open-weight coding agents (42.2% Pass@1, 71.0% Pass@16). DeepSWE is trained using rLLM, our framework for post-training language agents. We’ve open sourced everything—our dataset, code, training, and eval logs, for everyone to progress on scaling and improving agents with RL."
• rLLM: A Framework for Post-Training Language Agents - nice, learned about this from blog about DeepSWE above; good perspective on going beyond simple reasoning models
• Detecting misbehavior in frontier reasoning models - OpenAI
• What comes next with reinforcement learning - Nathan Lambert blog
• Kimina-Prover: Applying Test-time RL Search on Large Formal Reasoning Models - mathematics and theorem proving: "Our key innovations include: 1) Test-Time Reinforcement Learning Search: A trainable agentic proving framework that enables the model to recursively discover, combine and apply multiple lemmas to construct complex proofs, building on a novel lemma-enabled pattern. 2) Error-Fixing Capability: Kimina-Prover can read and interpret Lean’s error messages and propose targeted fixes, demonstrating significantly higher sample efficiency compared to regenerating proofs from scratch. These advancements enable Kimina-Prover to solve challenging mathematical problems and surpass prior methods. As shown in Figure 1, on the widely used miniF2F benchmark, Kimina-Prover achieves a state-of-the-art pass rate of 92.2%."
• Bourbaki (7b): SOTA 7B Algorithms for Putnam Bench (Part I: Reasoning MDPs)
• Context Rot: How Increasing Input Tokens Impacts LLM Performance - not specifically about reasoning models but interesting for complex problem solving in general and related areas though
• Scaling Test Time Compute to Multi-Agent Civilizations: Noam Brown - awesome interview and blog (o1 etc.)
• Interviewing Ross Taylor on the state of AI: Chinese open models, scaling reasoning, useful tools, and what comes next - Nathan Lambert blog; nice topics and discussion especially in 2nd half of the interview
• Interactive Reasoning Benchmarks | ARC-AGI-3 Preview - nice presentation of ARC Prize July-August 2025 challenges with game exploration environments
• Kimina-Prover-R1 - HuggingFace blog, showing how they train on formal reasoning with Lean; "A slimmed-down training pipeline from Kimina Prover, with core features and full compatibility with verl."
• https://www.open-thoughts.ai/blog - "A DataComp and Bespoke Labs community effort to curate the best open reasoning datasets."; this is the blog of the people behind OpenThoughts3 etc. recent datasets.
• Inside vLLM: Anatomy of a High-Throughput LLM Inference System - not directly reasoning related but a great blog for understanding vLLM, for model training purposes
• Why We Think - really nice blog post by Lilian Weng
• The Hidden Drivers of HRM's Performance on ARC-AGI - a blog by ARC prize team examining the recent Hierarchical Reasoning Model performance (27M parameter model that achieved high performance on AGI-1 leaderboard)
• DeepMath: A lightweight math reasoning Agent with smolagents
• Building Olmo 3 Think - Nathan Lambert video
• The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data - interesting blog; relevant about evals and benchmark saturation
• DeepSeek-V4: a million-token context that agents can actually use - nice HuggingFace blog with some of the new architectural features of DeepSeek-V4
• Nathan Lambert blog - Frontier post-training recipe review with Finbarr Timbers - also has YouTube https://www.youtube.com/watch?v=sbXEPxIazqY
• Anthropic - Making Claude a Chemist

Workshops

• Reasoning and Planning for Large Language Models, ICLR 2025, April 28 2025, Singapore
• XLLM @ ACL 2025 Shared Task-III: LLM for Structural Reasoning (LLM-SR) - saw several papers and submissions to this, e.g. LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation and LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning

---

Repos

• HuggingFace Open R1 - A fully open reproduction of DeepSeek-R1
• LLaMA-O1: Open Large Reasoning Model Frameworks For Training, Inference and Evaluation With PyTorch and HuggingFace
• Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions
• veRL: Volcano Engine Reinforcement Learning for LLM
• Recipes to scale inference-time compute of open models
• Facebook Research COCONUT paper - Training Large Language Model to Reason in a Continuous Latent Space
• Simple Reinforcement Learning for Reasoning - replication DeepSeek-R1-Zero and DeepSeek-R1 training on small models with limited data
• The Path to Open-Sourcing the DeepSeek Inference Engine - DeepSeek Open Infra Github, as of April 2025 they announce that they will open source their inference engine
• DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition - see the paper also in the Github
• PageIndex: Document Index for Reasoning-based RAG

Collections

• Linked from paper https://arxiv.org/abs/2501.02497 - This repository contains the resources for Test-time Computing: from System-1 Thinking to System-2 Thinking
• Curated collection of papers and resources on how to unlock the reasoning ability of LLMs and MLLMs.
• A curated list of language modeling researches for code (and other software engineering activities), plus related datasets. - has related subjects around "code enhances reasoning" general area