2016 – 2022
Ph.D. in Computer Science
Oregon State University · Oregon, USA
Thesis: Investigating Latent State and Uncertainty Representations in Reinforcement Learning
about
Anurag Koul अनुराग कौल
Lead Research Scientist @ Salesforce
Palo Alto, California
I am a researcher who has been generally curious to understand human intelligence and building general intelligent systems. In early days, this formulated as understanding reinforcement learning agents — how to represent the world, the role of memory, and how to do decision-time planning to improve an agent's behavior in complex environments. This further evolved into learning from offline data, multi-agent setups, and a systematic study of goal-conditioned RL. More recently, I have delved into post-training of large agentic models and continue to focus on improving RL techniques and reasoning abilities of the language models.
- Reasoning
- Reinforcement Learning
- Planning
- Large Language Models
experience
Where I've worked
- Jul 2026 — present
Lead Research Scientist
Research on post-training of agentic large language models.
- Dec 2025 — May 2026
Member of Engineering
LLM post-training for agentic models (Laguna M.1 / XS.2) with focus on agentic mathematical reasoning and instruction following.
- Aug 2024 — Nov 2025
Applied Scientist II
LLM fine-tuning for code generation — RL with code-execution/rule-based feedback, internal evaluation benchmarks, and multi-LLM routing.
- Oct 2022 — Jul 2024
Postdoctoral Researcher
Goal-conditioned reinforcement learning with hierarchical abstractions for representation learning, planning, and large transformer models, under John Langford.
- Jun 2022 — Sep 2022
Research Intern
Safe model-based reinforcement learning for compute allocation in system control, under Siddhartha Sen.
- Jun 2020 — Mar 2021
Research Intern
Model-based reinforcement learning for control with decision-time planning and look-ahead search (MCTS, Rollout) and self-play, under Somdeb Majumdar.
- May 2019 — Aug 2019
Research Intern
Collaborative multi-agent reinforcement learning for inventory management; built ma-gym, an open-source collection of multi-agent environments, under Davood Hajinezhad and Afshin Oroojlooy.
- Sep 2015 — Aug 2016
Senior Software Engineer
Full-stack web development: Network Pricing Engine, Customer Enquiry Portal, and Network Packet Management.
- Aug 2014 — Sep 2015
Software Engineer
Developed web-crawlers and agents for gathering web data and automating general web tasks.
education
Education
2010 – 2014
Bachelor of Engineering (B.E.) in Computer Engineering
University of Mumbai · Maharashtra, India
research
Selected papers
2025
SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion
conference ACL
LLMs excel at code tasks but struggle in realistic repositories where project-specific APIs and cross-file dependencies matter, and tight inference-time latency budgets limit retrieval quality. SpecAgent is an agent that improves both latency and code-generation quality by proactively exploring repository files during indexing and constructing speculative context that anticipates future edits. Computing context asynchronously at indexing time masks latency, while its speculative nature improves generation. We further identify future-context leakage in existing benchmarks and build a synthetic, leakage-free benchmark, on which SpecAgent achieves absolute gains of 9-11% (48-58% relative) over the strongest baselines while significantly reducing inference latency.
2024
PcLast: Discovering Plannable Continuous Latent States
conference ICML
Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. We learn a representation that associates reachable states together for effective planning and goal-conditioned policy learning. PcLast couples a temporal contrastive objective with multi-step inverse dynamics to recover representations that capture the geometry of the environment despite exogenous noise, improving sample efficiency and planning quality.
2022
Offline Policy Comparison with Confidence: Benchmarks and Baselines
workshop DRL Workshop, ICLR
Decision makers often wish to use offline historical data to compare sequential-decision-making policies. We construct benchmarks (OPCC) for offline policy comparison with confidence, formalizing the problem of estimating the probability that one policy outperforms another from logged data, and provide baselines and analysis to spur progress on uncertainty-aware offline evaluation.
talks
Talks & presentations
May 2024
Towards Efficient Representation and Future-Context Transformers for Goal-Conditioned Planning
Tea Talk
Learning task-agnostic policies is essential for versatile AI agents. This talk first highlights limitations of current latent representations in capturing environment geometry, and introduces Plannable Continuous Latent States (PcLast) — combining temporal contrastive loss with multi-step inverse dynamics to build robust representations despite exogenous noise. It then examines the limitations of decision transformers in goal-conditioned tasks with changing world configurations, and improves decision-making by integrating implicit local dynamics through short-term imagined future rollouts as context — the Future Context Decision Transformer.
Feb 2024
Explaining RL Agents from the Lens of Perception, Memory, and Uncertainties
eXplainable AI approaches for Deep RL Workshop @ AAAI 2024
Understanding the causal reasoning behind RL agents’ decisions is vital for trust and adaptability in real-world scenarios. Several factors influence the decision-making process of these agents — how they perceive the world, how they use memory in conjunction with perception, and how they express the uncertainties inherent in decision-making. This talk outlines the advances achieved in each of these domains throughout my doctoral research.
projects
Open source & academic
maze-world
Random maze environments of varying size and complexity for reinforcement learning and planning research.
Towards Real-World Reinforcement Learning
Investigating real-world control data for offline reinforcement learning.
Learning Discrete Latent Dynamics for Planning
Investigating finite-state representations of a world model and learning optimal policies via policy iteration over the learned dynamics.
Learning 'n-step' Actions for Control Tasks
Learning a policy that outputs both an action and the number of time-steps for which the action should be repeated.
ma-gym
A collection of environments for multi-agent reinforcement learning research, built on top of OpenAI Gym.
contact
Let's talk
Interested in my research or want to collaborate? Reach out.