Anurag Koul

Anurag Koul

(अनुराग कौल)

Applied Scientist 2

Amazon

Biography

Hi! I’m an Applied Scientist at Amazon. Previously, I had finished my Ph.D. at Oregon State University under supervision of Prof. Alan Fern. My research interest broadly lies in reinforcement learning (RL) and have researched explainability in artificial intelligence, model-based RL, planning, offline RL, abstractions and in-context policy improvement with transformer models. Currently, I investigate Large Language Models (LLMs) for code generation. If you are interested in my research or want to collaborate, please feel free to reach out.

Interests
  • Artificial Intelligence
  • Deep Reinforcement Learning
  • Planning
  • Representation Learning
Education

Experience

 
 
 
 
 
Amazon
Applied Scientist 2
August 2024 – Present New York
  • Research and development of large language models for code generation.
 
 
 
 
 
Microsoft Research
PostDoctoral Researcher
October 2022 – July 2024 New York
  • Research on planning and latent-state representation for reinforcement learning
 
 
 
 
 
Microsoft Research
Research Intern
June 2022 – September 2022 New York
  • Research on safe reinforcement learning for systems
 
 
 
 
 
Intel AI Labs
Research Intern
June 2020 – September 2020 Remote
  • Research on model-based reinforcement learning for control
 
 
 
 
 
SAS
Research Intern
May 2019 – August 2019 North Carolina
  • Research on multi-agent reinforcement learning
 
 
 
 
 
Capgemini
Senior Software Engineer
August 2015 – August 2016 Gurugram, India
  • Developed full-stack web-products for telecommunication space.
 
 
 
 
 
Capgemini
Software Engineer
August 2014 – August 2015 Gurugram, India

Research Papers

Quickly discover relevant content by filtering research papers.
(2024). PcLast: Discovering Plannable Continuous Latent States. ICML.

PDF Cite

(2022). Offline Policy Comparison with Confidence: Benchmarks and Baselines. DRL Workshop, ICLR.

PDF Cite Poster Slides Docs

(2021). Re-understanding finite-state representations of recurrent policy networks. ICML.

PDF Cite

(2020). Dream and search to control: Latent space planning for continuous control. DRL Workshop, ICLR.

PDF Cite Code Poster Slides

(2019). Explainable reinforcement learning via reward decomposition. IJCAI.

PDF Cite

Academic Projects

maze-world
Random maze environments with different size and complexity for reinforcement learning and planning research.
maze-world
Towards Real World Reinforcement Learning
Investigating real-world control data for offline reinforcement learning.
Towards Real World Reinforcement Learning
Learning `n-step’ actions for control tasks
Learn a policy that outputs an action as well as the time-step for which the action should be repeated.
Learning `n-step' actions for control tasks
Learning Discrete Latent Dynamics for Planning
Investigating learning finite state representation of a world and learning optimal policy via policy iteration.
Learning Discrete Latent Dynamics for Planning
ma-gym
A collection of environment for multi-agent reinforcement learning research, built on top of openai-gym.
ma-gym

Contact