Note

This project is in the early stages of development.

IntroRL Provides A Framework For Exploring Reinforcement Learning.

It is based on the textbook “Reinforcement Learning An Introduction” By Sutton & Barto.

The textbook is also available in PDF format at the authors’ site.

This documentation of IntroRL is organized around the chapter structure of the Sutton & Barto textbook.

Many of the examples and figures are reproduced here in order to validate the IntroRL code.

There is another site by Shangtong Zhang that was of great help to me and which covers many areas of the textbook not covered here.

---

See the Code at: https://github.com/sonofeft/IntroRL

See the Docs at: http://introrl.readthedocs.org/en/latest/

See PyPI page at:https://pypi.python.org/pypi/introrl

IntroRL

Contents:

• QuickStart
  • Install IntroRL
  • Installation From Source
  • Running IntroRL
  • pip Error Messages
• IntroRL Overview
  • Models
  • 1) MDPs
  • Sample MDP
  • 2) Simulations
  • Sample Simulation
• Chapter 3 Finite MDP
  • Example 3.5
  • Example 3.8
• Chapter 4 DP
  • Policy Evaluation
  • Policy Improvement
  • Policy Iteration
  • Value Iteration
  • Example 4.1 Gridworld
  • Figure 4.2 Car Rental
  • Figure 4.3 Gambler’s Problem
• Chapter 5 Monte Carlo
  • First Visit MC Prediction
  • First Visit MC Exploring Starts
  • First Visit Epsilon-Greedy MC
• Chapter 6 TD Learning
  • Tabular TD(0) Prediction
  • Example 6.2 Random Walk
  • Figure 6.2 Batch Training
  • Sarsa On-Policy TD Control
  • Example 6.5 Windy Gridworld
  • Example 6.6 Cliff Walking
  • Figure 6.3 Expected Sarsa
  • Figure 6.5 Double Qlearning
• Chapter 7 n-step Bootstrapping
  • Figure 7.2 Random Walk
  • Few Examples
• Chapter 8 Planning & Learning
  • Tabular DynaQ
  • Figure 8.2 DynaQ Grid
  • Figure 8.4 Blocking Maze
  • Figure 8.5 Shortcut Maze
  • Example 8.4 Prioritized Sweeping
• Chapter 9 On-Policy Prediction with Approximation
  • Gradient MC Prediction
  • Figure 9.1
  • Semi-Gradient TD(0) Prediction
• Copyright
  • License
• Authors
  • Development Lead
  • Patches and Ideas
  • Initial Project Layout
• History
  • GitHub Log
  • PyHatch

Indices and tables

• Index
• Module Index
• Search Page