Reinforcement Learning

Module 1: K-Armed Bandit Problem
– Sequential Decision Making with Evaluative Feedback
– Learning Action Values
– Estimating Action Values Incrementally
– Optimistic initial values
– UCB Action Selection
– Contextual Bandits for Real World RL
Hands-on Lab: Understand expected values from different gambling games

Module 2: Markov Decision Processes
– Examples of MDPs
– The Reward Hypothesis
– Continuing Tasks
– Episodic and Continuing Tasks
Hands-on Lab: Understand Markov Decision Processes by creating a classifier for thrash random writing

Module 3: Value Functions and Bellman Equations
– Specifying Policies
– Value Functions
– Bellman Equation Derivation
– Optimal Policies
– Optimal Value Functions
– Using Optimal Value Functions to get Optimal Policies

Module 4: Dynamic Programming
– Iterative Policy Evaluation
– Policy Iteration
– Efficiency of Dynamic Programming
Hands-on Lab: Implement the Bellman equations and Dynamic Programming for a grid world game

Module 5: Monte Carlo for Prediction and Control
– What is Monte Carlo?
– Prediction
– Action Values
– Blackjack example
– Epsilon-soft policies
– Off-policy learning
Hands-on Lab:
– Implement the off-policy learning methods and compare them with the single policy ones for a grid world game
– Understand Markov Decision Processes by creating a classifier for thrash random writing

Module 6: On-policy Prediction with Approximation
– Parameterized Functions
– Generalization and Discrimination
– Value Error Objective
– Gradient Descent
Hands-on Lab: Implement your own Flappy Bird AI that learn to play only from the environment