Reinforcement Learning - COMP4600/5300
Prof. Reza Ahmadzadeh
This class will provide a solid introduction to the field of Reinforcement Learning and Decision Making. The students will learn about the basic blocks, main approaches, and core challenges of Reinforcement Learning including tabular methods, Finite Markov Decision Processes, Dynamic Programming, Monte Carlo methods, Temporal-Difference learning, policy search, function approximation, exploration, and generalization. Through a combination of lectures, and written and coding assignments, students will become well versed in key ideas and techniques for RL. Assignments will include the basics of reinforcement learning. In addition, students will advance their understanding and the field of RL through a final project.
Course Description
This class will provide a solid introduction to the field of Reinforcement Learning and Decision Making. The students will learn about the basic blocks, main approaches, and core challenges of Reinforcement Learning including tabular methods, Finite Markov Decision Processes, Dynamic Programming, Monte Carlo methods, Temporal-Difference learning, policy search, function approximation, exploration, and generalization. Through a combination of lectures, and written and coding assignments, students will become well versed in key ideas and techniques for RL. Assignments will include the basics of reinforcement learning. In addition, students will advance their understanding and the field of RL through a final project.
Why Take this Course?
This course will prepare you to participate in the Reinforcement Learning research community. You will also be able to engage in exciting Reinforcement Learning industrial projects.
Course Objectives
By the end of the class students should be able to:
- Define the process of decision making using the finite Markov Decision Process.
- Describe and differentiate Model-based and Model-free RL approaches.
- Describe and differentiate Off-policy and On-policy RL approaches.
- Describe and differentiate Dynamic Programming, Monte Carlo, and Temporal difference
- Describe and differentiate function approximation methods
- Describe the exploration-exploitation dilemma and compare and contrast at least two approaches for addressing this challenge.
- Given an application problem, decide if it should be formulated as a RL problem; if yes be able to define it in terms of the state-space, action-space, transition model and reward model, state and justify what algorithm is best suited for addressing it.
- Implement common RL algorithms using Python
- Describe multiple criteria for analyzing RL algorithms and evaluate algorithms on these metrics.
Prerequisites
The only formal prerequisite for this class is COMP1020 - Computing II. However, we expect the following non-official prerequisites:
- Basic Calculus and Linear Algebra: You should be comfortable taking derivatives and understanding matrix vector operations and notation.
- Basic Statistics and Probabilities: Gaussian distributions, mean, standard deviation, etc.
- Proficiency in Python: All class assignments and projects will be in Python. If you have a lot of programming experience but in different language (e.g. C/C++/Matlab) you will probably be fine by going through a few python tutorials.
- Foundation of Machine Learning: Familiarity with basic Machine Learning techniques such as formulating cost functions, taking derivatives and performing optimization with gradient descent is a plus.
References
The official textbook for the course is [1] for which the PDF version is available for free through the authors' website.
[1] Reinforcement Learning: An Introduction, Sutton and Barto, 2nd Edition, 2018.
When will this course be offered?
This course will be offered in Fall 2021.
Need more info?
Contact the instructor: Prof. Reza Ahmadzadeh [reza {at} cs {dot} uml {dot} edu]
