ENN582 Reinforcement Learning and Optimal Control
To view more information for this unit, select Unit Outline from the list below. Please note the teaching period for which the Unit Outline is relevant.
| Unit code: | ENN582 |
|---|---|
| Prerequisite(s): | EGH431 |
| Assumed Knowledge: | Assumed knowledge from the undergraduate unit on state-space control, including state space, vector ODEs, some intuition about optimisation, and vector functions |
| Credit points: | 6 |
| Timetable | Details in HiQ, if available |
| Availabilities |
|
| CSP student contribution | $559 |
| Domestic tuition unit fee | $1,836 |
| International unit fee | $2,520 |
Unit Outline: Semester 2 - 6 Week C 2024, Gardens Point, Internal
| Unit code: | ENN582 |
|---|---|
| Credit points: | 6 |
| Pre-requisite: | EGH431 |
| Assumed Knowledge: | Assumed knowledge on state-space control, stochastic processes, optimisation and vector functions. Assumed familiarity with programming in Python. |
| Coordinator: | Daniel Quevedo | daniel.quevedo@qut.edu.au |
Overview
Robots are autonomous systems that rely on a variety of control technologies. This unit provides understanding of key concepts in optimal control and reinforcement learning concepts for use in the design of robotic systems. This unit prepares students for building robotics systems in future careers.
Learning Outcomes
On successful completion of this unit you will be able to:
- Critically analyse and reflect on various control design approaches.
- Design nonlinear control approaches for complex problems
- Implement a non-linear optimal controller
Content
Interactive lectures will concentrate on the following content
1) Introduction to dynamic programming and applications to deterministic optimal control
2) Stochastic control problems and Dynamic Programming
3) Approximate dynamic programming and reinforcement learning approaches to optimal control
Learning Approaches
Lectures: are used to provide an introduction to material, and immediate application of the material with small focussed problems. Principles are introduced, discussed and dissected in the lecture.
Simulation Experience: Simulation work is conducted in structured numerical experiments that provide exposure to decision and control systems. The simulation experiments link the theoretical elements of the lectures to practice. Computer labs are conducted individually, and assessed by short reports.
Feedback on Learning and Assessment
Feedback will be provided regularly throughout the unit by tutors and lecturers. Tutors and lecturers are available for feedback and advice in the lab sessions. In addition, you are encouraged to constructively discuss emerging ideas with your peers throughout the Unit.
Assessment
Overview
Assessment in this unit will be based on an individual portfolio involving the design and evaluation of prototypical decision and control systems. This will be complemented by a written exam that focuses on system analysis and choice of design methods.
Unit Grading Scheme
7- point scale
Assessment Tasks
Assessment: Problem Solving Tasks
During this unit, you will undertake a series of computer-based practical labs on reinforcement learning and optimal control concepts including topics of dynamic programming, stochastic control and reinforcement learning.
Assessment: Written Exam
You will independently address a series of control system analysis and design challenges, of varying difficulty.
Academic Integrity
Students are expected to engage in learning and assessment at QUT with honesty, transparency and fairness. Maintaining academic integrity means upholding these principles and demonstrating valuable professional capabilities based on ethical foundations.
Failure to maintain academic integrity can take many forms. It includes cheating in examinations, plagiarism, self-plagiarism, collusion, and submitting an assessment item completed by another person (e.g. contract cheating). It can also include providing your assessment to another entity, such as to a person or website.
You are encouraged to make use of QUT’s learning support services, resources and tools to assure the academic integrity of your assessment. This includes the use of text matching software that may be available to assist with self-assessing your academic integrity as part of the assessment submission process.
Further details of QUT’s approach to academic integrity are outlined in the Academic integrity policy and the Student Code of Conduct. Breaching QUT’s Academic integrity policy is regarded as student misconduct and can lead to the imposition of penalties ranging from a grade reduction to exclusion from QUT.
Resources
You have access to lab spaces and workshops at QUT and can use a range of tools after receiving an induction.
Learning material in this unit will be managed from its Canvas site
Resource Materials
Recommended text(s)
Textbook:
Dimitri Bertsekas: "Reinforcement Learning and Optimal Control"
Athena Scientific, 2019
Risk Assessment Statement
There are no unusual health or safety risks associated with this unit. You will be made aware of evacuation procedures and assembly areas in the first few weeks. In the event of a fire alarm sounding, or on a lecturer's or tutor's instruction, you should leave the room and assemble in the designated area which will be indicated to you. You should be conscious of your health and safety at all times whilst on campus.
Course Learning Outcomes
This unit is designed to support your development of the following course/study area learning outcomes.EN52 Master of Robotics and Artificial Intelligence
- Demonstrate and apply advanced and specialist discipline knowledge, concepts and practices in Robotics and AI
Relates to: Written Exam - Communicate complex information effectively and succinctly in oral and written form for diverse purposes and audiences
Relates to: Written Exam
EN72 Master of Advanced Robotics and Artificial Intelligence
- Demonstrate and apply advanced and specialist discipline knowledge, concepts and practices in Advanced Robotics and AI and Data Analytics domains
Relates to: Problem Solving Tasks, Written Exam - Communicate complex information effectively and succinctly in oral and written form for diverse purposes and audiences
Relates to: Written Exam
EN79 Graduate Diploma in Engineering Studies
- Demonstrate and apply advanced discipline knowledge, concepts and practices as they relate to contemporary Engineering practice
Relates to: Problem Solving Tasks, Written Exam - Effectively communicate Engineering problems, related complex data and information, and solutions in contemporary professional formats for diverse purposes and audiences
Relates to: Written Exam
Unit Outline: Semester 2 - 6 Week D 2024, Gardens Point, Internal
| Unit code: | ENN582 |
|---|---|
| Credit points: | 6 |
| Pre-requisite: | EGH431 |
| Assumed Knowledge: | Assumed knowledge on state-space control, stochastic processes, optimisation and vector functions. Assumed familiarity with programming in Python. |
| Coordinator: | Daniel Quevedo | daniel.quevedo@qut.edu.au |
Overview
Robots are autonomous systems that rely on a variety of control technologies. This unit provides understanding of key concepts in optimal control and reinforcement learning concepts for use in the design of robotic systems. This unit prepares students for building robotics systems in future careers.
Learning Outcomes
On successful completion of this unit you will be able to:
- Critically analyse and reflect on various control design approaches.
- Design nonlinear control approaches for complex problems
- Implement a non-linear optimal controller
Content
Interactive lectures will concentrate on the following content
1) Introduction to dynamic programming and applications to deterministic optimal control
2) Stochastic control problems and Dynamic Programming
3) Approximate dynamic programming and reinforcement learning approaches to optimal control
Learning Approaches
Lectures: are used to provide an introduction to material, and immediate application of the material with small focussed problems. Principles are introduced, discussed and dissected in the lecture.
Simulation Experience: Simulation work is conducted in structured numerical experiments that provide exposure to decision and control systems. The simulation experiments link the theoretical elements of the lectures to practice. Computer labs are conducted individually, and assessed by short reports.
Feedback on Learning and Assessment
Feedback will be provided regularly throughout the unit by tutors and lecturers. Tutors and lecturers are available for feedback and advice in the lab sessions. In addition, you are encouraged to constructively discuss emerging ideas with your peers throughout the Unit.
Assessment
Overview
Assessment in this unit will be based on an individual portfolio involving the design and evaluation of prototypical decision and control systems. This will be complemented by a written exam that focuses on system analysis and choice of design methods.
Unit Grading Scheme
7- point scale
Assessment Tasks
Assessment: Problem Solving Tasks
During this unit, you will undertake a series of computer-based practical labs on reinforcement learning and optimal control concepts including topics of dynamic programming, stochastic control and reinforcement learning.
Assessment: Written Exam
You will independently address a series of control system analysis and design challenges, of varying difficulty.
Academic Integrity
Students are expected to engage in learning and assessment at QUT with honesty, transparency and fairness. Maintaining academic integrity means upholding these principles and demonstrating valuable professional capabilities based on ethical foundations.
Failure to maintain academic integrity can take many forms. It includes cheating in examinations, plagiarism, self-plagiarism, collusion, and submitting an assessment item completed by another person (e.g. contract cheating). It can also include providing your assessment to another entity, such as to a person or website.
You are encouraged to make use of QUT’s learning support services, resources and tools to assure the academic integrity of your assessment. This includes the use of text matching software that may be available to assist with self-assessing your academic integrity as part of the assessment submission process.
Further details of QUT’s approach to academic integrity are outlined in the Academic integrity policy and the Student Code of Conduct. Breaching QUT’s Academic integrity policy is regarded as student misconduct and can lead to the imposition of penalties ranging from a grade reduction to exclusion from QUT.
Resources
You have access to lab spaces and workshops at QUT and can use a range of tools after receiving an induction.
Learning material in this unit will be managed from its Canvas site
Resource Materials
Recommended text(s)
Textbook:
Dimitri Bertsekas: "Reinforcement Learning and Optimal Control"
Athena Scientific, 2019
Risk Assessment Statement
There are no unusual health or safety risks associated with this unit. You will be made aware of evacuation procedures and assembly areas in the first few weeks. In the event of a fire alarm sounding, or on a lecturer's or tutor's instruction, you should leave the room and assemble in the designated area which will be indicated to you. You should be conscious of your health and safety at all times whilst on campus.
Course Learning Outcomes
This unit is designed to support your development of the following course/study area learning outcomes.EN52 Master of Robotics and Artificial Intelligence
- Demonstrate and apply advanced and specialist discipline knowledge, concepts and practices in Robotics and AI
Relates to: Written Exam - Communicate complex information effectively and succinctly in oral and written form for diverse purposes and audiences
Relates to: Written Exam
EN72 Master of Advanced Robotics and Artificial Intelligence
- Demonstrate and apply advanced and specialist discipline knowledge, concepts and practices in Advanced Robotics and AI and Data Analytics domains
Relates to: Problem Solving Tasks, Written Exam - Communicate complex information effectively and succinctly in oral and written form for diverse purposes and audiences
Relates to: Written Exam
EN79 Graduate Diploma in Engineering Studies
- Demonstrate and apply advanced discipline knowledge, concepts and practices as they relate to contemporary Engineering practice
Relates to: Problem Solving Tasks, Written Exam - Effectively communicate Engineering problems, related complex data and information, and solutions in contemporary professional formats for diverse purposes and audiences
Relates to: Written Exam
Unit Outline: Semester 2 - 6 Week D 2024, Online
| Unit code: | ENN582 |
|---|---|
| Credit points: | 6 |
| Pre-requisite: | EGH431 |
| Assumed Knowledge: | Assumed knowledge on state-space control, stochastic processes, optimisation and vector functions. Assumed familiarity with programming in Python. |
Overview
Robots are autonomous systems that rely on a variety of control technologies. This unit provides understanding of key concepts in optimal control and reinforcement learning concepts for use in the design of robotic systems. This unit prepares students for building robotics systems in future careers.
Learning Outcomes
On successful completion of this unit you will be able to:
- Critically analyse and reflect on various control design approaches.
- Design nonlinear control approaches for complex problems
- Implement a non-linear optimal controller
Content
Interactive lectures will concentrate on the following content
1) Introduction to dynamic programming and applications to deterministic optimal control
2) Stochastic control problems and Dynamic Programming
3) Approximate dynamic programming and reinforcement learning approaches to optimal control
Learning Approaches
Lectures: are used to provide an introduction to material, and immediate application of the material with small focussed problems. Principles are introduced, discussed and dissected in the lecture.
Simulation Experience: Simulation work is conducted in structured numerical experiments that provide exposure to decision and control systems. The simulation experiments link the theoretical elements of the lectures to practice. Computer labs are conducted individually, and assessed by short reports.
Feedback on Learning and Assessment
Feedback will be provided regularly throughout the unit by tutors and lecturers. Tutors and lecturers are available for feedback and advice in the lab sessions. In addition, you are encouraged to constructively discuss emerging ideas with your peers throughout the Unit.
Assessment
Overview
Assessment in this unit will be based on an individual portfolio involving the design and evaluation of prototypical decision and control systems. This will be complemented by a written exam that focuses on system analysis and choice of design methods.
Unit Grading Scheme
7- point scale
Assessment Tasks
Assessment: Problem Solving Tasks
During this unit, you will undertake a series of computer-based practical labs on reinforcement learning and optimal control concepts including topics of dynamic programming, stochastic control and reinforcement learning.
Assessment: Written Exam
You will independently address a series of control system analysis and design challenges, of varying difficulty.
Academic Integrity
Students are expected to engage in learning and assessment at QUT with honesty, transparency and fairness. Maintaining academic integrity means upholding these principles and demonstrating valuable professional capabilities based on ethical foundations.
Failure to maintain academic integrity can take many forms. It includes cheating in examinations, plagiarism, self-plagiarism, collusion, and submitting an assessment item completed by another person (e.g. contract cheating). It can also include providing your assessment to another entity, such as to a person or website.
You are encouraged to make use of QUT’s learning support services, resources and tools to assure the academic integrity of your assessment. This includes the use of text matching software that may be available to assist with self-assessing your academic integrity as part of the assessment submission process.
Further details of QUT’s approach to academic integrity are outlined in the Academic integrity policy and the Student Code of Conduct. Breaching QUT’s Academic integrity policy is regarded as student misconduct and can lead to the imposition of penalties ranging from a grade reduction to exclusion from QUT.
Resources
You have access to lab spaces and workshops at QUT and can use a range of tools after receiving an induction.
Learning material in this unit will be managed from its Canvas site
Resource Materials
Recommended text(s)
Textbook:
Dimitri Bertsekas: "Reinforcement Learning and Optimal Control"
Athena Scientific, 2019
Risk Assessment Statement
There are no unusual health or safety risks associated with this unit. You will be made aware of evacuation procedures and assembly areas in the first few weeks. In the event of a fire alarm sounding, or on a lecturer's or tutor's instruction, you should leave the room and assemble in the designated area which will be indicated to you. You should be conscious of your health and safety at all times whilst on campus.
Course Learning Outcomes
This unit is designed to support your development of the following course/study area learning outcomes.EN52 Master of Robotics and Artificial Intelligence
- Demonstrate and apply advanced and specialist discipline knowledge, concepts and practices in Robotics and AI
Relates to: Written Exam - Communicate complex information effectively and succinctly in oral and written form for diverse purposes and audiences
Relates to: Written Exam
EN72 Master of Advanced Robotics and Artificial Intelligence
- Demonstrate and apply advanced and specialist discipline knowledge, concepts and practices in Advanced Robotics and AI and Data Analytics domains
Relates to: Problem Solving Tasks, Written Exam - Communicate complex information effectively and succinctly in oral and written form for diverse purposes and audiences
Relates to: Written Exam
EN79 Graduate Diploma in Engineering Studies
- Demonstrate and apply advanced discipline knowledge, concepts and practices as they relate to contemporary Engineering practice
Relates to: Problem Solving Tasks, Written Exam - Effectively communicate Engineering problems, related complex data and information, and solutions in contemporary professional formats for diverse purposes and audiences
Relates to: Written Exam