Multimodal Machine Learning

Course Description

This seminar introduces the foundations of multimodal machine learning and their applications on simulated and actual robot platforms. It covers the basics of machine learning, preliminary concepts of sensors (e.g., depth and color cameras, IMU, position encoders, etc.), sensor fusion techniques, specifications of robot platforms (e.g., Nao and the Pepper) and simulators. Students will work in teams on a topic in (deep) multimodal learning for robotics, and present their experimental results in oral presentations and written reports.

Update: According to HU's summer semester policy and due to Covid-19, the lectures, assignments, and presentations will be held online via Zoom.


Some experience with machine learning, programming, and robotics is required. Although we will not assume that you have an extensive background in these fields, basic understanding below-listed items will be necessary to follow the content of the course.

During the first weeks of the course, we will point out supplementary resources to introduce the bare minimum requirements of the course.


Upon completion of this seminar course, students will be able to:


The banner image was downloaded from here and it is subjected to copyright by softbank robotics.