direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Inhalt des Dokuments

Interactive Perception

Manipulating objects as dexterously as humans remains an open problem in robotics - not so much in carefully controlled environments such as factories, but in every day household environments, so-called unstructured environments. 

Interactive Perception, as the name suggests, is about acting to improve perception. The fundamental assumption is that the domains of perception and action cannot be separated, but form a complex which needs to be studied in its entirety. Using this approach, we try to design robot that explore their environment actively, in a way that reminds of how a baby explores a new toy.

Online Interactive Perception

We developed an RGB-D-based online algorithm for the interactive perception of articulated objects. In contrast to existing solutions to this problem, the online-nature of the algorithm permits perception during the interaction and addresses a number of shortcomings of existing methods. Our algorithm consists of three interconnected recursive estimation loops. The interplay of these loops is the key to the robustness of our proposed approach. The robustness stems from the feedback from our algorithm which can be used to adapt the robot's behavior.

Contact: Roberto Martín Martín

Active Outcome Recognition

Robotic behavior can not only reveal properties of the environment, like kinematic degrees of freedom, but it can also be adapted to better understand the interaction between robot and environment itself. Robots need to assess the outcomes of their own actions and the sensory data created from task-directed behavior may not be sufficient for such an estimation. We work on adapting robotic behavior such that it is easier to estimate the outcomes of actions. This is a problem of learning interactive perception.

Contact: Manuel Baum

Acquiring Kinematic Background Knowledge with Relational Reinforcement Learning

Click to see youtube video
Click to open video (youtube)

If a robot faces a novel, unseen object, it must first acquire information about the object’s kinematic structure by interacting with it. But there is an infinite number of possible ways to interact with an object. The robot therefore needs kinematic background knowledge: knowledge about the regularities that hint at the kinematic structure.

We developed a method for the efficient extraction of kinematic background knowledge from interactions with the world. We use relational model-based reinforcement learning, an approach that combines concepts from first-order logic (a relational representation) and reinforcement learning. Relational representations allow the robot to conceptualize the world as object parts and their relationship, and reinforcement learning enables it to learn from the experience it collects by interacting with the world. Using this approach, the robot is able to collect experiences and extract kinematic background knowledge that generalizes to previously unseen objects.




Alexander von Humboldt professorship - awarded by the Alexander von Humboldt foundation and funded through the Ministry of Education and Research, BMBF,
July 2009 - June 2014  


Roberto Martín-Martín and Oliver Brock. Coupled recursive estimation for online interactive perception of articulated objects. The International Journal of Robotics Research OnlineFirst:1-33, 2019.

Download Bibtex entry

Roberto Martín-Martín and Clemens Eppner and Oliver Brock. The RBO dataset of articulated objects and interactions. The International Journal of Robotics Research 39(9):1013-1019, 2019.

Download Bibtex entry

Clemens Eppner and Roberto Roberto Martín-Martín and Oliver Brock. Physics-Based Selection of Informative Actions for Interactive Perception. Proceedings of the IEEE International Conference on Robotics and Automation, pp. 7427-7432, 2018.

Download Bibtex entry

Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, Gaurav S. Sukhatme. Interactive Perception: Leveraging Action in Perception and Perception in Action. IEEE Transactions on Robotics 33(6):1273-1291, 2017.

Download Bibtex entry

Manuel Baum and Matthew Bernstein and Roberto Martín-Martín and Sebastian Höfer and Johannes Kulick and Marc Toussaint and Alex Kacelnik and Oliver Brock. Opening a Lockbox through Physical Exploration. Proceedings of the IEEE International Conference on Humanoid Robots (Humanoids), 2017.

Download Bibtex entry

Zusatzinformationen / Extras

Quick Access:

Schnellnavigation zur Seite über Nummerneingabe

Auxiliary Functions