In order to communicate with humans, robots can have warning lights to convey the current operating state. Furthermore, machine control panels (so-called tech pendants in robotics) have to take instruc- tions from the user or have to assemble work programs. Some robots have control knobs placed direc- tly on the robot arm and/or an operation panel as a touch screen placed in the head area of the robot. The interaction does not take place directly with the robot but with additional, external interfaces. This makes the user turn away from the actual object and the user can only indirectly control and perceive the robot. Warning lights are only located at a central point of the object and can not be perceived from any position by users. This is especially true with small distances between man and machine in direct physical collaboration.


The novel solution uses the complete robot surface to interact with the user, both as an input interface and as an interactive display. For this purpose, cell-shaped elements are distributed over the robot surface, which serve as a carrier platform for various tactile sensors (e.g., touch, distance, vibration, movement) and for multicolor pixels or pixel arrays (e.g., LEDs, OLEDS, LCDs) for visualization. Our technical solution uses an automatic 3D reconstruction of the surface and an automatic estimation of the robot kinematics to register the surface to this model. The 3D surface reconstruction can now be used for the automatic display of pixel-based animations or images on the robot surface. This allows for the projection of individual control buttons or completely interactive menus onto any place on the robot surface. Furthermore, it is possible to visualize a large number of context-based warnings and information. Ultimately, the invention improves not only the man-machine interaction, but also allows non-expert communication between robots and humans. Special symbols also allow the mutual regis- tration of the coordinate systems of several machines by using the surface as active optical markers.


  • Intuitive human/robot collaboration.
  • Easy man/machine communication
  • Context-based communication
  • Automatic robot/robot recognition



Dr. Tobias Steinel
Reference Number:
+49 (0) 89 5480177 - 39