Their vision is flexible production in which people tell or show robots what to do. Intuitive communication between man and machine should make production flexible and dynamic. Both are extremely important for domestic companies in order to survive on the global market. How can this work?
Gallien: The latest developments in the field of language-based generative AI are the game changers in collaboration with robots. Text-based image generators such as DALL-E2, Midjourney or Stable Diffusion interpret text input and generate the most likely matching image. This is made possible by so-called Visual Language Foundation Models (VLFMs), which give images a meaning and can process this information. The special feature here is the incredibly large amount of data with which these models are trained. The data set for the CLIP (Contrastive Language Image Pretraining) model published by OpenAI, for example, comprised more than 400 million text-image pairs. Some of these VLFMs are available to the general public and are increasingly being used in machine vision algorithms. This means the following: If you feed a robot with this incredible number of image-text pairs, it is able, for example, to recognise a situation using a camera and interpret it in real time. In short, the robot immediately understands the scene in the room. It knows what a glass, a table or a person is and reacts to visual commands using the camera. This means that programming is skipped for simple work steps. This is collaborative cognitive robotics at the next level.
What does it take to integrate modern robot systems into dynamic production processes?
Gallien: The challenge is great, as dynamic production processes require a high degree of flexibility and adaptability. A key aspect of this is perception, interpretation and decision-making in real time. To this end, autonomous robot systems process a wide range of sensor data from cameras, 3D scanners and radar sensors, for example. Cognitive robotics refers to robot systems that primarily utilise machine vision methods to process this data and arrive at an appropriate interpretation of the scene. The semantic reference is created by the Visual Language Foundation Models with which industrial robots can be equipped. This is a revolution!
What role would this synergetic collaboration between man and machine play in modern production processes?
Gallien: The advantages are obvious: the extensive basic semantic knowledge that robots could be born with means that so-called zero-shot models can be designed. These are able to perform tasks for which they have not been explicitly trained. This is extremely important for modern production, as there is no longer time for lengthy programming work. In future, robots will be able to find their way around dynamic environments because they can perceive and interpret their surroundings in real time. Instructions can be given verbally or by hand signals, which enables particularly intuitive and natural control of robots in complex production environments. These features significantly expand the applicability of assistive production robotics.
Can you give us an example?
Gallien: With the AI turbo, you can talk to a robot and tell it to lift an object, for example. That is all. The robot will do what you tell it to do. In the conventional way, object detection models have to be pre-trained at great expense. The major disadvantage of this is that all object classes must be represented in a statistically relevant way in the training data set. Consequently, these methods fail to adapt to new, previously unknown environments. Furthermore, annotating the data sets is tedious and time-consuming. The difference in the time component is obvious.
What is the situation in Europe, in Austria?
Gallien: Digitalisation and, in particular, the rapid developments in the field of artificial intelligence are affecting all areas of life and leading to a profound change in the manufacturing landscape. In Europe too, of course. But Europe must endeavour to keep pace with the USA and China. Enormous sums are being invested in AI there. We can see that industrial companies based in Austria, for example, are obtaining expertise from the USA. This is a shame, but it also encourages us in our endeavours to establish collaborative cognitive robotics in Austria. At JOANNEUM RESEARCH ROBOTICS, we specialise in methods of collaborative robotics and flexible production. The logical consequence is now to take our existing expertise to the next level and expand it using artificial intelligence. In collaboration with the DIGITAL (Institute for Digital Technologies) and MATERIALS (Institute for Sensors, Photonics and Manufacturing Technologies) institutes and the existing infrastructure, we are ideally positioned for the next level of robotics.
Interview: Elke Zenz
