The “Auf der Leiten” vineyard in Klagenfurt am Wörthersee serves as the demonstration site. The project partners are ARTI Robots GmbH and Alpenspirit GmbH. According to ARTI, robots can assist with viticulture, for example, by spraying the vines in a targeted manner, mowing, monitoring the vines or helping with the harvest. These tasks in the vineyards place high demands on autonomous navigation: robots must travel along specified rows and paths, whilst vegetation, ground conditions and visibility vary significantly throughout the year. As a result, maps and real-time sensor data may differ from one another. This makes it difficult to localise the robot and can lead to poor navigation decisions.
utonomous mobile robots must be able to navigate reliably outdoors even when their environment is constantly changing. Plants grow, ground conditions change and light conditions vary. In the FFG project SEASON-Twin, researchers at JOANNEUM RESEARCH are therefore developing a digital twin of the vineyard and investigating methods to account for seasonal and environmental changes and uncertainties in navigation.
Through the vineyard with a LiDAR rucksack and a 360-degree camera
Markus Tauchhammer was responsible for developing the digital twin and the team from the Digital Twin Lab have already been out in the field on several occasions. Portable LiDAR backpack systems and 360-degree panoramic imaging were used. “LiDAR stands for ‘Light Detection and Ranging’. It measures distances to objects using laser pulses. Millions of individual measurement points are used to create three-dimensional ‘point clouds’, which reproduce the geometry of the surroundings in detail,” explains Tauchhammer.
The measurement campaigns carried out to date have recorded different vegetation and environmental conditions. Further surveys, including a winter survey, are being planned. This will gradually result in a multi-seasonal dataset that can be used to compare and quantitatively describe changes in the environment over the course of the year.
“To create a robust digital twin, it is not enough to capture an environment once with the greatest possible geometric accuracy. It is crucial to map its changes over time in a structured way. We want to use the 3D data to determine which features remain stable, which change seasonally, and how these differences can be utilised for simulation and navigation,” Tauchhammer continued.
From point clouds to navigation systems for robots
The captured 3D data is then processed semantically. The point clouds are segmented, classified and annotated. This enables different areas and objects in the environment to be not only captured geometrically, but also described functionally for robot navigation. The resulting maps allow for an assessment of the different areas. The assessment is based on how well – or with what effort or risk – a robot can navigate the terrain.
The digital twin is therefore intended to be far more than a static 3D model. The aim is to create a virtual environment capable of simulation, in which spatial structures, seasonal changes and uncertainties can all be taken into account simultaneously.
Uncertainties become part of navigation
Another key focus of SEASON-Twin is uncertainty modelling, i.e. the modelling and quantification of uncertainties. This involves investigating the extent to which environmental conditions, map information or sensor data can vary, and how these uncertainties should be taken into account during navigation.
“Autonomous outdoor navigation is particularly challenging because a robot never operates in a fully known and unchanging environment. Our aim is therefore not to deal with uncertainties only once they become a problem, but to systematically factor in their effects right from the navigation stage. This should make localisation and path planning more robust in the face of seasonal and environmental variations,” says Benjamin Breiling, project manager at JOANNEUM RESEARCH ROBOTICS.
Using the digital twin as a basis, test scenarios are also being developed to enable the targeted investigation of seasonal influences. The technologies employed include generative AI, procedural 3D scene generation and model-based modifications to virtual environments. This makes it possible to systematically generate and test different environmental conditions without first having to recreate each situation in reality.
Real-world navigation tests in viticulture
JOANNEUM RESEARCH is coordinating the SEASON-Twin project. In terms of content, the DIGITAL and ROBOTICS institutes are collaborating on this project. The Digital Twin Lab at DIGITAL is creating a digital twin of the vineyard and developing methods to analyse, quantify and visualise environmental variability. Simulation and test scenarios are then developed on this basis.
Among other things, JOANNEUM RESEARCH ROBOTICS is investigating methods for simulation, uncertainty modelling, and risk and performance assessment. Furthermore, the digital twin and the methods developed are being integrated into an existing autonomous navigation system from ARTI and validated in real-world navigation tests.
Viticulture serves as a challenging use case in this context. However, the methods developed are not limited to this field. The findings may also be relevant for autonomous systems in agriculture, forestry and environmental monitoring. In the long term, more robust navigation methods can help to reduce downtime and maintenance requirements for mobile robotic systems.
