3D vision and 3D visualisation in Mars exploration

Together with our partner VRVis - Centre for Virtual Reality and Visualisation - we have created a unique framework for processing and visualising planetary surface data (Mars, Moon, Asteroids). (Mars, Moon, Asteroids). The PRoViP" and PRo3D" elements interlock seamlessly for real-time fusion and visualisation of 3D data at all scales from millimetres to thousands of kilometres.

Together with our partner VRVis and the scientific interpretation by the Austrian Academy of Sciences, we are producing 3D analyses of camera images from the Mars 2020 Perseverance mission for the NASA Mars 2020 mission.

For the ExoMars 2030 mission, we are providing 3D processing for the three stereo camera systems PanCam, NavCam and LocCam.

Integrated into the scientific teams of the three rover missions Mars 2020 (Perseverance), MSL (Curiosity) and ExoMars (Rosalind Franklin), we contribute to top-class publications in geology, climatology, hydrology and soil physics of our outer neighbour planet with three-dimensional evaluations and visualisations. Our solutions include in particular mechanisms of stereo reconstruction, 3D data fusion, camera calibration, and data processing for real-time visualisation and analysis. In particular, our solutions include mechanisms for stereo reconstruction, 3D data fusion, camera calibration and data preparation for real-time visualisation and analysis.

Our expert Gerhard Paar is Mastcam-Z Co-Investigator (Co-I), PanCam Lead Co-I for 3D, ExoMars CLUPI (Close-Up Imager) Co-I and MSL Collaborator and DIGITAL scientist Piluca Caballois a member of the Mars 2020 science team.

Reference projects

• Mars-3D (FFG/ASAP 892662) überbrückt in Zusammenarbeit mit VRVis und ÖAW die Vorbereitungen für den 3D-Vision-Beitrag zur derzeit eingestellten ExoMars-Mission mit 3D-Vision-Betrieb von PanCam-Simulationen. Der Mastcam-Z 3D-Vision- und Impaktforschungs-Beitrag erfolgt im selben Zeitraum mit maximaler Synergie. Hauptziel ist die Publikation wissenschaftlicher Ergebnisse aus Geologie, Morphologie, Aeologie und Impaktforschung, die sich speziell aus den Möglichkeiten der 3D Vision und Visualisierung eröffnen, gemeinsam mit den jeweiligen Instrumententeams.
• Mars-4D (FFG/ASAP 911911) wird sich mit der Operations-Unterstützung und Publikationsaktivitäten basierend auf 3D-Vision in der Mars 2020 Mission im Zeitraum Oktober 2024 bis Juni 2026 befassen, hauptsächlich gekoppelt an Produkte des Mastcam-Z Stereokamera-Instruments. Mars-4D wird Mastcam-Z 3D-Vision-basierte wissenschaftliche Mars-Publikationsaktivitäten auf hohem Niveau in den Bereichen Geologie, Morphologie, Hydrologie und Äologie manifestieren, die vor allem von den einzigartigen Möglichkeiten profitieren, die die österreichischen 3D-Vision- und Visualisierungslösungen und -dienste bieten können.
• MSL-3D MSL-3D will enable researchers on the MSL (Curiosity) rover mission to accurately reconstruct and analyse rock outcrops that are important for understanding the geology of Gale Crater and for understanding Mars' climate history and potential habitability. Over the coming months, researchers will use MSL-3D to reconstruct the surface conditions and processes that led to the deposition of the Gediz-Vallis ridge and sulphate inclusions, and to measure outcrops to assess the rockfall risk associated with the traverses and rover mobility in near-real time. It will also explore several other regions of Gale Crater that have not been studied in detail, particularly in 3D. This will support the scientific agenda of two British scientists (Dr Davis and Dr Banham from Imperial College London). Mastcam stereo images from Mars Rover Curiosity will be processed to study surface flow events and palaeoenvironmental reconstruction along the Curiosity traverse, with occasional presentations and recording requests for the Mastcam instrument dedicated to the research of Dr Davis and Dr Banham, in conjunction with Gerhard Paar of DIGITAL as part of the MSL science team. 3D data fusion with the terrain models from HiRISE and CTX satellite imagery will allow characterisation of features from the close-up (boulder size, grain size) to the kilometre scale (slope, also as m-scale statistics) from both these satellite and Mastcam 3D data products. Further unique strengths of the Austrian JR/VRVis processing and visualisation expertise will improve the geometric calibration of the Mastcam and compile the mission history in a geological and public relations sense through annotated video documentation with unprecedented accuracy, using multi-scale 3D models from the MSL data products published via PDS (Planetary Data System) and "long-baseline" stereo configurations.
• Rosalind-3D (FFG/ASAP 911919) deals with the preparation of scientific operations for the ExoMars Rover (Rosalind Franklin) mission (landing on Mars in 2030) in the period May 2024 to December 2026, based on 3D vision and visualisation for the PanCam scientific stereo camera system. It will include adaptation of data interfaces, processing and visualisation to the latest scientific requirements, training activities and, in particular, participation in biannual tests in the environment of the Rover Operations Control Centre at ALTEC in Turin. This will be supported by updates to the calibration of the camera system, participation in preparatory publications documenting the tests, and public information activities. The Rosalind 3D activity proposed here will lay the foundations for high-level scientific Mars research publication activities based on Rosalind Franklin 3D vision in the fields of geology, morphology, hydrology and etiology, which will particularly benefit from the unique opportunities that Austrian 3D vision and visualisation solutions and services can offer after the rover lands in 2030.