Towards Immersive Skill Training for First Responders with Biosensor-based Assessment of Situation Awareness

First responders engage highly stressful situations at the emergency site that may induce stress, fear, panic and a collapse of clear thinking (Putnam, 1995). However, their physiological and cognitive readiness is of highest importance to enable appropriate decision making (Frye & Wearing, 2014). A major aspect for keeping control of the situation is that first responders should always maintain situation awareness. Training of the first responders’ routines improves resilience towards stressors in severe hazard conditions. Advanced training of first responders, in particular, the use of VR environments for the training of situation awareness for stressresilient decisionmaking behavior is one of the upcoming challenges of the near future.Methods A Virtual Reality environment was developed with a typical firefighter scenario to train and to evaluate the situation awareness (SA) of squad leaders. The operator has two major tasks, (i) searching for pocket of embers on the ground by using an infrared camera and whenever detected to call for a team member to work on it, (ii) to survey her team of 5 first responders for their visible presence as well as for their health condition and consequently to react in proper time instants whenever team members would be injured or disappear. The movement of the team members within a predefined skilloriented territory is activated by an AIdriven method. A wearable multisensorybased noninvasive measurement suite is mounted on the operator including VR inbuilt eyetracking, biosensors for cardiovascular, electrodermal and temperature data capture, to report by means of digital human factors analytics. From the sensor data stream we intend to deduce psychological constructs, such as, situation awareness, physiological strain, cognitiveemotional stress, and fatigue. A pilot study is planned with N=20 students, with pre and poststudy cognitive tests, such as, determination test (Schuhfried, 1987) and Psychomotor Vigilance Test (PVT; Dinges & Powell, 1985), as well as the Situation Awareness Rating Technique (SART; Taylor, 1990). From the SART we will research for to situation awarenessspecific digital biomarkers by means of correlation analysis. This multitasking configuration will enable to measure (a) executive functions, such as, cognitive flexibility, and (b) level 1, 2 and 3 SA by gazedriven virtual events, such as, viewing first responder avatars. Results: We will describe the system architecture as well as the features of the VRbased skill evaluation system (VR with eyetracking, biosensors, treadmill, dashboard). Test users will move on a treadmill that triggers the VRbased experience accordingly, by means of a Cyberith Visualizer platform. Videos that were captured from the use of the platform and application of biosensors with eye tracking will demonstrate the usefulness for the evaluation of SA. We will report from a first usability study with feedback from friendly users. At the conference, we will be able to report on first results of the pilot study.Conclusion: Advanced training of first responders and emergency staff with typical operation scenarios is one of the upcoming challenges of the near future. First results motivate the use of VR environments for the training of situation aware and stressresilient decisionmaking behavior of firefighters.