UC Davis

Future long duration exploration missions (LDEMs) conducted by NASA will have an increased need for crew autonomy during routine and emergency procedures due to the increased distance from Earth causing time delays in communications. Presently, ISS in-space tasks are completed by astronauts using simple text-based procedures supplemented with real-time communication between the crewmembers and mission control personnel. As LDEMs require increased crew autonomy, more information must be stored on-board such that it can be accessed by crewmembers in a timely and context-appropriate manner during procedure execution. Emergent technologies in multimodal interactions such as Internet-Of-Things (IoT) sensors and enhanced visual displays are likely to play essential roles in safe crew-autonomous procedure execution. With this in mind, two studies were conducted: a study in NASA HERA to test enhanced multimodal procedures in a spacecraft analog environment, and a Davis study to determine how individual multimodal enhancements affect task performance. The goal was to determine how subjects’ task performance on a complicated manual repair task differed between enhanced procedures and traditional unimodal PDF procedures. An Enhanced Procedure Viewer system was developed that provided a variety of procedural enhancements: step navigation, enhanced visuals, real-time sensor feedback, and laser guidance. Results concluded that different enhancements in the multimodal procedure could decrease task completion time, increase task completion accuracy, decrease subjects’ perceived workload, and increase the level of trust subjects had in the procedure system.