UC Irvine

In recent years, gesture recognition has become an important tool for enabling interactionwith augmented and mixed reality devices. This thesis explores the deployment of a gesture recognition application on the Microsoft Hololens 2 using a Convolutional Neural Network (CNN) with multiple early exits to improve efficiency in real-time scenarios. The use of early exit mechanisms at different layers of the network helps reduce energy consumption and processing time while maintaining reasonable accuracy. However, the average frames per second (FPS) could be further improved for smoother interaction. The project also examines sensor integration, studying how multiple Hololens 2 sensors work together. Lastly, a multi-user application is developed to enable real-time interaction among users in a shared augmented reality environment. These efforts aim to improve the usability and practicality of mixed reality systems for various applications.