UC San Diego

Ubiquitous acoustic sensors and actuators, i.e., microphones and loudspeakers, are among the most common components in consumer electronic devices. Traditionally, these components have been primarily used for sound-related tasks, including voice-user interfaces, sound playback, and sound event detection. However, with the growing demand for consumer electronics to deliver more intelligent, cost-effective, human-centric, and trustworthy ambient intelligence while reducing costs, energy consumption, and computational overhead, there is an increasing need to unlock the full potential of these components for cross-modality sensing applications.

In this dissertation, I explore the repurposing of acoustic components for cross-modality sensing by employing advanced signal processing techniques and deep learning models. I develop end-to-end systems that integrate hardware design, sensor placement optimization, advanced signal processing, deep neural network architectures, and system-level optimizations. By leveraging a comprehensive understanding of the strengths and limitations of acoustic sensors and actuators, this dissertation uncovers novel functionalities in ubiquitous acoustic devices, including speech privacy protection, speech enhancement, and daily life and health monitoring.