UC Davis

Deep learning has revolutionized problem-solving by leveraging the power of deep neural networks. AlexNet and ImageNet marked a significant milestone, demonstrating the immense potential of scaling both data and computational resources to enhance model performance. This trend is particularly evident in neural language processing, where scaling Transformers has driven the development of the large language models (LLMs). Ultimately, data and computational power form the core foundations of deep learning’s success.

As models grow more complex, their demand for computational resources increases, leading to higher costs and energy consumption. These rising expenses are progressively limiting machine learning research to large industry labs. For instance, while many recent studies are open-sourced, the cost of reproducing them restricts AI research for most academic institutions. To address this, developing affordable, efficient AI models that are accessible to academic labs is a crucial step toward democratizing AI research. Achieving this will require optimizing models for both data efficiency and computational resource usage.

Moreover, Accessibility, affordability, and trustworthiness are crucial factors in the development of AI models. However, many deep learning models are designed for high-end, expensive hardware, limiting their broader adoption. Additionally, reliance on centralized computing raises significant privacy concerns, as user data must be transferred to remote servers for processing, diminishes trust in AI systems. Edge computing offers a promising alternative by processing data locally on devices, making it more cost-effective, energy-efficient, and enhancing both accessibility and trust. Ideally, AI models should be efficient and optimized for edge devices, reducing dependency on centralized systems.

These motivations inspire me to explore new approaches for enhancing the efficiency of deep learning models. My research focuses on various aspects of efficiency, including data efficiency, parameter efficiency, training compute efficiency, and inference efficiency. By prioritizing efficiency, I aim to bridge the gap between cutting-edge research and deployment of these models in real-world applications, while also fostering diversity in AI development. Ultimately, my goal is to make AI inclusive and accessible to all. I believe that meaningful progress builds on the contributions of many past works, making it crucial to expand access to AI for a broader range of researchers and developers, thereby accelerating advancements in the field.