UCLA

Intelligence, one of the most profound phenomena on Earth, has evolved over 600 million years, transforming from simple neural systems into human cognition capable of unraveling universal mysteries and creating silicon-based intelligence. This evolutionary process, with its inherent drive towards increasing complexity, seemingly defies thermodynamic principles, suggesting the existence of self-evolving mechanisms in life. Recreating such mechanisms within artificial systems is a crucial milestone on the path toward Artificial General Intelligence (AGI).

Automated Machine Learning (AutoML) represents a significant step in this direction. By enabling AI systems to optimize their own design processes, AutoML reformulates machine learning pipeline construction as a search problem, automating the selection of architectures, optimizers, hyperparameters, and even reasoning paths from an expansive search space. Despite its current limitations, AutoML has already demonstrated remarkable success in advancing machine learning across diverse applications.

This thesis delves into the synergistic interplay between AutoML and large foundation models, particularly the recent breakthroughs in large-scale generative models like large language models (LLMs) and diffusion models. These models exhibit emergent behaviors, highlighting machine learning systems as complex entities where scaling produces unpredictable and potentially transformative capabilities.We emphasize the application of AutoML in automating the design of training and inference processes, crucial for the continued advancement of these models.

Ultimately, we aspire that this research contributes a meaningful step towards the ambitious pursuit of Artificial General Intelligence (AGI).