UC San Diego

The development of a general-purpose robot capable of performing diverse tasks in home and office environments remains a central challenge in robotics. While breakthroughs in foundation models for language and vision have demonstrated the potential of scaling data and computation, robotics faces unique hurdles due to its reliance on physical-world data, such as 3D geometry, dynamics, and spatiotemporal information. These challenges create a significant data bottleneck, limiting progress toward general-purpose robots. This dissertation advocates for a data-centric approach to robotics, focusing on three core components: 1) building environments for scalable data collection, 2) developing efficient methods for collecting diverse robotic demonstrations, and 3) leveraging this data to learn robust and generalizable robotic policies.

To address these challenges, this dissertation introduces ManiSkill, a large-scale simulation benchmark designed for learning and evaluating diverse robotic manipulation skills. ManiSkill features extensive 3D assets, diverse tasks, and large-scale demonstration datasets, providing a robust environment for data-centric research. For scalable demonstration collection, this work proposes DrS, a method for learning reusable dense rewards from sparse rewards and demonstrations, enabling efficient reinforcement learning for short-horizon tasks. Additionally, it presents TR2, a framework that generates demonstrations for long-horizon tasks by translating abstract trajectories into executable robot actions. Finally, this dissertation introduces Policy Decorator, a model-agnostic method for refining imitation learning-based policies through controlled online interactions, significantly improving performance while preserving smooth motion.

By integrating these contributions, this dissertation establishes a comprehensive framework for advancing data-centric robotics, addressing critical bottlenecks in data collection and policy learning, and paving the way toward general-purpose robotic systems.