Some modes of locomotion, such as legged walking and flapping flight, are inherently unsteady due to the cyclic interaction of the subject's limbs with a changing and largely unmodelled environment. Bio-inspired robots that locomote this way, present a unique chal- lenge to indoor navigation because the sensors used for exteroception record this unsteadiness in their readings. A legged crawler's unsteady dynamics are explored with an emphasis on how these affect optical flow estimation, which mediates navigation. A triaxial gyroscope is sampled concurrently with the on-board video and is used to disambiguate the motion estimates through image derotation. The optical flow algorithm's gains are further tuned using policy gradient reinforcement learning so as to improve motion estimation for specific unsteady regimes. This approach is demonstrated in an obstacle avoidance scenario.

Bio-inspired millirobots exhibit unique locomotion modalities that allow them to traverse complex, unstructured terrain that traditional robots cannot.

This makes them potentially useful in urban search and rescue, structure inspection, environmental monitoring, and surveillance, all of which require some level of situational awareness.

The low-cost, lightweight design and difficult-to-model dynamics of these robots also create unique challenges when applying state-of-the-art Simultaneous Localization and Mapping (SLAM) approaches usually used to gain this awareness.

In this thesis, we develop a collection of estimation and control techniques that addresses these challenges, allowing teams of millirobots to localize within and map complex, unstructured environments.

The analysis covers several facets of the problem, including low-cost millirobot team design, motion modeling, cooperative state estimation, and mapping.

We first show the utility of disposable low-cost robots in hazardous environments by using teams of “picket” and “observer” robots for exploration.

Next, we explore a data-driven motion modeling approach to approximate the non-linear stochastic dynamics of legged millirobots.

Then we develop an inter-robot pose estimation technique using monocular vision and active markers that can operate in visually feature-poor environments, and can scale to teams of computationally constrained robots.

We demonstrate this technique first on autonomous underwater vehicles, and then extend it to a team of ground robots cooperatively navigating three-dimensional terrain.

Finally, we explore a simple scanning laser technique that can leverage cooperating robots with cameras to map an environment.

Deep learning has shown promising results in robotics, but we are still far from having intelligent systems that can operate in the unstructured settings of the real world, where disturbances, variations, and unobserved factors lead to a dynamic environment. The premise of the work in this thesis is that model-based deep RL provides an efficient and effective framework for making sense of the world, thus allowing for reasoning and adaptation capabilities that are necessary for successful operation in the dynamic settings of the world.

We first build up a model-based deep RL framework and demonstrate that it can indeed allow for efficient skill acquisition, as well as the ability to repurpose models to solve a variety of tasks. We then scale up these approaches to enable locomotion with a 6-DoF legged robot on varying terrains in the real world, as well as dexterous manipulation with a 24-DoF anthropomorphic hand in the real world. Next, we focus on the inevitable mismatch between an agent's training conditions and the test conditions in which it may actually be deployed, thus illuminating the need for adaptive systems. Inspired by the ability of humans and animals to adapt quickly in the face of unexpected changes, we present a meta-learning algorithm within this model-based RL framework to enable online adaptation of large, high-capacity models using only small amounts of data from the new task. We demonstrate these fast adaptation capabilities in both simulation and the real-world, with experiments such as a 6-legged robot adapting online to an unexpected payload or suddenly losing a leg. We then further extend the capabilities of our robotic systems by enabling the agents to reason directly from raw image observations. Bridging the benefits of representation learning techniques with the adaptation capabilities of meta-RL, we present a unified framework for effective meta-RL from images. With robotic arms in the real world that learn peg insertion and ethernet cable insertion to varying targets, we show the fast acquisition of new skills, directly from raw image observations in the real world. Finally, we conclude by discussing the key limitations of our existing approaches and present promising directions for future work in the area of model-based deep RL for robotic systems.

Flapping-winged flight is very complex, and it is difficult to efficiently model the unsteady airflow and nonlinear dynamics for online control. While steady state flight is well understood, transitions between flight regimes are not readily modeled or controlled. Maneuverability in non-equilibrium flight, which birds and insects readily exhibit in nature, is necessary to operate in the types of cluttered environments that small-scale flapping-winged robots are best suited for. The advantages of flapping wings over quadrotors and fixed-wing fliers are realized in the ability to transition from forward flight to hover to gliding. Flight in the transitions between these regimes necessitates the development of novel modeling techniques and online control techniques to accurately complete these types of maneuvers.

In this thesis, methods for modeling and controlling the transitions between takeoff and diving maneuvers are developed for a flapping-winged micro aerial vehicle (MAV), the H2Bird. To transition into takeoff and steady state flight, a cooperative launching system is developed for the H2Bird by carrying it on the back of a 32 gram hexapedal millirobot, the VelociRoACH. The necessary initial velocity and pitch angle are determined for take off using force data collected in a wind tunnel, and the VelociRoACH is used to reach these initial conditions for successful launch. The models for the diving maneuver are generated using an automatic piece-wise affine identification technique. The flight conditions during the maneuver are segmented into separate regions and least-squares is used to estimate affine linear models for each modeling region. These models are used to compute the reachability sets for the recovery conditions for safe diving, and linear quadratic regulator controllers are used to maintain stable conditions before and after the dive. The data-driven automatic modeling techniques and controller design processes can be extended to additional flight maneuvers.

Recent advances in the development of meso-scale legged robots have enabled high performance over a variety of terrains. However, there are still many environments where terrestrial legged robots are limited. The addition of aerodynamic capabilities provides a method of further extending the performance of small mobile robots towards all-terrain mobility.

This dissertation presents the development of DASH+Wings and BOLT, two small hybrid legged and winged robots. DASH+Wings is a six-legged, two-winged robot capable of wing-assisted terrestrial running and controlled aerial descent. BOLT is a two-legged, four-winged robot capable of high-speed terrestrial running and sustained flight. While the dynamics of legged locomotion have been extensively studied, the interaction between legs and flapping wings during terrestrial locomotion is poorly understood. WingSLIP, an extension to the canonical SLIP model for understanding wing-assisted terrestrial locomotion, is introduced. Analysis of the leg/wing phasing and leg stiffnesses elucidates the interaction between the legs and wings. The model suggests the presence of passively stable gaits for high-speed wing-assisted terrestrial running. The dynamics of wing-assisted terrestrial locomotion for a quasi-static and dynamic gait are examined using BOLT with the addition of an on-board accelerometer and rate gyroscope. The flapping wings enable BOLT to run at over two ms^-1 and take off in as little as one meter of space. In addition to flapping wings, the passive effects of aerodynamics on a high-speed legged robot (VelociRoACH) are also examined. Passive aerodynamic stabilization is shown to improve the performance and stability of the system. An examination of avian flight evolution is performed using DASH+Wings to examine the consequences of adding wings to a cursorial locomotor. We note that current support for the diverse theories of avian flight origins derive from limited fossil evidence, the adult behavior of extant flying birds, and developmental stages of already volant taxa. The addition of flapping wings increased the maximum horizontal running speed and the maximum terrestrial incline angle of ascent, along with decreasing the aerial descent angle. To better examine avian flight evolution, a scaled model of Archaeopteryx is developed, and the effects of flapping frequency and amplitude are examined through the use of wind tunnel measurements. The findings are discussed in the context of existing hypotheses for the origins of flapping flight in vertebrates.

A suspension comprised of angled fibers is proposed as a new means for achieving high strain, high stress, energy dense electrostatic actuators. Angled fiber arrays have low density and can be placed between the electrodes of a parallel plate or comb-drive type actuator to create a self-contained actuator sheet with low mass and volume. Angled fibers also have a Poisson's ratio of zero, allowing the use of robust, rigid electrodes, and they can be composed of stiff materials with low viscoelastic properties. This is in contrast to the alternative technology of dielectric elastomers that depend on unreliable compliant electrodes and highly viscoelastic dielectrics. Performance limits of an ideal nanometer-scale actuator, such as energy density, stress and strain, and efficiency are considered through theoretical modeling. A micrometer scale prototype is fabricated using a novel fiber peeling technique that easily produces high-aspect-ratio (1.8 μm radius, 66 μm long), angled microfibers. The microfibers are used as a suspension for a parallel plate actuator. The prototype actuator is characterized through static and dynamic tests, to reveal a maximum static strain of 3.4% at a static stress of 0.8 kPa (electric field of 13.9 V/μm), a fast unloaded step response of <2 ms, a Q of 12.9 and a power density of 12.8 W/kg when driving an inertial load in resonance at 845 Hz.

This thesis presents the design of autonomous flight control algorithms for a flapping-wing aerial robot with onboard sensing and computational resources. We use a 13 gram ornithopter with biologically inspired clap-and-fling mechanism. For autonomous flight control, we have developed 1.0 gram control electronics integrated with a microcontroller, inertial and visual sensors, communication electronics, and motor drivers. We have also developed a simplified aerodynamic model of ornithopter flight to reduce the order of the control system. With the aerodynamic model and the orientation estimation from on-board inertial sensors, we present flight control of an ornithopter capable of flying toward a target using onboard sensing and computational resources only. To this end, we have developed a dead-reckoning algorithm to recover from the temporary loss of the target which can occur with a visual sensor with a narrow field of view. With closed-loop height regulation of the ornithopter, we propose a method for identifying the discrepancy between the tethered flight force measurement and the free flight aerodynamic force. Lastly, we present a nondimensionalized analysis of a motor-driven flapping-wing system and experimentally demonstrate a resonant drive to reduce average battery power consumption for DC motor-driven flapping-wing robots.

This thesis explores milli- and meso-scale legged robot

design and fabrication with compliant mechanisms. Our approach

makes use of a process that integrates compliant flexure hinges

and rigid links to form parallel kinematic structures through the

folding of flat-fabricated sheets of articulated parts. Using

screw theory, we propose the formulation of an equivalent mechanism

compliance for a class of parallel mechanisms, and we use that

compliance to evaluate a scalar performance metric based on the

strain energy stored in a mechanism subjected to an arbitrary load.

Results from the model are supported by experimental measurements of a

representative mechanism. With the insight gained from the kinematic

mechanism design analysis, we propose and demonstrate compliant

designs for two six-legged robots comprising the robotic, autonomous,

crawling hexapod (RoACH) family of robots. RoACH is a two

degree of freedom, 2.4 gram, 3 cm long robot capable of untethered,

sustained, steerable locomotion. RoACH's successor, DynaRoach,

is 10 cm long, has one actuated degree of freedom and is capable of

running speeds of up to 1.4 m/s. DynaRoACH employs compliant legs

to help enable dynamic running and maneuvering and is three orders of

magnitude more efficient than its milli-scale predecessor. We experimentally

demonstrate the feasibility of a biologically-inspired approach to turning

control and dynamic maneuvering by adjusting leg stiffness. While the result

agrees qualitatively with predictions from existing reduced order models,

initial data suggest the full 3-dimensional dynamics play an important

role in six-legged turning.

This thesis explores the design of systems that can climb vertical surfaces with non-negligible dynamics in the sagittal plane. The development of a low-dimensional model addresses a lack of understanding of sagittal plane dynamics during climbing in the space of reduced-order dynamic models of legged systems. Using a construction derived from the well-known and well-studied Spring-Loaded Inverted Pendulum (SLIP), we propose a two-legged system with both torsional and linear compliance driven by a position-controlled rotational actuator. Two simple foot models are considered to explore their effect on the dynamics and stability of the system. Results of the model indicate the existence of passively stable gaits during climbing as well as during inverted running and also suggest mechanical tuning parameters for physical climbing systems. A robotic platform capable of producing dynamic climbing behaviors is introduced. A reduced profile, sprawled posture, and improved internal mechanics allow the CLASH platform to be adapted to different climbing substrates. A passive claw engagement mechanism is proposed and tested with simulated steps to verify the design. With these mechanisms, CLASH becomes the first robotic platform capable of climbing loose cloth and climbs vertically at 15cm/s or 1.5 body-lengths per second. When climbing ferromagnetic surfaces, the system is capable of climbing at 1.8 body-lengths per second. To climb smooth, hard surfaces, a foot with a passively aligning ankle with a tendon-loaded gecko-inspired adhesive is designed and tested using simulated steps. With these engagement mechanisms, the system is able to climb at 1 body-length per second on acrylic with a 70-degree incline. A simple foot-impact model is created to explain the robot's inability to climb faster or up steeper inclines due to the sagittal plane reaction forces created during rapid running.