Observation of acoustic scatter from fish at multiple views can be used to estimate their size and orientation in situ.

Marine protected areas (MPAs) have been established worldwide to protect coastal ecosystems and the species inhabiting them. However, it is difficult to quantify whether these areas are adequately protecting the targeted species. Current monitoring methods, such as diver surveys, allow fish species to be identified in situ, but are known to alter fish presence and behavior. Other methods, such as acoustic telemetry, are relatively invasive, requiring the implantation of a transmitter tag into the fish. Additionally, both these approaches are laborious and expensive, relying on good weather and a talented pool of fisherman and divers. Methods that are non-invasive, such as passive acoustics, offer good spatial and temporal coverage, but ascribing specific calls or sounds to the species creating them is difficult, particularly for fishes. Camera deployments allow for in situ observations of behavior, diversity and frequency of occurrence of a wide variety of animals but are often hindered by low-light and battery limitations. Here, I developed passive acoustic and optical imaging tools to study sound-producing fish that allow improved performance over contemporary methods. These tools were used to study chorusing fish in protected kelp forests along the southern California coast. First, an autonomous Wave Glider was equipped with a passive acoustic recorder to map the distribution of five soniferous fish spawning aggregations. The fish choruses started near sunset and ended before sunrise, and were almost exclusively recorded offshore of kelp forests. Second, a low-cost underwater optical imaging system that utilizes a consumer-grade camera to capture high-quality images in low-light aquatic habitats without artificial lighting was designed and developed. The system was used to captured >1,500 images per day over 14 days, which revealed biologically important behaviors as well as daily patterns of presence/absence. Lastly, an underwater controlled source of known position was used to improve an acoustic localization algorithm to track fish to a resolution of a few meters. The fish remained outside of the MPA while vocalizing. This work demonstrates the promise of these tools to non-invasively monitor animal behavior, biodiversity and frequency of occurrence in MPAs as well as other nearshore areas.

The ocean is a complex and dynamic environment with physical and biological processes that span across all scales. Though scientific studies in laboratory setting can provide insights into these complicated processes, it is difficult to fully replicate the natural environment in the lab. As a result, studies of processes in the natural environment, or in situ studies, can help bridge information from laboratory experiments to their actual implication in the nature. Recent technological advances have enabled the ocean to become more accessible for scientific research and exploration. Particularly, breakthroughs in imaging technology have enabled new imaging techniques to be accessible at smaller form factors and lower costs, such that they are suitable for oceanic deployments. Here I developed four underwater imaging systems and demonstrated their capability as tools for studying both physical and biological processes in the ocean. The first system is a particle imaging system that can be used to resolve average flow velocity by applying a modified particle image velocimetry algorithm. The formulated algorithm allows the system to be constructed with low-cost components that permits the whole system to be more affordable. The second system is an underwater microscope for zooplankton study that is based on open-sourced hardware and software. The system was used to investigate the emergence pattern and the spectral response of demersal zooplankton in the Kaneohe Bay, Hawaii. The third system is a stereoscopic imaging system that implements a tilted lens approach to increase the shared field of view between the two cameras. The method allows the stereoscopic system to perform in situ trait measurements of aquatic invertebrates. The system was deployed in Kendall-Frost saltmarsh, San Diego, to determine the relationship between the swimming speed and the body length of aquatic insects, Trichocorixa californica. Finally, the fourth system is an underwater microscope that has sufficient resolution and contrast perform label-free imaging of marine microorganisms. The instrument utilizes a laser-pulsed darkfield microscopy technique to image individual cells of marine microorganisms and to detect fluorescence signals from natural pigments. The system was used in an in situ observational study on biofilm formation on a man-made substrate.

Since scientists began to study the ocean, they have had to develop new observational methods. Modern marine biologists and ecologists have increasingly used imaging systems to help address their most persistent and vexing questions. These techniques have allowed researchers to take samples at higher spatial or temporal frequency than ever before. While yielding marvelous new insight into difficult to observe phenomena, image-based sampling creates a new data problem: there is too much of it. Without techniques to classify and analyze oceanographic images, much of the data sits idle.

Two instruments developed in the Jaffe Laboratory for Underwater Imaging are em- blematic of this trend. The Sub Sea Holodeck (SSH) is an immersive virtual aquarium devel- oped to study cephalopod camouflage in the laboratory. The Scripps Plankton Camera System (SPCS) is a pair of in situ microscopes built to observe undisturbed plankton populations over long periods of time. In this thesis, new techniques, grounded in current machine learning methodologies, are developed to speed the analysis of these information rich big data sets.

Data from the SSH are treated as textures and classified using texton dictionaries. In addition to being highly accurate, the texton-based method is entirely data driven – the metric for separating it is derived directly from the images. As a consequence, a new criterion for classifying cephalopods is proposed that explicitly treats samples that do not conform to the prescribed three class system.

Images from the SPCS and several other in situ plankton imaging systems are used to evaluate the performance of new deep learning methods. Several Convolutional Neural Net- works (CNNs) are trained from the data and tested against each other. The results underscore the startling representational power of CNNs and suggest that neural methods outperform other approaches for annotating plankton images.

Finally, an automated classifier is deployed on SPCS data to explore the dynamics of a host-parasite relationship. A CNN is used to build a high-resolution time series tracking fluc- tuations in Oithona similis and the parasite Paradinium sp. The subsequent analysis identifies a possible time scale of the parasite’s internal life stage and its effect on the overall O. similis population.

Many fundamental biological and physical processes taking place in benthic marine environments such as coral reefs, kelp forests, and sea grass beds occur at scales of a millimeter or less. For example, large reef ecosystems are cumulatively built by many coral colonies, these colonies are in turn composed of numerous individual coral polyps each on the order of one millimeter in diameter. The polyps feed on micro-scale plankton and get photosynthetic energy from single-celled symbiotic algae. Furthermore, micro-scale fluid motions and mixing surrounding coral polyps impact their ability to exchange critical dissolved solutes such as oxygen and nutrients with the surrounding water. Natural ocean environments are also complex and dynamic systems that are difficult or impossible to fully replicate in the lab. A distinct needthus exists for direct observations of important marine processes in situ. This thesis describes the development of novel underwater microscopic imaging and velocity measurement techniques for studying micro-scale biological and physical processes at the seafloor in the natural environment. These techniques are then applied specifically to study reef-building corals.

First, I describe the development and application of the Benthic Underwater Microscope (BUM). This diver-operated instrument performs non-invasive imaging of subjects on the seafloor in situ at nearly micrometer resolution. The instrument’s optical system combines a long working distance microscopic objective lens, a flexible electronically tunable lens, and focused reflectance illumination. The BUM was deployed in the ocean to study coordinated coral polyp behaviors, interspecies coral polyp competition, and algal colonization of bleached coral colonies.

Second, I modify the BUM to measure two-dimensional fluid velocities at micro-scales using micro-particle tracking velocimetry (μPTV). This is achieved by incorporating dark field illumination, rapid image pair acquisition, active particle seeding, and application of particle tracking algorithms. The system was used to measure fluid velocity fields at sub-millimeter scales around coral polyps in the ocean, and a viscous boundary layer was observed directly at the coral’s surface.

Finally, the underwater μPTV instrument was used to study spatiotemporal dynamics of fluid motion around individual coral polyps in situ. We measure shear stress at the coral’s surface and show waves periodically modulating the near-surface flow environment. The tools developed here provide a means to bring the lab into the ocean, offering a new window into important marine processes.

To understand ocean health, it is crucial to carefully monitor and analyze marine plankton – the microorganisms that form the base of the marine food web and are responsible for the uptake of atmospheric carbon. With the recent development of in situ microscopes, that collect images of these organisms in vast quantities, the use of deep learning methods to taxonomically identify them has come to the forefront. Given this data, two questions arise: 1) How well do deep learning methods such as Convolutional Neural Networks (CNNs) identify these marine organisms using data from in situ microscopes? 2) How well do CNN derived estimates of abundance agree with established net- and bottle-based sampling?

Using images collected by the in situ Scripps Plankton Camera (SPC) system, we trained a CNN to recognize 10 species of phytoplankton that are identified as associated with Harmful Algal Blooms (HABs). The success of the CNNs is characterized using standard evaluation metrics. To compare abundance estimates, we fit a linear model between the number of organisms of each species counted in a known volume in the lab with the number of organisms collected by the in situ microscope sampling at the same time.

The CNNs evaluated on 26 independent natural samples collected at Scripps Pier achieved an averaged accuracy of 92%, with 7 of 10 target categories above 85%. The linear fit between lab and in situ counts of several key HAB species suggests in the case of these dinoflagellates there is good correspondence between the two methods. The linear relationship derived for other organisms that were not as abundant is not as conclusive as well as the failure of the SPC systems to successfully detect the diatom chain, Pseudo-nitzschia.

Given the excellent correlation between lab counts and in situ microscope counts on key species, the methodology proposed here provides a way to estimate an equivalent volume in which the employed microscope can identify in focus organisms and obtain reasonably confident estimates of abundance. Given the ease of collection and success of the method, we are hopeful that future systems can automatically monitor HAB formers and other plankton.