Advances in high-throughput experimental technologies now allow for the probing of the fitness of up to millions of biological sequences. Parallel developments in machine learning (ML) techniques set the stage for a new class of ML-based computational approaches to biological sequence engineering. These approaches can largely be regarded as solving one of two related problems: forward and inverse problems. In a forward problem, the aim is to estimate a fitness function from high-throughput experimental data. In an inverse problem, the aim is to use a predictive model of fitness to design a sequence or library of sequences that satisfies a particular engineering goal. In this dissertation, we will discuss various aspects of both of these sets of problems, and propose solutions for certain scenarios.

First, we present a method for solving one instantiation of an inverse problem. This method, Conditioning by Adaptive Sampling (CbAS), is designed to optimize models of protein fitness when those models make biased predictions in regions of sequence space far from the set of training data. CbAS is an example of a broad class of optimization methods known as Estimation of Distribution Algorithms (EDAs). We show that many EDAs can be viewed as a form of Expectation-Maximization (EM), a popular inference algorithm in probabilistic ML.

Next, we explore the forward problem in more depth. It is an open problem to characterize how much data is needed for accurate estimation of fitness functions. There is a growing body of evidence demonstrating that empirical fitness functions display substantial sparsity when represented in terms of epistatic interactions. Moreover, the theory of Compressed Sensing provides scaling laws for the number of samples required to exactly recover a sparse function. Motivated by these results, we study the sparsity of fitness functions sampled from a generalization of the NK model, a widely-used random field model for fitness functions. In particular, we present theoretical results that allow us to test the effect of the Generalized NK (GNK) model's interpretable parameters---sequence length, alphabet size, and assumed interactions between sequence positions---on the sparsity of fitness functions sampled from the model, and use these results to determine the number of measurements required to exactly recover GNK fitness functions. Further, we show that GNK fitness functions with parameters set according to protein structural contacts accurately approximate the sparsity of empirical fitness functions and can be used to approximate the number of experimental measurements required to effectively estimate a protein fitness function.

Finally, we describe an end-to-end modeling approach for designing a library of insertion sequences to the Adeno-Associated Virus (AAV) capsid with the goal of improving the ability of the capsid to fold and package the viral genome. AAV is a promising vector for delivering gene therapies and packaging is a requirement for effective delivery to a target tissue; a library with improved packaging ability is more likely to infect these tissues in a downstream experiment. Our approach to library design requires solutions to both forward and inverse problems. In particular, we first fit a predictive model to a set of high-throughput experimental data that reports on packaging fitness. We then present a novel method to design libraries that optimally balance average fitness and sequence diversity, and apply this method to the design of AAV insertion sequence libraries.

In summary, this dissertation presents a number of ML-based techniques for biological sequence engineering, as well as theoretical insights that can guide future methodological developments. This work provides further motivation for the biological community to adopt machine learning as a tool to be used alongside experiments in order to effectively engineer sequences.

Proteins serve many crucial functions in maintaining life, but have also been co-opted for human endeavors such as gene editing, immunotherapy, and plastic degradation. To better serve such needs, we re-engineer naturally existing proteins or design de novo proteins. In recent years, data at an unprecedented scale from evolution, 3D structures, and experiments became available for learning to design and engineer proteins. These distinct data types provide different yet complementary information about proteins. This thesis presents new machine learning methods that learn from multiple types of data for the problems of sequence-based protein fitness prediction and structure-based fixed backbone protein design.

For sequence-based protein fitness prediction, machine learning-based models typically learn from either unlabelled, evolutionarily-related sequences, or variant sequences with experimentally measured labels. For regimes where only limited experimental data are available, combining both sources of information could improve protein fitness prediction. Toward that goal, we propose a simple combination approach that is competitive with, and on average outperforms more sophisticated methods. The comparative analysis also highlights the importance of systematic evaluations and sufficient baselines.

For structure-based fixed backbone protein design, prior machine learning approaches to this problem have been limited by the number of available experimentally determined protein structures. We present a strategy to augment the training data by nearly three orders of magnitude by predicting millions of structures using AlphaFold2. Graph neural network and Transformer models trained with this additional data achieves an overall improvement of almost 10 percentage points over existing methods in native sequence recovery rate. We also study the generalization to a variety of more complex tasks including design of protein complexes, partially masked structures, binding interfaces, and multiple states.

Optical microscopes have been an indispensable tool in biology and medicine for over three centuries. Unlike their simple predecessors, contemporary microscopes often employ complex robotic automation and customized algorithms. In the past decade, advances in high-performance computer processors, the ease of collecting massive datasets, and machine learning have created many new possibilities for data-driven approaches to microscope control and image analysis.This dissertation covers the challenges and opportunities in modern microscopy. First, it shows how neural networks can be used to create microscopes that adapt to the samples they are imaging in real time. For example, this paradigm can be used to quickly focus microscopes using inexpensive hardware or visualize developing immune responses at large scales. Next, new open-source software that facilitates development of these and other microscopy techniques is presented. Next, it turns to how microscopes can make measurements of the intrinsic optical properties of cells, from which their biological function can be inferred. Development of techniques that do so requires comparing approaches on standardized datasets, and the creation of such a dataset containing hundreds of thousands of images of single cells is described. Finally, a new theoretical framework for modeling the information transmission of both microscopes and image-processing algorithms is introduced. This perspective provides a new set of engineering principles for microscopes and opens a range of new research questions.

Using next-generation sequencing, it is now possible to screen up to billions of protein or DNA sequences in parallel for a property of interest. Consequently, high-throughput sequencing has vastly accelerated the rate of biological discovery for both basic scientific inquiry and for engineering novel enzymes, therapeutics, antibodies, regulatory elements, and beyond. In such high-throughput sequencing-based screens and selections, the quality of the starting sequence library greatly influences the overall chance of successfully identifying sequences with the desired property. Generalizable in silico methods for designing high-quality sequence libraries promise to reduce wetlab experimental burden and improve the speed with which new, functional sequences can be discovered. Machine learning, in particular, provides a useful set of tools for implementing such methods, as it is well-suited to analyzing the large quantities of data produced by high-throughput sequencing. In this dissertation, we will discuss several aspects of machine learning-guided library design, and propose solutions to challenges posed by existing technologies.

First, we introduce a framework for machine learning-guided library design, and showcase its ability to design diverse, functional libraries in a gene therapy context. Specifically, we (i) outline a modeling approach for predicting the property selected for in a high-throughput sequencing-based selection experiment that explicitly accounts for uncertainty in the observed sequencing data, and (ii) describe a novel machine learning-guided design procedure that optimally trades off between a library's average predicted property values and its sequence diversity. We use these methods to design a clinically-relevant adeno-associated virus (AAV) peptide insertion library. AAVs hold tremendous promise as delivery vectors for clinical gene therapy, and packaging is a general prerequisite for delivering genetic material to a target tissue. Standard diversified libraries for engineering effective AAV delivery vectors contain a high proportion of variants that are unable to assemble or package their genomes, which often limits the effectiveness of downstream selections for desired properties such as efficient infection of human tissues. Using our machine learning-guided design framework, we systematically design effective starting libraries that are as diverse as possible whilst being biased towards variants that are able to assemble and package the viral genome efficiently. Specifically, we design a library of peptide insertions into the AAV capsid that achieves five-fold higher packaging fitness than the standard insertion library---known as the ``NNK'' library---with negligible reduction in diversity. We further demonstrate the general utility of our designed library on a downstream task to which our design approach was agnostic: infection of primary human brain tissue. Compared to the standard NNK library, our machine learning-designed library contains approximately 10-fold more variants that successfully infect the human brain.

Next, we highlight a key shortcoming of the above predictive modeling approach---namely, its extremely limited ability to share information across related but non-identical reads---that prevents it from making effective use of sequencing data in many settings of interest. We introduce model-based enrichment (MBE) to overcome this shortcoming. MBE is based on a new perspective of differential sequencing analysis that uses sound theoretical principles from the density ratio estimation field in machine learning, is easy to implement, and can trivially make use of advances in modern-day machine learning classification architectures or related innovations. We evaluate MBE empirically, both in simulation and on real experimental data, and show that it improves accuracy compared to current ways of performing sequencing-based differential analyses---including the previous section's predictive modeling approach. The greater flexibility of our new approach enables effective analysis across a broader range of common experimental setups than can currently be achieved, thereby expanding the set of biological applications for which one can learn accurate predictive models to guide library design.

Finally, we highlight some remaining challenges for machine learning-guided library design, including research opportunities into combining multiple sources of biological information in the design process. In summary, this dissertation presents a number of machine learning techniques that can be brought to bear on the problem of designing improved starting libraries for biological screens and selection experiments. The insights from this work provide further motivation for researchers to combine laboratory experiments with tools from machine learning to efficiently engineer novel functional protein and DNA sequences.

The last decade has witnessed rapid development and deployment of machine-learning systems across science. Such systems can supply predictions about scientific phenomena far more quickly and cheaply than gold-standard experiments, and are being used in efforts to both discover scientific knowledge and design new biomolecules. However, an important question remains unanswered: since machine-learning systems make errors, how can we use them in a trustworthy way for scientific discovery and design? This dissertation takes steps toward helping to ensure that the biomolecules we design and the scientific conclusions we draw using machine learning can be trusted.

We begin in the setting of machine learning-based design. The goal in this setting is to propose novel objects such as proteins, small molecules, or materials with desired properties, in a way that is guided by machine-learning models of such properties. Toward addressing model trustworthiness for design, we propose (i) a method for learning models that accounts for the distribution shifts inherent to design, and (ii) a method for constructing statistically valid confidence sets for the properties of objects designed using machine learning.

Finally, we examine the trustworthy use of machine learning for drawing scientific conclusions. In particular, we consider the increasingly relevant setting of treating predictions made by machine-learning systems as “data” in estimating quantities of scientific interest. We propose prediction-powered inference, a novel statistical framework for constructing valid confidence sets in this setting, which enables researchers to incorporate evidence from machine-learning systems into their scientific inquiry in a standardized and principled way.