As the cost of short read DNA sequencing continues to drop, new experiments are being developed which allow scientists to probe subtle biological phenomenon on a genome wide scale. These new experiments bring new analytical challenges in the form of large amounts of noisy data. Here we present models and tools for the analysis of high throughput sequencing data.

Clustering is an important unsupervised classification technique. In supervised classification, we are provided with a collection of labeled (pre-classified) patterns and the problem is to label a newly encountered, yet unlabeled, pattern. At first, we consider clustering in Euclidean space in large dimensions. Then, we delve into the discrete setting of networks. We go into the issues related to network modeling and then into a specific method of clustering in networks.

In the first chapter, we consider the problem of estimation and deriving theoretical properties of the estimators for the elliptical distributions. The class of elliptical distributions have distributions with varied tail behavior. So, estimation under class of elliptic distributions lead to automatic robust estimators. The goal of the chapter is to propose efficient and adaptive regularized estimators for the nonparametric component, mean and covariance matrix of the elliptical distributions in both high and fixed dimensional situations. An algorithm for regularized estimation of mixture of elliptical distributions will also lead to an algorithm for finding elliptical clusters in high dimensional space and such an approach is also given in the chapter. In clustering, one of the main challenges is the detection of number of clusters. Most clustering algorithms need the number of clusters to be specified beforehand. In chapter two, we propose a new method of selecting number of clusters, based on hypothesis testing.

The study of networks has received increased attention recently not only from the social sciences and statistics but also from physicists,computer scientists and mathematicians. But a proper statistical analysis of features of different stochastic models of networks is still underway. In chapter three, we give an account of different network models and then we analyze a specific nonparametric model for networks. We consider the nonparametric estimate of link probabilities in dense social graphs in the context of network modeling and exploratory statistics.

In chapter four, we also propose bootstrap methods for finding empirical distribution of count features or `moments' and smooth functions of these for the networks. Using these methods, we can not only estimate variance of count features but also get good estimates of such feature counts, which are usually expensive to compute numerically in large networks. In our paper, we prove theoretical properties of the bootstrap variance estimates of the count features as well as show their efficacy through simulation. We also use the method on publicly available Facebook network data for estimate of variance and expectation of some count features.

In chapter five, we propose a clustering or community detection scheme for networks. One of the principal problem in networks is community detection. Many algorithms have been proposed for community finding but most of them do not have have theoretical guarantee for sparse networks and networks close to phase transition boundary proposed by physicists. There are some exceptions but all have incomplete theoretical basis. Here we propose an algorithm based on the graph distance of vertices in the network. We give theoretical guarantees that our method works in identifying communities for sparse block models. We illustrate on a network of political blogs, Facebook networks and some other networks.

State space models are powerful modeling tools for stochastic dynamical systems and have been an important research area in the statistics community in the last several decades. This thesis makes contributions to the filtering problem, a key inference problem in general state space models. Our work in this area is motivated by both high-dimensional, nonlinear applications such as numerical weather forecasting and fundamental theoretical problems such as the convergence of filters.

First we study the ensemble Kalman filters (EnKF), a popular class of filtering methods in geophysics because they are easy to implement in large systems. However, their behavior in non-Gaussian situations is only partially understood. We compare two common versions of EnKF's under non-Gaussianity from a robustness perspective. The results support previous empirical studies on the same issue and provide additional insight in choosing a free parameter in the EnKF algorithms.

Second, we consider the filtering problem in high dimensional situations such as numerical weather forecasting. We review the EnKF from a statistical perspective and analyze its sources of bias. Then we propose a new method to reduces the bias, namely the non-linear ensemble adjustment filter (NLEAF). The one-step consistency of the NLEAF is studied and the performance is examined through simulations in two common testbeds in the weather forecasting literature. Finally we look at the theoretical properties of another popular class of filtering methods, the sequential Monte Carlo (SMC) filter. The convergence of SMC filters has been a challenging problem in both probability and statistics. The previous results either depend on strong mixing conditions which only hold in compact spaces or provide no rates of convergence or are under weak notions of distance, limiting the application of their practical use. We provide checkable sufficient conditions under which explicit rates of convergence of the SMC filter can be derived. The conditions essentially requires the regularity of the tail behavior of the process and they are general enough to include a wide class of autoregressive models as well as Gaussian linear models.

In multivariate analysis, canonical correlation analysis is a method that enable us to gain insight into the relationships between the two sets of variables. It

determines linear combinations of variables of each type with maximal correlation

between the two linear combinations. However, in high dimensional data analysis, insufficient sample size may lead to computational problems, inconsistent estimates of parameters.

In Chapter 1, three new methods of regularization are presented to improve the traditional CCA estimator in high dimensional settings. Theoretical results have been derived and the methods are evaluated using simulated data.

While the linear methods are successful in many circumstances, it certainly has some limitations, especially in cases where strong nonlinear dependencies exist. In Chapter 2, I investigate some other measures of dependence, including the rank correlation and its extensions, which can capture some non-linear relationship between variables. Finally the Renyi correlation is considered in Chapter 3. I also complement my analysis with simulations that demonstrate the theoretical results.

Enhancer discovery through computational means has long been a goal of the genomics community. The tools developed for this purpose, however, tend to underperform when tested on completely held out test sets. Here we use the pregrastrula patterning network of Drosophila melanogaster to demonstrate that loss in accuracy in held out data results from heterogeneity of functional signatures in enhancer elements. We show that at least two classes of enhancer are active during early Drosophila embryogenesis and that by focusing on a single, relatively homogeneous class of elements, extremely high (>98%) prediction accuracy can be achieved in a balanced, held-out test set. The homogenous set is composed predominantly of enhancers driving multi-stage, large segmentation patterns in the early embryo, and hence we term them segmentation driving enhancers (SDE). Prediction is primarily driven by transcription factors DNA occupancy with almost no power derived from histone modifications, including H3K27ac, casting further doubt on the utility of histone modifications to demarcate enhancer elements. The transcription factors used in the prediction process constitute over half of the transcription factors identified in genetic screens as patterning the early embryo, and hence provide a remarkably expansive view of this process. Applying this method to a genome-wide scan, we predict 1,600 SDEs, 916 of which are novel, covering approximately 1.6% of the euchromatic genome. We verified these predictions by testing 41 novel SDEs using in situ whole embryo imaging of stably integrated reporter constructs. We confirmed 39 of these predictions, a 95% precision on a genome-wide scan with an estimated recall of 98%, indicating that our reported collection of SDEs may be close to comprehensive.

The study of biological networks has been central to our understanding of life and its complex, dynamic nature. The elucidation of molecular networks began with the discovery and characterization of key cellular processes including metabolism, response to stimuli and control of gene expression. In the last several decades, genomics has emerged as a foundational pursuit within the life sciences. The size of datasets defined in relation to sequenced genomes has grown faster than exponentially, leading to the need for advanced analytical and computational methods. I present here three studies of large RNA-sequencing-based data sets. First, a study of the steady state transcriptional composition for Drosophila cell lines, tissues, developmental stages and biological perturbations provide a deeper understanding of spatiotemporally-resolved regulation in Drosophila, the first and still central genetic model system. This dataset, at the time of my analysis, was the largest and most complete transcriptional atlas ever composed. It was also the first large strand-specific study of its kind, which presented new opportunities and challenges. Second, a study of the RNA targets of 20 RNA binding proteins provides a map for one layer of post-transcriptional regulation, which contributes to the steady states presented in the first study. Finally, a study of transcriptional responses to the principle developmental hormone in arthropods, ecdysone, across 41 different and physiologically distinct cell lines sheds light on the dynamic, responsive nature of gene-regulatory networks that enable cells to differentiate into the diverse tissues that compose developing and mature organisms. These studies provide foundational knowledge, as well as models for future work in systems biology, as genome-scale studies across larger, more diverse cellular states become increasingly prevalent.

First part of the thesis focuses on sparse covariance matrices estimation under the scenario of large dimension p and small sample size n. In particular, we consider a class of covariance matrices which are approximately block diagonal under unknown permutations. We propose a block recovery estimator and show it achieves minimax optimal convergence rate for the class, which is the same as if the permutation were known. The problem is also related to sparse PCA and k-densest subgraphs, where the spike model is a special case of their intersection. Simulations of the spike model and multiple block model, together with a real world application, confirm that the proposed estimator is both statistically and computationally efficient.

Second part of the thesis focuses on variable selection in linear regression, also under the high dimensional scenario of large p and small n. We propose a general framework to search variables based on their covariance structures, with a specific variable selection algorithm called kForward which iteratively fits local/small linear models among relatively highly correlated variables. For simulation experiments and a real world data set, we compare kForward to other popular methods including the Lasso, Elastic Net, SCAD, MC+, FoBa for both variable selection and prediction.

Networks are abstract representations of relationships between a set of entities. As such they can be used to represent data in a variety of complex interactive systems such as people and their social connections, researchers and their collaborations, proteins and their interactions, and so on. Vast amounts of such interaction data are being collected routinely in a range of disciplines and thus call for the attention of the statistician. Due to their large size (number of observations scales as the square of the number of nodes), traditional statistical methods are usually not scalable and one needs to come up with more computationally feasible inference techniques.

A concrete example of the issue is the problem of community detection in networks. Traditional likelihood-based methods are computationally intractable, so researchers have come up with various computation-friendly alternatives. Although these methods work well on small to moderately large networks, most of them cannot handle truly large networks in a reasonable amount of time.

In this dissertation, we first advance divide and conquer strategies for community detection. We propose two algorithms which perform clustering on a number of small subgraphs and finally patch the results into a single clustering. The main advantage of these algorithms is that they bring down significantly the computational cost of traditional algorithms, including spectral clustering, semidefinite programs, modularity based methods, likelihood based methods, etc., without losing on accuracy and even improving accuracy at times. These algorithms are also, by nature, parallelizable. Thus, exploiting the facts that most traditional algorithms are accurate and the corresponding optimization problems are much simpler in small problems, our divide and conquer methods provide an omnibus recipe for scaling traditional algorithms up to large networks. We prove consistency of these algorithms under various subgraph selection procedures and perform extensive simulations and real data analysis to understand the advantages of the divide and conquer approach in various settings.

We then extend these divide and conquer methods to the more realistic situation of mixed memberships. Models that can be tackled are the mixed membership blockmodel, topic models, etc.

Next we focus on the problem of network comparison. We tackle two aspects of this problem: clustering and changepoint detection.

While being able to cluster within a network, in the sense of community detection, is important, there are emerging needs to be able to \emph{cluster multiple networks}. This is largely motivated by the routine collection of network data that are generated from potentially different populations. These networks may or may not have node correspondence. For example, brain networks of a group of patients have node correspondence, whereas collaboration networks of researchers in different disciplines such as Computer Science, Mathematics or Statistics will have little node correspondence. When node correspondence is present, we cluster networks by summarizing a network by its graphon estimate, whereas when node correspondence is not present, we propose a novel solution for clustering such networks by associating a computationally feasible feature vector to each network based on traces of powers of the adjacency matrix. We illustrate our methods using both simulated and real data sets, and theoretical justifications are provided in terms of consistency.

In the changepoint problem, one observes a series of networks indexed by time and wishes the check if there is some significant change in the structure of these networks at some point of time. Potential applications are in, for instance, brain imaging, where one has brain scans of individuals collected over time and is looking for abnormalities, ecological networks observed over time, where one wonders if there is a structural change. We consider a CUSUM (short for cumulative sum) statistic for this problem, and prove its consistency. We find that in this high dimensional setting, the estimation error rate is better than the classical rate for fixed dimensional changepoint problems. As applications, we detect changepoints in the MIT reality mining data and the US senate roll call data.

During the past two decades, technological advances have led to a proliferation of high-dimensional problems in data analysis. The characteristic feature of such problems is that they involve large numbers of unknown parameters and relatively few observations. As the study of high-dimensional statistical models has developed, linear models have taken on a special status for their widespread application and extensive theory. Even so, much of the theoretical research on high-dimensional linear models has been concentrated on the problems of prediction and estimation, and many inferential questions regarding hypothesis tests and confidence intervals remain open.

In this dissertation, we explore two sets of inferential questions arising in high-dimensional linear models. The first set deals with the residual bootstrap (RB) method and the distributional approximation of regression contrasts. The second set addresses the issue of unknown sparsity in the signal processing framework of compressed sensing. Although these topics involve distinct methods and applications, the dissertation is unified by an overall focus on the interplay between model structure and inference. Specifically, our work is motivated by an interest in using inferential methods to confirm the existence of model structure, and in developing new inferential methods that have minimal reliance on structural assumptions.

The residual bootstrap method is a general approach to approximating the sampling distribution of statistics derived from estimated regression coefficients. When the number of regression coefficients p is small compared to the number of observations n, classical results show that RB consistently approximates the laws of contrasts obtained from least-squares coefficients. However, when p/n~1, it is known that there exist contrasts for which RB fails --- when applied to least-squares residuals. As a remedy, we propose an alternative method that is tailored to regression models involving near low-rank design matrices. In this situation, we prove that resampling the residuals of a ridge regression estimator can alleviate some of the problems that occur for least-squares residuals. Notably, our approach does not depend on sparsity in the true regression coefficients. Furthermore, the assumption of a near low-rank design is one that is satisfied in many applications and can be inspected directly in practice.

In the second portion of the dissertation, we turn our attention to the subject of compressed sensing, which deals with the recovery of sparse high-dimensional signals from a limited number of linear measurements. Although the theory of compressed sensing offers strong recovery guarantees, many of its basic results depend on prior knowledge of the signal's sparsity level --- a parameter that is rarely known in practice. Towards a resolution of this issue, we introduce a generalized family of sparsity parameters that can be estimated in a way that is free of structural assumptions. We show that our estimator is ratio-consistent with a dimension-free rate of convergence, and also derive the estimator's limiting distribution. In turn, these results make it possible to set confidence intervals for the sparsity level and to test the hypothesis of sparsity in a precise sense.

Data that quantify various aspects of medicine and biology are constantly improving and changing. As new techniques become available, opportunities arise to improve our understanding of diseases and other biological properties. With these ever changing data modalities new statistical techniques are required to do proper analysis. In this thesis we analyze, and develop new methods when necessary, for three types of biological data, bulk and single cell RNA-sequencing as well as magnetic resonance imaging (MRI).

First, we develop a statistic for analyzing high-throughput RNA-sequencing data in the context of drug discovery. Changes in gene expression between disease and normal tissues can be used to understand the genomic signature of a disease. When those diseased cells are exposed to a large number of drugs and other perturbations, we can systematically search for the perturbations that reverse the expression of the genes which are altered in the disease. In Chapter 2, we present a new method for quantifying this relationship between disease and drug that outperforms existing methods in simulation, decreases computation time, and is comparable in real data. Additionally, we show an improved ability to quantify the involvement of individual genes in effective drugs. An accompanying software package makes this method easily applicable to new data.

In the second study we extend an existing semi-supervised learning method called GeneFishing for application to single cell RNA-sequencing data (scRNA-seq). Single cell data has unique properties that make its analysis different from the bulk data used in Chapter 2. While direct application of GeneFishing was not always possible in scRNA-seq, due to dissimilar sources of variation and a marked increase in technical noise, the provided modifications allowed for the analysis to be possible. In addition to using GeneFishing as an effective gene prioritization method in single cell data, we show that it can be used as a way to understand how to measure gene-gene co-expression and as a context specific feature selection method for downstream analyses. Again, we present the accompanying software package, scGeneFishing, for performing GeneFishing in a variety of datasets, including scRNA-seq.

Finally in Chapter 4 of this dissertation we analyze a different type of medical data, brain images. We use MRI scans of patients with primary progressive multiple sclerosis to study the association of regional brain volume at baseline with disease progression. We find that statistics summarizing volume in pre-defined regions of interest (ROIs) are not more predictive of progression than using traditional clinical and MRI variables. However, by deriving data-driven ROIs through voxel-level clustering we are able to achieve better predictive performance.