With the advancement of technologies, high-dimensional data have become more and more popular. One aspect of these data is that each subject often contains many features and has a possibly complicated intrinsic structure. It has motivated enormous research to tackle these challenges. This dissertation works on three different but related problems. Two of them are rooted in functional data, an extreme case of high-dimensional data, where each subject is a smooth curve and has infinite many values. For the third problem, we analyze the bootstrap method in the context of high-dimensional principal component analysis (PCA).In the first proportion of this dissertation, we discuss the scalar-on-function linear regression model in the sparse observation case and propose two methods to estimate the linear relationship. This also leads to a new framework for prediction with sparse functional covariates. Rates of convergence in estimation and prediction are established for both approaches. Next, we look at the emerging field of applying modern deep learning to functional data. We focus on both regression and classification tasks with functional input and finite (possible non-linear) index relations. The new methodology learns to apply parsimonious dimension reduction to functional inputs and focus only on information relevant to the target rather than irrelevant variation in the input function in an end-to-end fashion, removing the manual selection of principal components in classical solutions. Lastly, we turn to the bootstrap method and analyze how well it can approximate the joint distribution of the leading eigenvalues of the sample covariance matrix in high dimensions and establish non-asymptotic convergence rates of approximation. It is also demonstrated that applying a transformation to the sample eigenvalues prior to bootstrapping can lead to inference benefits.