Scientific discoveries are advanced by flexible and efficient statistical models. Grounded on Bayesian nonparametric modeling techniques, this thesis provides a toolbox for ordinal regression. The toolbox comprises models tailored for various settings, with shared characteristics of flexibility and efficiency. A key building block of the proposed models is a sequential mechanism to treat the ordinal response. Such mechanism implies a factorization of the response distribution that allows efficient, scalable computation through (partial) parallel sampling regarding the response categories. For problems under a cross-sectional setting, we develop nonparametric mixture models, leveraging the same sequential structure to define covariate-dependent mixture weights. Even though covariates are incorporated via linear functions, the mixture models admit flexible ordinal regression relationships, and they relax parametric assumptions for the response distribution. Moving towards modeling the dynamic evolution of ordinal responses from longitudinal studies, the critical insight is to treat the subjects measurements as stochastic process realizations at the corresponding time points. We propose a hierarchical framework that models the mean and covariance structure of the processes nonparametrically and simultaneously, a useful byproduct being a practical method for making predictions on any time scale. For all proposed models, we craft read- ily implementable Markov chain Monte Carlo algorithms that avoid specialized updates or tuning steps. A variety of synthetic and real data examples are used to illustrate the methods. In particular, the models for cross-sectional ordinal regression, along with their extensions, are examined in the context of risk assessment for developmental toxicity studies. We also present a case study in evolutionary biology, in which our method for longitudinal ordinal responses is adapted to identify the impact of temperature on transgenerational responses, using repeated measurements on fish maturation data.