This dissertation builds a modeling framework for non-Gaussian spatial processes, time series, and point processes, with a Bayesian inference paradigm that provides uncertainty quantification. Our methodological development emphasizes direct modeling of non-Gaussianity, in contrast with traditional approaches that consider data transformations or modeling through functionals of the data probability distribution. We achieve the goal by defining a joint distribution through factorization into a product of univariate conditional distributions according to a directed acyclic graph which implies conditional independence. We model each conditional distribution as a weighted combination of first-order conditionals, with weights that can be locally adaptive, for each one of a given number of parents which correspond to spatial nearest-neighbors or temporal lags. Such a formulation features specification of bivariate distributions that define the first-order conditionals for flexible, parsimonious modeling of multivariate non-Gaussian distributions. We obtain, in time, high-order Markov models with stationary marginals, and point process models for limited memory, dependent renewals, and duration clustering; and in space, nearest-neighbor mixture models for spatial processes. Regarding computation, representing the framework by directed acyclic graphs, with a mixture model formulation for the conditionals, gains efficiency and scalability relative to many non-Gaussian models. We develop Markov chain Monte Carlo algorithms for implementation of posterior inference and prediction, with data illustrations in biological, environmental, and social sciences.