The determination of the hidden states of coupled nonlinear systems is frustrated by the presence of high-dimensionality, chaos, and sparse observability. This problem resides naturally in a Bayesian context: an underlying physical process produces a data stream, which -- though noisy and incomplete -- can in principle be inverted to express the likelihood of the underlying process itself. A large class of well-developed methods treat this problem in a sequential predict-and-correct manner that alternates information from the presumed dynamical model with information from the data.
One might instead formulate this problem in a temporally global, non-sequential manner, which suggests new avenues of approach within an optimization context, but also poses new challenges in numerical implementation. The variational annealing (VA) technique is proposed to address these problems by leveraging an inherent separability between the convex and nonconvex contributions of the resulting functional forms. VA is shown to reliably track unobservable states in sparsely observed chaotic systems, as well as in minimally-observed biophysical neural models.
Second, this problem can be formally cast in continuous time as a Wiener path integral, which then suggests classical solutions derived from Hamilton's principle. These solutions come with their own difficulties in that they comprise an unstable boundary-value problem. Accordingly, a further technique called Hamiltonian variational annealing is proposed, which again exploits an existing separability of convexity and nonlinearity, this time in a an enlarged manifold constrained by underlying symmetries.
A running theme in this thesis is that the optimal estimate of a nonlinear system is itself a dynamical system that lives in an unstable, symplectic manifold. When this system is recast in a variational context, instability is manifested as nonconvexity, the central idea being that when this nonconvexity is incorporated in a systematic and gradual way, the classical solutions can be tracked reliably.
