How do we characterize the symmetries of a theory? How should we respond to the presence of ‘excess structure’ in a theory? When are two theories equivalent? This dissertation is an exploration of these questions in the context of a particular formulation of classical theories known as the constrained Hamiltonian formalism.

A theme running through the dissertation is that progress can be made on these questions by making precise the geometric structure of a constrained Hamiltonian theory. I argue that this geometric structure can (1) be used to resolve a debate about how to correctly characterize the relationship between constraints and gauge transformations, (2) shed light on the relationship between the constrained Hamiltonian formalism and the Lagrangian formalism, and (3) provide an avenue for formulating a constrained Hamiltonian theory that resolves an apparent tension between gauge variables being ‘excess structure’ and playing an ineliminable role.