This dissertation mainly addresses the generic types of ageostrophic instability in general continuously differentiable, interior, horizontal and vertical shear flows without special "edges" (vertical, side or equatorial boundaries or frontal outcropping). In contrast to the classic barotropic and baroclinic instabilities, whose nonlinear dynamics (geostrophic turbulence) have an ''inverse cascade" characteristic, the ageostrophic instabilities serve as a local route for the breakdown of balance in the interior ocean or atmosphere, leading to an efficient energy cascade towards small scales. For the first part of this dissertation, the linear instabilities, both momentum-balanced and unbalanced, in several different U(y) shear profiles are investigated in the rotating shallow water equations. The unbalanced instabilities are strongly ageostrophic and involve inertia-gravity wave motions, occurring only for finite Rossby (Ro) and Froude (Fr) numbers. Aside from the classic shear instability among balanced shear wave modes (i.e., B-B type), two types of ageostrophic instability (B-G and G-G) are found. Here B represents balanced shear wave mode, and G represents inertia-gravity wave mode. The B-G instability has attributes of both a balanced shear wave mode and an inertia-gravity wave mode. The G-G instability occurs as a sharp resonance between two inertia-gravity wave modes. The criterion for the occurrence of the ageostrophic instability is associated with the second stability condition of Ripa 1983, which requires a sufficiently large local Froude number. When Ro and especially Fr increase, the balanced instability is suppressed, while the ageostrophic instabilities are enhanced. The profile of the mean flow also affects the strength of the balanced and ageostrophic instabilities. For the second part of this dissertation, the linear instabilities of several rotating, stably stratified, interior vertical shear flows U(z) are solved in Boussinesq equations. Two types of baroclinic, ageostrophic instability, AI1 and AI2, are found in antisymmetric U(z) for intermediate Rossby number (Ro). AI1 is a stationary (zero frequency) instability, which appears in a continuous transformation of the unstable mode properties between classic baroclinic instability (BCI) and centrifugal instability (CI). It begins to occur at intermediate Ro values and horizontal wave numbers (k, l) that are far from l = 0 or k = 0. AI1 grows by drawing energy from the kinetic energy of the mean flow. The instability AI2 always has inertial critical layers at certain heights; and hence it is associated with an inertia-gravity wave. For an unstable AI2 mode, the coupling is either between an interior balanced shear wave and an inertia-gravity wave (B-G), or between two inertia-gravity waves (G-G). The main energy source for an unstable B-G mode is the mean kinetic energy, while the main energy source for an unstable G-G mode is the mean available potential energy. AI1 and AI2 of the B-G type occur in the neighborhood of A-S = 0 (McWilliams et al. 1998), while AI2 of the G-G type arises beyond this condition (A-S denotes absolute vertical vorticity minus strain rate in isentropic coordinates). Both AI1 and AI2 are unbalanced instabilities, which lead to a loss of balance in 3D interior flows.