Quantitative optical microscopy continues to be a powerful tool for biomedical research and the sciences at-large. Building upon centuries of optical theory, computational microscopy leverages large-scale numerical computation to dramatically extend and improve the capabilities of existing optical microscopes through high-speed capture, super-resolution, or opening new application spaces such as quantitative phase imaging. This dissertation describes the theory and reduction to practice of several novel computational microscopy techniques which make use of jointly-designed coded illumination hardware and physics-based reconstruction algorithms. We first introduce label-free quantitative phase imaging using differential phase contrast, and demonstrate a novel single-shot variant using color-multiplexing. Second, we explore coded illumination for high-throughput imaging, and demonstrate a temporal-coding technique which enables significantly higher SNR for high-speed slide scanning and neuropathology applications. For fluorescent imaging, this method can provide up to a 10x improvement in reconstruction SNR compared to conventional high-speed imaging techniques. Next, we explore the design and fabrication of LED illumination devices, including a quasi-dome LED illuminator which enables high-angle illumination for a variety of applications. To address practical calibration concerns for computational microscopy systems, we demonstrate two examples of algorithmic self-calibration. These include aberration recovery using differential phase contrast as well as source calibration for the quasi-dome LED array, using both image-based calibration and an online method based on Fourier ptychography. These works demonstrate the benefits and practical challenges related to computational optical microscopy with coded illumination.