Myocardial infarction leads to complex changes in the fiber architecture of the heart. Here, we present a novel optical approach to characterize these changes in intact hearts in three dimensions. Optical coherence tomography (OCT) was used to derive a depth-resolved field of orientation on which tractography was performed. Tractography of healthy myocardium revealed a smooth linear transition in fiber inclination or helix angle from the epicardium to endocardium. Conversely, in infarcted hearts, no coherent microstructure could be identified in the infarct with OCT Additional characterization of the infarct was performed by the measurement of light attenuation and with two-photon microscopy. Myofibers were imaged using autofluorescence and collagen fibers using second harmonic generation. This revealed the presence of two distinct microstructural patterns in areas of the infarct with high light attenuation. In the presence of residual myofibers, the surrounding collagen fibers were aligned in a coherent manner parallel to the myofibers. In the absence of residual myofibers, the collagen fibers were randomly oriented and lacked any microstructural coherence. The presence of residual myofibers thus exerts a profound effect on the microstructural properties of the infarct scar and consequently the risk of aneurysm formation and arrhythmias. Catheter-based approaches to segment and image myocardial microstructure in humans are feasible and could play a valuable role in guiding the development of strategies to improve infarct healing.