Specialized sound localization circuit development requires synapse strengthening, refinement, and pruning. Many of these functions are carried out by microglia, immune cells that aid in regulating neurogenesis, synaptogenesis, apoptosis, and synaptic removal. We previously showed that postnatal treatment with BLZ945 (BLZ), an inhibitor of colony stimulating factor 1 receptor (CSF1R), eliminates microglia in the brainstem and disables calyceal pruning and maturation of astrocytes in the medial nucleus of the trapezoid body (MNTB). BLZ treatment results in elevated hearing thresholds and delayed signal propagation as measured by auditory brainstem responses (ABR). However, when microglia repopulate the brain following the cessation of BLZ, most of the deficits are repaired. It is unknown whether this recovery is achievable without the return of microglia. Here, we induced sustained microglial elimination with a two-drug approach using BLZ and PLX5622 (PLX). We found that BLZ/PLX treated mice had impaired calyceal pruning, diminished astrocytic GFAP in the lateral, low frequency, region of MNTB, and elevated glycine transporter 2 (GLYT2) levels. BLZ/PLX treated mice had elevated hearing thresholds, diminished peak amplitudes, and altered latencies and inter-peak latencies. These findings suggest that microglia are required to repopulate the brain in order to rectify deficits from their ablation.