Despite a persistent interest in verb processing, data on the neural underpinnings of verb retrieval are fragmentary. The present study is the first to analyze the contributions of both grey and white matter damage affecting verb retrieval through action naming in stroke. We used voxel-based lesion-symptom mapping (VLSM) with an action naming task in 40 left-hemisphere stroke patients. Within the grey matter, we revealed the critical involvement of the left precentral and inferior frontal gyri, insula, and parts of basal ganglia. An overlay of white matter tract probability masks on the VLSM lesion map revealed involvement of left-hemisphere long and short association tracts with terminations in the frontal areas; and several projection tracts. The involvement of these structures is interpreted in the light of existing picture naming models, semantic control processes, and the embodiment cognition framework. Our results stress the importance of both cortico-cortical and cortico-subcortical networks of language processing.