Using space to think about time appears to be a human universal, but the specifics of people’s space-time mappings vary across individuals and groups. What determines how people spatialize time in their minds? Here we hypothesized that motor experience contributes to individuals’ space-time mappings, independent of linguistic and cultural factors. Many everyday motor actions require using the hands in a particular sequence (e.g., positioning a nail with the nondominant hand before swinging the hammer with the dominant hand), establishing a correlation between space (left hand, right hand) and time (earlier action, later action) that reverses with handedness. These action sequences reinforce Westerners’ typical left-to-right mental timeline for right-handers, but contradict this timeline for left-handers. Accordingly, we find that right-handers associate leftward space with earlier times and rightward space with later times more strongly than left-handers, for whom cultural and bodily experiences present contrasting relationships between space and time.