Recent imaging studies have found that in simple arithmetic processing, addition is lateralized to the right hemisphere, whereas multiplication to the left. Here we aimed to investigate the cognitive mechanism underlying complicated arithmetic processing with a dual task paradigm. Participants were asked to complete a calculation task (addition or multiplication) and a letter judgment task (rhyme or shape judgment) simultaneously. We found that participants’ performance in addition and multiplication was interfered more by the simultaneous shape judgment task than the rhyme judgment task. This effect suggested that both complicated addition and multiplication relied more on right-lateralized visuospatial than left-lateralized phonological/verbal processing. The shift from left- to more right-lateralized processing in complicated multiplication suggests that participants may have adopted a visuospatial strategy to approximate numerosity when the calculation involved large numbers. These results suggest that the cognitive mechanism involved in arithmetic processing depends on both the operation and the context.