A critical function of animal coloration is avoiding attack, either by warning predators or reducing detectability. Evolution of these divergent strategies may depend on prey palatability and apparency to predators: conspicuous coloration may be favoured if species are distasteful, or habitats make hiding difficult; by contrast, camouflage may be effective if prey lack defences or environments are visually complex. For insect herbivores, host plants provide both chemical defence and the background against which they are detected or obscured; thus, plant traits may be key to coloration in these foundational terrestrial organisms. We use 1808 species of larval Lepidoptera to explore macroevolution of protective coloration strategy. We find that colour and pattern evolve jointly in caterpillars, similar to an array of species across the animal kingdom, while individual elements of coloration evolve closely with diet ecology. Consistent with key tenets of plant defence and plant-herbivore coevolutionary theory, conspicuous colours are associated with herbaceous host plants-thought to be defended by toxins-while camouflage colours and patterns are associated with woody plants and grasses. Contrary to theory, dietary specialization is not associated with conspicuous coloration. Our results add valuable insights into the evolutionary forces shaping colour and pattern in nature.