Proceedings of the Annual Meeting of the Cognitive Science Society

Over the last few decades, cognitive scientists have developed sophisticated formal models of human categorization,and computer vision researchers have achieved increasingly impressive performance in natural image classification. In thispaper, we combine the strengths of these approaches, using representations from a convolutional neural network to evaluatecognitive models of categorization against >300,000 human judgments of natural images. We find that a prototype modelperforms best overall, and that an exemplar model performs best when the network’s most abstract features are used. Altogether,our results demonstrate that the optimal categorization strategy over a set of stimuli is deeply linked to how they are represented,suggesting that any satisfying characterization of categorization behavior over naturalistic stimuli must consider it the resultof a dual process of feature learning and strategy selection. The paradigm we present herein offers one avenue to begin thisundertaking.