We provide a generalized, normative model of visual detection that accounts for key asymmetries between decisions about presence and about absence. In our model, decisions about presence are made based on the visibility of presented stimuli, but decisions about absence are made based on counterfactual visibility: beliefs about the degree to which a stimulus would have been visible if present. Behavioral patterns in visual detection experiments under different levels of partial occlusion validate key model predictions. Specifically, we find that unlike decisions about presence, the confidence and speed of decisions about absence are largely independent of perceptual evidence, but are sensitive to the counterfactual visibility of absent stimuli. Finally, we reveal robust individual differences in counterfactual perception, with some participants systematically incorporating counterfactual visibility into perceptual decisions in a different fashion from others. We discuss implications for the varied and inferential nature of visual perception more broadly.