Beta-amyloid (Aβ) is thought to be a key contributor to the pathogenesis of Alzheimer disease (AD) in the general population and in adults with Down syndrome (DS). Different assembly states of Aβ have been identified that may be neurotoxic. Aβ oligomers can assemble into soluble prefibrillar oligomers, soluble fibrillar oligomers and insoluble fibrils. Using a novel antibody, OC, recognizing fibrils and soluble fibrillar oligomers, we characterized fibrillar Aβ deposits in AD and DS cases. We further compared human specimens to those obtained from the Tg2576 mouse model of AD. Our results show that accumulation of fibrillar immunoreactivity is significantly increased in AD relative to nondemented aged subjects and those with select cognitive impairments (p < 0.0001). Further, there was a significant correlation between the extent of frontal cortex fibrillar deposit accumulation and dementia severity (MMSE r = −0.72). In DS, we observe an early age of onset and age-dependent accumulation of fibrillar OC immunoreactivity with little pathology in similarly aged non-DS individuals. Tg2576 mice show fibrillar accumulation that can be detected as young as 6 months. Interestingly, fibril-specific immunoreactivity was observed in diffuse, thioflavine S-negative Aβ deposits in addition to more mature neuritic plaques. These results suggest that fibrillar deposits are associated with disease in both AD and in adults with DS and their distribution within early Aβ pathology associated with diffuse plaques and correlation with MMSE suggest that these deposits may not be as benign as previously thought.