Male infertility has been a primary cause of global infertility, affecting 8-12% of couples worldwide. Previous studies have shown that semen quality decreases with advanced aging with an increased presence of inflammatory cells. In this study, we examined changes in the epigenome across a diverse cohort that includes both fertile and infertile men. We also compare the age-associated changes in semen to those observed in buccal swabs in order to characterize differences in epigenetic aging across diverse tissues. We found that variations in the semen methylome associated with aging are linked to inflammatory genes. Many age-associated sites are demethylated with advanced aging and are associated with the activation of inflammatory pathways. By contrast, we do not observe age-associated changes in inflammatory genes in buccal swab methylomes, which instead are characterized by changes to bivalent promoters. Our findings highlight the potential of epigenetic markers as indicators of male reproductive health.