Oncogenic pathways that drive cancer progression reflect both genetic changes and epigenetic regulation. Here we stratified primary tumors from each of 24 TCGA adult cancer types based on the gene expression patterns of epigenetic factors (epifactors). The tumors for five cancer types (ACC, KIRC, LGG, LIHC, and LUAD) separated into two robust clusters that were better than grade or epithelial-to-mesenchymal transition in predicting clinical outcomes. The majority of epifactors that drove the clustering were also individually prognostic. A pan-cancer machine learning model deploying epifactor expression data for these five cancer types successfully separated the patients into poor and better outcome groups. Single-cell analysis of adult and pediatric tumors revealed that expression patterns associated with poor or worse outcomes were present in individual cells within tumors. Our study provides an epigenetic map of cancer types and lays a foundation for discovering pan-cancer targetable epifactors.