Trauma-exposed adults with high levels of posttraumatic stress symptoms (PTSS) report poorer health-related quality of life (HRQOL), but less is known about the persistence of this relationship over time. Participants from the Vietnam Era Twin Study of Aging reported on PTSS, health, and sociodemographic characteristics at average age 38; 775 participants reported having been exposed to trauma. Later, at average ages 56 and 62, mental and physical HRQOL were assessed with the Short-Form 36. Premorbid risk for anxiety/neuroticism was evaluated with a polygenic risk score derived from a large genome-wide association study meta-analysis. In multivariate mixed models, having higher levels of PTSS, poorer self-rated health, lower income, and less education at age 38 were associated with worse physical and mental HRQOL two decades later. Chronic health problems at age 38 predicted midlife physical but not mental HRQOL. Although genetic risk for neuroticism was correlated with HRQOL and PTSS, it was no longer significant in multivariate models. Health-related quality of life (HRQOL) predicts morbidity and mortality independently of objective health measures; early interventions may help to mitigate the ongoing impact of trauma on quality of life.