Variants at 21 genetic loci have been associated with an increased risk for Alzheimer's disease (AD). An important unresolved question is whether multiple genetic risk factors can be combined to increase the power to detect changes in neuroimaging biomarkers for AD. We acquired high-resolution structural images of the hippocampus in 66 healthy, older human subjects. For 45 of these subjects, longitudinal 2-year follow-up data were also available. We calculated an additive AD genetic risk score for each participant and contrasted this with a weighted risk score (WRS) approach. Each score included APOE (apolipoprotein E), CLU (clusterin), PICALM (phosphatidylinositol binding clathrin assembly protein), and family history of AD. Both unweighted risk score (URS) and WRS correlated strongly with the percentage change in thickness across the whole hippocampal complex (URS: r = -0.40; p = 0.003; WRS: r = -0.25, p = 0.048), driven by a strong relationship to entorhinal cortex thinning (URS: r = -0.35; p = 0.009; WRS: r = -0.35, p = 0.009). By contrast, at baseline the risk scores showed no relationship to thickness in any hippocampal complex subregion. These results provide compelling evidence that polygenic AD risk scores may be especially sensitive to structural change over time in regions affected early in AD, like the hippocampus and adjacent entorhinal cortex. This work also supports the paradigm of studying genetic risk for disease in healthy volunteers. Together, these findings will inform clinical trial design by supporting the idea that genetic prescreening in healthy control subjects can be useful to maximize the ability to detect an effect on a longitudinal neuroimaging endpoint, like hippocampal complex cortical thickness.