There is growing evidence for the influence of plant intraspecific variation on associated multi-trophic communities, but the traits driving such effects are largely unknown. We conducted a field experiment with selected genetic lines of the dioecious shrub Baceharis salicifolia to investigate the effects of plant growth rate (two-fold variation) and gender (males vs. females of the same growth rate) on above- and belowground insect and fungal associates. We documented variation in associate density to test for effects occurring through plant-based habitat quality (controlling for effects of plant size) as well as variation in associate abundance to test for effects occurring through both habitat quality and abundance (including effects of plant size). Whereas the dietary specialist aphid Uroleucon macaolai was unaffected by plant sex and growth rate, the generalist aphid Aphis gossypii and its tending ants (Linepithema humile) had higher abundances and densities on male (vs. female) plants, suggesting males provide greater habitat quality. In contrast, Aphis and ant abundance and density were unaffected by plant growth rate, while Aphis parasitoids were unaffected by either plant sex or growth rate. Arbuscular mycorrhizal fungi had higher abundance and density (both marginally significant) on females (vs. males), suggesting females provide greater habitat quality, but lower abundances (marginally significant) and higher densities on slow- (vs. fast-) growing genotypes, suggesting slow-growing genotypes provided lower resource abundance but greater habitat quality. Overall, plant sex and growth rate effects on associates acted independently (i.e., no interactive effects), and these effects were of a greater magnitude than those coming from other axes of plant genetic variation. These findings thus demonstrate that plant genetic effects on associated communities may be driven by a small number of trait-specific mechanisms.