When people plan, they often do so in the face of uncertainty. However, little is known about how uncertainty affects planning. To study these effects, we used a reward gathering task in which the we varied the reliability of announced rewards varied from certain to completely random. We quantitatively compared several planning models. We found that participants used a suboptimal approach, failing to directly incorporate stochasticity into their planning. Instead, they "compensated" for uncertainty by decreasing their planning effort as stochasticity increased. First-move response time correspondingly decreased with increasing stochasticity. Our findings generalized to a manipulation of transition uncertainty. Together, these findings open the door to a more comprehensive and computationally grounded understanding of the role of stochasticity in planning.