How does the brain identify stimuli that are relevant for predicting important events and how does it distinguish spurious relationships from truly predictive ones? We examined two contrasting theoretical frameworks: in the first, learning proceeds by considering a fixed hypothesis of the environment’s statistical structure (the set of predictive and causal relationships) and adjusting strength parameters for these relationships to optimize predictions. In contrast, the second approach directly assesses ambiguity in predictive relationships by evaluating multiple hypothesis of the environment’s statistical structure. We compared these frameworks in an animal model of aversive conditioning, allowing us to also manipulate the underlying brain systems. We show that when facing novel predictive stimuli, rats initially adopt a structure learning strategy, but switch to updating parameters during subsequent learning