Research on social inferences demonstrates that when thinking about minds similar to our own, we anchor and adjust away from ourselves (Tamir & Mitchell, 2013). However, research on relational self theory (Andersen & Chen, 2002) suggests the possibility of using knowledge about others as an anchor when they are more similar to a target than ourselves. We investigated whether social inferences are made on the basis of significant other knowledge through an anchor and adjustment process, and whether this ability would be reduced under load. Participants answered questions about their likes and habits, as well as the likes and habits of a significant other, a target similar to their significant other, and a yoked control. We found that differences between the significant other and similar target were related to participants’ reaction time, and found the opposite effect for self and target differences, suggesting anchoring and adjustment from the significant other rather than the self. However, inferences about the others tended to be more similar to the self under load, suggesting that the self serves as the primary source of information about others.