When people act on objects, their goals can depend on the objects’ intrinsic properties and conventional uses (e.g., using forks, not knives, to eat spaghetti), locations (e.g., clearing the table, regardless of what is on it), or both (eating with the fork next to your plate, not your dining partner’s). For adults, objects’ intrinsic properties matter more than their locations in most action contexts. Whereas 5-month-old infants privilege objects’ intrinsic properties in attributing goals to people reaching for objects, 3-month-old infants do not. Do younger infants fail to view reaching as goal-directed, or are they uncertain which properties of objects are relevant in different contexts? Here we show that 3-month-old infants attribute goals to others’ reaching actions when given information that their actions depend on what, not where, an object is. Our findings suggest that 3-month-old infants can learn about others’ object goals, before they reach for objects themselves.