Communication is highly overloaded. Despite this, even young children are good at leveraging context to understand ambiguous signals. We propose a computational account of overloaded signaling from a shared agency perspective which we call the Imagined We for Communication. Under this framework, communication is a way for cooperators to coordinate their perspectives, allowing them to act together to achieve shared goals. We assume agents are rational, utility maximizing cooperators, which puts constraints on how signals can be sent and interpreted. We implement this model in a set of simulations which demonstrate this model’s success under increasing ambiguity as well as increasing layers of reasoning. Our model is capable of improving performance with deeper recursive reasoning; however, it outperforms comparison baselines at even the shallowest level of reasoning, highlighting how shared knowledge and cooperative logic can do much of the heavy-lifting in language.