For decades, rhizobia were thought to be the only nitrogen-fixing inhabitants of legume nodules, and biases in culture techniques prolonged this belief. However, other bacteria, which are not typical rhizobia, are often detected within nodules obtained from soil, thus revealing the existence of a phytomicrobiome where the interaction among the individuals is not only complex, but also likely to affect the behavior and fitness of the host plant. Many of these nonrhizobial bacteria are nitrogen fixers, and some also induce nitrogen-fixing nodules on legume roots. Even more striking is the incredibly diverse population of bacteria residing within nodules that elicit neither nodulation nor nitrogen fixation. Yet, this community exists within the nodule, albeit clearly out-numbered by nitrogen-fixing rhizobia. Few studies of the function of these nodule-associated bacteria in nodules have been performed, and to date, it is not known whether their presence in nodules is biologically important or not. Do they confer any benefits to the Rhizobium-legume nitrogen-fixing symbiosis, or are they parasites/saprophytes, contaminants, or commensals? In this review, we highlight the lesser-known bacteria that dwell within nitrogen-fixing nodules and discuss their possible role in this enclosed community as well as any likely benefits to the host plant or to the rhizobial inhabitants of the nodule. Although many of these nodule inhabitants are not capable of nitrogen fixation, they have the potential to enhance legume survival especially under conditions of environmental stress. This knowledge will be useful in defining strategies to employ these bacteria as bioinoculants by themselves or combined with rhizobia. Such an approach will enhance rhizobial performance or persistence as well as decrease the usage of chemical fertilizers and pesticides.