Magnetic fields can have profound effects on the motion of electrons in quantum materials. Two-dimensional electron systems subject to strong magnetic fields are expected to exhibit quantized Hall conductivity, chiral edge currents and distinctive collective modes referred to as magnetoplasmons and magnetoexcitons. Generating these propagating collective modes in charge-neutral samples and imaging them at their native nanometre length scales have thus far been experimentally elusive. Here we visualize propagating magnetoexciton polaritons at their native length scales and report their magnetic-field-tunable dispersion in near-charge-neutral graphene. Imaging these collective modes and their associated nano-electro-optical responses allows us to identify polariton-modulated optical and photo-thermal electric effects at the sample edges, which are the most pronounced near charge neutrality. Our work is enabled by innovations in cryogenic near-field optical microscopy techniques that allow for the nano-imaging of the near-field responses of two-dimensional materials under magnetic fields up to 7 T. This nano-magneto-optics approach allows us to explore and manipulate magnetopolaritons in specimens with low carrier doping via harnessing high magnetic fields.