Immune checkpoint inhibitors (ICI), such as pembrolizumab, nivolumab, durvalumab and ipilimumab, have significantly enhanced survival rates for multiple cancer types such as non-small cell lung cancer, melanoma, Hodgkin lymphoma, and breast cancer, and they have emerged as an adjunct or primary therapy for malignant disease. Approximately 40% of patients with cancer on ICI therapy experience side effects called immune-related adverse events (irAE). While not the most common, pulmonary toxicities can be rapidly progressive, potentially fatal, and pose a three-fold increased risk for requiring intensive care unit-level of care. Pneumonitis is a focal or diffuse inflammation of the lung parenchyma, and clinical manifestations may be highly variable. While the onset is generally observed 6–12 weeks after the initiation of therapy, drug toxicity can develop rapidly within days after the first infusion or many months into therapy. Pneumonitis symptoms can be subtle or non-specific; therefore, a thorough and systematic evaluation considering other possible etiologies is crucial. Moreover, extrapulmonary findings, such as skin lesions, colitis, or endocrinopathies, should raise suspicion for irAE as drug toxicity can affect multiple organs simultaneously. Due to the significant overlap of clinical features between ICI-associated pneumonitis and respiratory infections, it can be challenging to differentiate the two conditions based on clinical presentation alone. A multidisciplinary approach to management is recommended for the treatment of ICI-associated pneumonitis, and classification of severity helps to guide interventions. Treatment options in more severe cases include systemic immunosuppression. Given the increased use of ICIs and greater probability that patients with ICI-associated pneumonitis will be seen in the emergency department, we aimed to provide a comprehensive framework for the diagnosis and management. In addition, identifying potential challenges in diagnosis and/or other contributors of respiratory symptoms and radiographic manifestations is highlighted.