Individuals with autism spectrum disorder (ASD) are commonly involved in interventions aimed at improving communication or other social communicative behaviors. However, the field of ASD intervention research faces significant limitations in its current practices of measuring treatment effectiveness. There is a need for the development of outcome measures that adequately address the limitations of the measures historically used and that can reliably detect changes in the social communicative behaviors of individuals with ASD, especially in a short period of time. The aim of the present study was to determine the utility of the BOSCC-Phrase Speech Young Fluent (PSYF) as an outcome measure of treatment response. Specifically, this study analyzed the factor structure of the measure, examined its initial psychometric properties, and provided evidence of its utility as a measure of change. The BOSCC coding scheme was applied to 345 video administrations from 160 participants diagnosed with ASD. Participants included individuals of any age with consistent and flexible phrase speech, or individuals under the age of 8 years with fluent, complex sentences. The BOSCC-PSYF has an underlying three-factor structure. Test-Retest reliability was good for the Early Communication domain, moderate for the Social Reciprocity/Language domain, and poor for the RRB domain. Inter-rater reliability was good for the Early Communication and Social Reciprocity/Language domains and fair for the RRB domain. Significant changes occurred over time in the Early Communication and Social Reciprocity/Language domains, and Core Total scores. Standardized effect sizes of change were larger in the BOSCC domains than in ADOS CSS and VABS Communication Standard Scores. The BOSCC provides a standardized, flexible, and minimally biased assessment of social communication changes in response to treatment. Its validation would have important implications for ASD intervention research, including the possibility of a low-cost measure that reliably measures changes in broad social communicative behaviors in a short period of time, can be conducted and coded by individuals of various skill levels, and is flexible enough to be used across various sites/studies.