UC Irvine

Social communication is one of the key components of successful interaction. People with autism can have significant challenges with social communication, resulting in some of the highest rates of depression and anxiety. In fact, young adults with autism have suicide rates that are 28 times higher than the general population. Thus, supporting social skills of people with autism could have a positive impact on both the social and mental wellbeing of individuals with autism. Although much research has focused on supporting social skills broadly, little attention has been paid to developing effective nonverbal behaviors, which are necessary to initiate, maintain, and gracefully terminate a social interaction.

The aim of the dissertation work is to design and evaluate the effect of realtime visualizations of prosody and proximity. To this end, the research questions are: 1. Does visualizing nonverbal behavior increase the percent of intervals users demonstrate normative proximity and prosody during neurodiverse interactions, and 2. What factors surrounding technological social skills intervention impact its efficacy and acceptance? These research questions are answered through three lab-based experiments that include measuring prosody and proximity in controlled and experimental conditions, as well as interviewing the participants and family members about their experience with these novel technologies.

By using sensory perceptual strength associated with an autistic profile, (i.e., superior visual perception regarding details), I have designed and evaluated three technological systems to assist people with autism to engage in socially expected behavior during a brief conversation in a laboratory setting. The single-case experiments show that visualizations of real time feedback improve nonverbal communication during social interactions for close to half of the participants (i.e., young adults and children with autism). The results from the interviews with participants and parents about their experiences highlight issues of usability, learnability, and comfortability of the systems. Deeper analysis of these combined findings culminates in an assistive technology design concept-Sensory Accommodation Framework-which provides four strategies for supporting sensory perception differences through computation. The contributions to this work are: the empirical findings from the three evaluation studies, the design guidelines from the design activities, and the conceptual framework.