There are no single-player videogames. When someone plays a game, they are not only interacting with the game's contents, but also engaging with the culture and connections that are built around that game. This channel by which people connect to one another gets cut off when the design of the game leaves a player feeling helpless and excluded. Informed by approaches from game studies, human computer interaction, and artificial intelligence (AI) research, this dissertation explores how we can fight this unintended exclusion.

From a theoretical standpoint, this work expands our vocabulary of failures in games, allowing us to differentiate in-loop from out-of-loop failures. It teaches how to design AI-powered assistance methods, allowing us to repurpose game-playing AI agents in service of game inclusivity. From a technical standpoint this work contributes three prototype systems: a dynamic tutorial framework, enabling games to respond to the skill levels of players; an AI intervention for motor accessibility, helping players to adapt dexterity based games for deliberative play; and a reinforcement learning-based navigation agent, enabling access to a portfolio of assistance methods.

As a whole, this dissertation constitutes a step towards making games more inclusive, not for the sake of games, but for the sake of the people who play them.

This project demonstrates a new kind of drawing tool with a novel interface. Recent research in machine learning for image generation has mainly been focused on producing high-quality results and its efficiency in doing so. There was little consideration for how these systems should be interacted with by their actual intended users. My project offers an interface allowing for users to generate images by specifying details in a composition using high-level textual descriptions. To validate this design, I conducted two studies involving expert artists, designers, and UX practitioners.

Open AI's Universe, intended to foster the development of general AI agents that could use a computer like humans do, fell short in a number of ways. As a result, attention shifted to smaller sets of environments such as Gym Retro or the Arcade Learning Environment. My thesis shows a way to overcome several of the limitations in the design of Universe by abstracting the execution of an interactive application in a full-system virtual machine. By using virtual machines, we regain the broad scope of the original Universe project, as well as the ability to pause, save, and restore at arbitrary moments of interaction, allowing the usage of several sophisticated exploration algorithms. I describe an API that implements the virtual machine strategy using Oracle's Virtual Box and ways to extend it with a framework for sensors and input actions that allow a broad range of manipulations of the virtual machines. I then walk through experiments designed to characterize the interaction latency and jitter associated with my implementation, as well as analyze the space costs associated with these large VMs and their associated save points. Finally, I demonstrate the utility of this implementation by showing automated play of two games.

Procgen is the field of making things that make things. Also known as procedural content generation (PCG), and overlapping with generative design and generative art, procgen is a growing field of inquiry in academic contexts and the focus of thriving scenes of craft practice. Procgen is frequently approached as if it was about getting something out of nothing. By contrast, this dissertation is founded on the observation that all information that comes out of a generative system must first have been inserted into the generative system, including information encoded into it by virtue of how the system was assembled from parts. Interpreting the compositional parts of a generative system, we can trace the relationship between inserted input and emergent output. Understanding how the modular compositional operations work in isolation makes it easier to recombine them into new generative systems.

My dissertation investigates the modularity of procgen via three research questions centered around aesthetics, composition, and applications. How does the structure of a generative system impact the aesthetic effect of the output? What are the lower level components that make up a generative system, and how do those components fit together? What new applications of generative systems are unlocked by taking a compositional view of their structure?

My work recomposes our collective understanding of procgen by building technical systems, applying humanistic interpretation to understand those systems in practice, and illuminating the connections between what we build and how we interpret what we build. I apply this method to existing generators, including some seen as canons in the field. I also assemble new generators using new capabilities emerging from the artificial intelligence literature. Taken as a whole, this dissertation establishes a practice of recomposing procgen: taking apart our understanding of procgen and putting it together again in new ways.

Media like videogames (and novels, film, music, etc.) play an important role in most people’s lives, and creation and comprehension of these works benefit from critical study. Scholarly analysis of media has been facilitated by information retrieval technology, which not only saves time, but also makes it possible to ask new kinds of questions. These benefits are relevant for any students or educators whose work includes the creation and interpretation of interactive media. However, current approaches treat games as monolithic artifacts because they are not capable of ingesting and crawling the content of games, and even modern search engines are better at capturing secondary text-based sources about games such as criticism, guides, and reviews. Moments that can take place within a game could instead be treated like pages in a book or like sites on the web. In this dissertation I describe research that identifies the needs of scholars and other experts for videogame moment retrieval, the development of two interactive systems that can meet a subset of those needs, and a publicly available corpus of student games that can enable artificial intelligence, information retrieval, and digital humanities research. My work contributes a new technical method to game studies, and a new domain of complex and culturally relevant data to information retrieval. Finally, my work contributes to game education with the design of search and visualization tools that could be used to analyze student game projects, to inspire future student work, and potentially to be incorporated directly into future student work processes.

Videogames are software systems expressed in structure code and data, but it’s far from obvious how changing one bit of code will impact the experience a human player might have with the game. Traditional playtesting and quality assurance testing remains the gold standard for ensuring a quality gaming experience, however there are considerable resource and morale requirements necessary to ensure that testing procedures will unveil problems in the user experience. Further, these testing traditions do not scale down to incremental, in-development, builds of videogame software, placing the entirety of videogame quality control at the ends of vast periods of coding.

With the advent of advanced AI gameplaying algorithms, I present a way to leverage gameplaying AI as an assistive tool for game developers. Rather than playing to win, these AI techniques aim to explore the playable areas of a videogame that a human player could encounter, potentially encountering areas of gameplay that developers did not intend to implement. I have developed two exploration techniques, one which relies on human gameplay traces, and one which self-improves with only a single seed of human gameplay. With this information, I also present a visualization workflow to generate visual reports of gameplay differences between versions of a videogame.

Software instruments are intelligent interfaces used by domain experts to manipulate complex artifacts. By contributing the concept of software instruments, this thesis connects the field of data visualization to its relations in both artificial intelligence and computer-supported collaborative work. Software instruments arise from projects carried out by teams with complex datasets in aerospace and narrative design, whom I have worked closely with, and which are developed and documented in this thesis. They are a lens for other computational media practitioners to introspect about their own design processes and artifacts. I argue that the evaluation of software instruments occurs in discussion with expert stakeholders, adopting the lens of critical technical practice to avoid disciplinary silos and narrow strategies of evaluation.

In this thesis, I describe a Monte Carlo Tree Search (MCTS) powered tool

that I created to help assess the impact of various design choices for in-development

games built on the Unity platform. MCTS shows promise for playing many games,

but the games must be engineered to oer a compatible interface. To circumvent this

obstacle, I developed a support library for augmenting Unity games, as well as exper-

iment templates in Jupyter Notebook for running machine playtesting experiments. I

also propose ways for designers to use this tool to ask and answer designs questions. To

illustrate this, I successfully integrated the library with It's Alive!, a game I am cur-

rently developing, as well as 2D Roguelike, an open source tutorial game available from

the Unity asset store.3 The integration took fewer than 100 lines of code (see Appendix

B). I demonstrate the tools capability to answer both game design and player modeling

questions, as well as provide the results of the system validation experiments.