Digital Arts and Culture 2009

This paper explores the use of language processing technologies for interactive artwork and studio art production. I consider text in multiple roles: as data, as index, and as a medium for interaction. After describing initial efforts with a dysfunctional chatbot, I discuss my recent work with language processing in the creation of studio art objects, and speculate about the extension of those techniques to address the large corpora of personal media we accumulate online.

Shrewsbury Museum Service invited Dew Harrison to create a work relating to Charles Darwin for the bicentenary of his birth in the UK town. Her research is practise-led and uses computer technology to interlink series of related thoughts and ideas, in multimedia form. Texts, images, animations and sounds are networked into one overarching ‘concept’. The complete concept is then exhibited as a looped projected film or interactive screen work offering a contemporary understanding of a complex issue. She had previously worked with the ideas encapsulated within the work of Marcel Duchamp, in particular his Large Glass, which she transposed together with his boxes of notes and associated previous work, into one hypermedia system. Duchamp being the instigator of current Conceptual practice, his thinking began the shift of value within art from aesthetic to idea. This new challenge was to explicate the ideas of Darwin by synthesising them into one concept which could be grasped through audience interaction. Harrison is interested in relational works that invite an audience to participate together in revealing an understanding of the ‘concept’ on display. Earlier works used mouse and keypad to access a work, now the interfaces can involve sensors and physical movements for more playful and instinctive engagement. To develop the new project, Harrison worked in collaboration with two programmers and an animator to explore the ‘big idea ‘ of evolution and elicit an understanding of Darwinian adaptation through interactive installation.

For the new installation entitled Shift-Life they have produced an alternate, or fantasy, biological life as a project which delivers an implicit understanding of Darwinian evolution and examples the rapid life changes necessary for survival in accelerated alternating climatic conditions. Shift-Life is an installation which focuses on ‘hands-on’ possibilities for witnessing an evolutionary process in alternate life forms as they struggle to adapt to a volatile environment. In response to Darwin’s idea, the aim of this work was to create an ‘alternate’ biological life as a set of artificial or virtual organisms that possess similar biological processes to their ‘real’ counterparts, such as growth, reproduction, and adaptation. The virtual life forms exist in a nutritional (trophic) relationship of prey/predator, and include both rooted (sessile) and free ranging (vagile) organisms. Animal-intelligence was programmed into the virtual organisms to allow them survival strategies. The project also involved the construction of an enhanced mixed reality-based virtual environment to support the organisms. The climate of the virtual environment was directly influenced by the data gathered by wireless sensors (phidgets) in the real world landscape (sand box), plus implements (lights, shakers, pourers…) that altered the parameters (temperature, humidity, acidity, stability…) and so allowed visitors to change the condition of the virtual landscape.

The installation comprised of a large ’sand-pit’ box representing the virtual world terrain, this encouraged interactivity for visitors who could physically manipulate a set of implements to radically alter the living conditions of the fantasy creatures in their virtual ecosystem, projected into the installation space. By pouring liquids, switching on lights, moving objects etc., in the sand box, visitors could see immediate responses to their actions played out in the animated ecosystem as the life forms adapted to survive. Interacting with the real world landscape and observing the instant affect a visitor’s actions had on the animated ecosystem projected into the installation space, proffered an understanding of how causing changes in environmental conditions, forces evolutionary developments on the life-forms in them.

2009 is the 200th anniversary of the birth of Charles Darwin in Shrewsbury and the 150th anniversary of the publication of On the Origin of Species. As part of the national celebrations underway, Shift-Life was exhibited at Shift-Time – a festival of ideas in Shrewsbury, summer 2009. It was still in its prototype stage and, following this beta-testing, it will be modified and enriched with extra behaviours and more sensitive environmental changes as we develop the project to more closely demonstrate Darwinian ideas for further exhibition.

The authoring of interactive, generative narrative is a task that typically requires an extensive multi-disciplinary background in computational and narrative theory. Wide Ruled is an authoring tool that aims to address this problem by providing a friendly, intuitive, story-centric interface to an author-goal driven textbased story planner. Over the past two years, this system has been used repeatedly by technical and non-technical users in multiple classroom settings, and evolved into a widely used and publically available story authoring system. In this work, we describe the successes and failures of Wide Ruled, and how it provides a critical evolutionary step in developing a truly usable, writerfriendly, and practical interactive story authoring environment.

I discuss the four Perl poetry generators I have developed in the ppg256 series. My discussion of each program begins with the entire 256 characters of code and continues with an explication of this code, a description of aspects of my development process, and a discussion of how my thinking about computation and poetry developed during that process. In writing these programs, I came to understand more about the importance of framing to the reception of texts as poems, about how computational poetic concepts of part of speech might differ from established linguistic ones, about morphological and syntactical variability, and about how to usefully think about possible texts as being drawn from a probability distribution.

There has been considerable effort over the past 10 years to define methods for preservation, documentation and archive of new media artworks that are characterized variously as ephemeral, performative, immersive, participatory, relational, unstable or technically obsolete. Much new media cultural heritage, consisting of diverse and hybrid art forms such as installation, performance, intervention, activities and events, are accessible to us as information, visual records and other relatively static documents designed to meet the needs of collecting institutions and archives rather than those of artists, students and researchers who want a more affectively vital way of experiencing the artist’s creative intentions. It is therefore imperative to evolve existing preservation strategies for new media art, to include material that provides a rich sense of what it was like to be dynamically present in the live artwork. This paper proposes that simulation strategies with the aesthetic, mechanics and dynamics of the videogame platform, are capable of delivering complex user experiences that can be applied to new media artworks for the purpose of providing users, who may never see the original artwork, with an appreciation of the artists full creative intentions. We further propose that such an approach should be differentiated from the documentation or recording of a particular instance of artwork. We suggest that new creative iterations or versions of new media artworks be designed specifically for this medium through collaboration between artists and game designers and/or developers. Such novel interactive iterations of new media artworks should be conceived so as to enable otherwise temporary artworks to persist in a different form into the future so they may inspire the next generation of artists and inform criticism and research through a direct interactive engagement with the art.

The model of videogames is one marked by diversity and flexibility of means that can exceed the verisimilitude required of simulation or direct mimesis of the live artworks. The creation of a new version of an artwork, which in itself is a construct of the creative imagination, can include virtual worlds and/or mixed reality environments and employ diverse interfaces, emergent behaviours using artificial intelligence and logic engines to fulfill conceptual and exploratory levels of immersion, agency and user experience. The application of videogame design and technology to artworks raises questions of theoretical and practical concern having to do with their desirability as a means of artistic expression, the potential capacity of such re-presentations to deliver affective experience of an artwork as well as their ability of extend the life of new media artworks.

Drawing upon cognitive science theories of conceptual blending and material anchors, as well as recent neuroscience results regarding mirror neurons, we argue that animated visual graphics, as embodied images whose understanding relies on our perceptual and motor apparati, connect both material and mental notions of images. Animated visual images mobilize a reflective process in which material-based imaginative construction and elaboration can take place. We call this process as “material-based imagination,” in contrast to the general notion of imagination as purely a mental activity. This kind of imagination is pervasive in today’s digitally mediated environments. By analyzing a range of digital artifacts from computer interfaces to digital artworks, we show the important role of imaginative blends of concepts in making multiple levels of meaning, including visceral sensation and metaphorical narrative imagining, to exemplify expressiveness and functionality. The implications of these analyses collectively form a step toward an embodied cognition approach to animation phenomena and toward recentralizing understanding of artistic and humanistic production in cognitive research.

In this paper we discuss the use of speculative design as an approach to developing technological fluency. We provide overviews of technological fluency and speculative design, trace their conceptual connections, and then outline the use of a speculative design approach to technology fluency programs, providing an example from a current project. We then conclude by discussing how a speculative design approach can extend the idea of technology fluency towards new directions: broadening common understandings of the practices of technology development and adding a dimension of criticality.

Rapid changes in science, technology and new media will lead to more sophisticated ideas about what it means to be human, in thought, body, emotional response and artistic expression. New relationships will form between humans, machines and animals with the human functioning as a networked resource that can be accessed globally over the internet.

This paper documents both the technical and theoretical development of the collaborative interactive new media video project “The Emotions (after Charles Darwin)” which explores some of the above concepts. “The Emotions” first tries to establish the existence of the universality of emotions at a biological level, as empirically measured and documented by the results of the control group (non-autistic subjects, as the goal is to document “normal”, i.e. universal emotional response) at the Brain Mind Institute in Switzerland. Secondly, it suggests the potential for subsequent futuristic misuse through genetic and or technological modification (demonstrated by the observer’s ability to interactively modify or transform a given emotion’s video stream at will).

Writing practices that integrate dynamic and interactive strategies into the making and reading of digital texts are proliferating as more of our reading experiences are mediated through the screen. In this paper we argue that rarely do current approaches to creating digital texts operate at the basic textual level, that of the letterform itself. We argue further that such neglect is partially the result of the fact that current font technology is based on print paradigms that make it difficult to work programmatically at the level of individual letters. We then discuss work that has been made with software produced in our lab that suggests the creative possibilities in being able to easily specify behaviours at such a level. We conclude by proposing that writers, typographers and programmers start thinking beyond Postscript-like formats such as OpenType or TrueType to collaboratively develop a new ComplexType format (or formats) that is designed for the twentyfirst century as opposed to a simulation of the fifteenth.