Interaction as Performance: Cases of Configuring Physical Interfaces in Mixed Media

Abstract
Mixed media, as artful assemblages of digital objects and physical artefacts, provide distinctive opportunities for experiential, presentational and representational interaction. In projects based learning of architecture design, participants staged spatial narratives with multiple projections, performed mixed objects and artefacts, and exploited bodily movements in mixed representations. Cases show, how physical interfaces in mixed media, acquire a spatial dimension, integrate physical artefacts and bodily movements, and propose configurability as a central feature. A perspective, based on anthropological concepts of performance, makes it possible to address these aspects in a coherent way, pointing to sense experience, to the individuality and collective emergence of expression, and its diachronic and event character. Under this perspective, interaction is part of expressive events aimed at generating new insights for participants (interchangeable performers and spectators) privileging sense experience. Events are the outcome of configurations of space, artefacts and digital media, and are characterised by a simultaneousness of doing and undergoing, of bodily presence and representation. More importantly the performance perspective suggests a particular temporal view on interaction, based on the concept of event, addressing a neglected granularity of analysis between the moment by moment unfolding of interaction and the longer term co-evolution of technology and practice. Implications of interaction as performance contribute to a wider program of interaction design providing alternatives to established human-computer interaction tenets: the notion of event as an alternative to the notion of task; perception in Dewey’s terms can be privileged over recognition proposing expression as an alternative to accountability and usability. Implications include looking at how space can be configured and staged instead of measured or simulated, how situations can be staged instead of sensed and recognised, privileging the sensing human to the sensing system.

CV
Giulio Jacucci, PhD is a senior researcher at the Helsinki Institute for Information Technology (www.hiit.fi), a joint research institute of the two leading research universities in Helsinki, the University of Helsinki (UH) and the Helsinki University of Technology (HUT). The present projects cover mobile and ubiquitous computing, user experience research, complex system computation, mobile content communities, semantic computing, and digital economy. Giulio Jacucci has worked at the University of Oulu where he is about to complet a PhD applying an anthropology of performance to the design of mixed media spaces. He has published over 20 journal and conference papers (some with the surname Iacucci) including technological works as well as ethnographic studies in the areas of ubiquitous computing, tangible interfaces, mixed reality applications and mobility. He has participated in a variety of international research projects including Atelier (part of the EU Disappearing Computer Initiative) building a mixed media environment for inspirational learning, where among other things he lead the development of a location-aware mobile application, eDiary, to support recording and re-experiencing of remote site visits for architects. His current research focuses on ubiquitous multimedia and changing practices of configuring-perceiving expressions in communicative events.