Designing for Engaged Experience

Jim Bizzocchi’s practice explores the creative possibilities of video as ambient background:

Ambient video art is an emergent form, simple to describe, but difficult to achieve. Ambient video works are designed to play in the background of our lives, yet they must be ready to reward our attention in any given moment. Like Brian Eno’s ambient music, they “must be as easy to ignore as notice”. However, whenever they are noticed, they must always yield visual pleasure. In this capacity, they must be equally proficient at rewarding a fleeting glance, a more direct look, or a longer contemplative gaze.

[ambient video website]

We are inspired by the potential of Jim’s concept in terms of that *extra-ordinary object* – the media data produced as part of ethnographic methodologies – and the potential of the extra-ordinary object to become a remediated new object in its own right:

Current Album: Jim Bizzocchi's images
Images:

Current Album: Jim Bizzocchi's images
Images:

There has been much recent discussion on place and especially on experiential richness (or the lack thereof) in digital places [11]. Yet, apart from counterfactual game-worlds, there seems to be little progress in developing digital environments that improve situated historical understanding, a sense of place, and an understanding of how people with different cultural frameworks experience that place. So, is new media unable to live up to the hype or do we need to rethink how historical and cultural understanding is most effectively transmitted via new technology? What are the requirements for a digital place to support historical understanding? Must it be persistent, mark-able, socially reconfigurable or reconfiguring? Could interaction actually obstruct and impede the appreciation of place and history? Most importantly, how can we learn about the views and cultural understanding of others?

A number of ways have been suggested in which different experiences of place can be evoked and transmitted. Firstly, via biofeedback, environments can be created whereby indirect and automatic participant responses are dynamically encoded into the environment. Not only does biofeedback allow for immediate and intuitive and immersive interaction, it allows for cultural and social filtering. Digital media can transfigure participant reactions in a more nuanced and subtle fashion than via the keyboard and mouse. Imagine your friends could trigger your aversions via digital media, and your bodily reactions could portray (or even betray) your innermost fears and phobias. To what extent would such social interaction turn into anti-social interaction? And to what extent could this technology be used positively, are there any guidelines to help direct us? And in light of the conference theme, could biofeedback be used both to convey additional senses of place, and to convey cultural aspects of place perception and place experience? This is not a pipe dream. A biofeedback environment has been created and evaluated it and there are ongoing evaluation issues we wish to discuss at the workshop. We might also wish to ponder ethical issues of biofeedback.

The second project is the cultural Turing test. A scenario example was begun but not completed in 2006. It will be revisited it in the near future. It involves reversing and reapplying the conventional view of Turing’s test of artificial intelligence, but in the context of cultural heritage. Instead of the participant evaluating and judging the artificial intelligence of the non-playing characters, what if they judged the intelligence and social attunement of the participant? Consider a role-playing game, the human participant must convince the NPCS he or she is actually a local. The NPCs are suspicious. The human participant must develop their sense of situated mimicry, copying the situated behaviour and socially framed responses of the local characters in order to avoid detection. How could this be evaluated for effectiveness, learning, and for fun? Are there historical or cultural scenarios that easily lend themselves to this method?

How can we draw upon a visitor’s familiar experience of a natural place so as to inspire the design of a digital technology that might mediate others’ experiences of that place? Whilst digital technology might never transfer the sense of embodiment in a place to those who have no in situ experience of the place, could it be used to augment the otherwise often flat and somewhat sterile depictions of place, for example, that one might come across in a tourist guide?

Egocentrically depicting visual resources supports interpreting the situated experience, as similarity between the speaker’s and listener’s eye-movements in viewing scenes promotes comprehension [4]. So egocentric Point of View (egoPOV) video can depict both fleeting and temporally-ongoing experiences that contribute to creating meaning during spatially-embedded interaction.

EgoPOV video was used to capture some of the meaning associated with a natural, unbuilt place (Alligator Creek in north Queensland) as visitors familiar with the location talk about why the place is special to them [5]. This was intended to capture reflexive tailoring of deictic communications with physical contingencies to construct meaning. The focus was different participants’ accumulated experience of places that had been visited regularly in the past. Natural places, that hold memories, enable uncovering ways that people recreate and augment place in managing space in subsequent visits [6]. This proposition was extended by framing the activity in participants’ accounts of the place in their personal history. The researcher and participant were co-located to provoke articulating references to physical resources. Participants were later interviewed, using the egoPOV video to assist recall. Analysis of the original video and later interviews is currently being used to provoke ideas and inspiration for design.

‘Strolls’ is an ongoing project inspired by research into the representation of experience and the problematic of REpresentation and remediation [7] of interaction designing. Transmission or sharing of experience is complex, the stuff of Art and Culture, subjective (walk a mile in my shoes) and ultimately enticing. Tolstoy argues for the goal of art being the handing on of experience; Huizinga [8] asks for the sense of being there, living that life; and Foucault notes the impossibility of the same across epistemes, his denial implying the desire:

“In the wonderment of this taxonomy, the thing that … is demonstrated as the exotic charm of another system of thought, is the limitation of our own, the stark impossibility of thinking that.” (1973 xv)

A number of projects seek to explore this desire within the rich three dimensional visualisation engines, typically used to produce game worlds (see [9]). The apparently simple dictum of “show, don’t tell” disguises an interesting nexus of issues and questions, particularly in the ramifications for cultural dimensions and participatory interaction design. Research into dealing with these two core issues led us to clarifying two distinct design parameters: firstly that it is essential to enable a direct conduit for the tellers of stories; and second, that the underlying design metaphor is a critical element in the subsequent shaping of the stories and experience.

The “Stroll” concept arises as an adjunct to this research. The first stroll was produced in order to share the experience of one’s immediate locale. The aim was to capture a vignette of daily life and mundane experience of the place as lived [10] rather than the iconic imagery of the public perspective. Subsequent sharing of the initial “sunday morning in rosalie” stroll inspired colleagues to make a similar record. Others were also moved to explore their own environs in a similar mode and we now have a developing public website where people are sharing their own small stories of place.

The project is successful in terms of its initial design metaphor – the stroll – but what is really interesting is that the data itself now manifests potential to become an object of interaction and offers opportunity for further designing.

Stroll Website