Hampi-Live was the second stage of the Australian Research Council Linkage Project grant, ‘Reformulating narrative in virtual heritage using a co-evolutionary model of immersive interactivity’. The work exploits the technological and expressive features of UNSW iCinema Centre’s Advanced Visualisation and Interaction Environment (AVIE) including camera tracking, omni-stereoscopic projection and 36 channel spatialised sound.

Hampi-Live augments the original stereoscopic panoramic content of PLACE - Hampi—a work navigable in AVIE using a custom joystick interface to explore the potentialities of co-evolutionary narrative wherein audience behaviour can influence and be influenced by the real-time behaviour of machine agents.

This experimental research results in two scenarios that exploit camera tracking as an interface. In scenario one, animated monkeys representing Hampi’s identification as Kishkindha from the Ramayana virtually inhabit the various panoramas. The monkeys ‘sense’ (i.e., ‘look’ into) the real world using information from the camera tracking system, ‘seeing’ the participants inside AVIE and responding according to certain dynamics exhibited by the visiting crowd. For example, when people are close to the screen the monkeys may move to the other side. The monkeys approach motionless people by exhibiting curiosity. The framework advanced in the research addresses a need articulated by virtual heritage scholars to treat the heritage object as an evolving and symmetrical experience in which the story told is not pre-rehearsed but emerges as an interactive dialogue between participants and machine agents. Combined with symbolic logic and cognitive programming of computer-generated characters, Hampi-Live co-evolves the narrative engagement between the intangible heritage of ‘place’ and the participants.

In scenario two, a virtual tourist wandering through the virtual scenery with a video camera detects the real visitors to the AVIE space and ‘corrals’ them to a particular location, then takes her video camera out to record them and displays this video to the visitors as a real-time recording. The visitors are captured from infrared cameras mounted on the perimeter of the AVIE screen, and their respective locations are known to the system through the tracking system.

High-level cognitive programming and virtual models of the stereographic panoramas enabled the characters to wander the Hampi landscape while interpreting gestures from the real-world space within AVIE. Through the affordances of new technologies in AVIE, Hampi-Live is able to transcend common interpretive frameworks to become a site of mediation, an entanglement of people-things from the past and present occupation of the site. Hampi-Live is thus a principal demonstrator for the theoretical advances of co-evolutionary models of immersive interactivity. Virtual environment research indicates that interactivity driven by (or with) machine agents is the most theoretically appropriate basis for delivering narrative experience in virtual heritage. Co-evolutionary narrative extends interactivity to embrace the concept of the autonomy of both machine agents and humans by furnishing the agents with a (modest) system of awareness. This system is acted out within a framework of multiple scripted scenarios that test the capacity of machine agents to convert their perceptions into deliberative actions.

The project introduces co-evolutionary systems as the first comprehensive basis for the production of digital heritage and museum infotainment. Co-evolutionary systems provide the scale and complexity necessary for realising the aesthetic potential of immersive digital interaction. It provides museums and the wider industry with an interactive architecture in which the scholarly and aesthetic content of the narrative is integrated into the engine of a co-evolutionary system in immersive settings with ‘screen free’ interfaces. These systems replace the current ‘computer game’ industry standard, wherein interaction is enacted through console interfaces and predictable, linear, goal-based narratives. Through its innovation of a qualitatively different system, co-evolutionary narrative provides a powerful presentational tool and a significant extension of the content portfolios available to the heritage and museum sectors as well as the entertainment fields.