Part One: The Central Argument

 

Liquid Sensations has investigated the use of interactive video installations to evoke the corporeal sensations related to being submerged in the water. This has been a studio-based research investigation that has resulted in three video installations for viewing in an art gallery. The research contributes to the area of interactive video installation art as a medium to evoke sensory experiences.

 

The primary reason I work with technology is not an interest in the process or possibilities it offers, but rather a curiosity about the final results that arise from applying this technology. Media technologies can create multi-sensory, dynamic, immersive environments that are radically different to what can be produced with other mediums of art-making [1] . Erik Davis when writing about materials being manipulated when using media technologies states:

 

So as we survey the expanding and converging landscape of electronic, virtual and immersive production, we might ask ourselves: what material is being worked here? Is it simply new organisations of photons, sounds waves and haptic cues? Or does the “holistic” fusion of different media and the construction of more immersive technologies actually suggest another, perhaps more fundamental material? I’d wager that the new material is indeed rather fundamental: human experience itself. [2]

 

Davis goes on to say that the human experience is a ‘vague and loaded concept’ and he defines it ‘as the phenomenal unfolding of awareness in real-time’. Davis is not alone in highlighting this aspect of media technologies [3] .

 

There is a growing awareness of the experiential nature of media technologies. An artist working in this way does not manipulate experience, the experience exists in the mind of the viewer. An artist works with the elements the viewer sees, hears, feels, touches and manipulates. The sum of these elements evokes and suggests experiences for the viewer. The approach Liquid Sensations has taken is to work with elements of an environment, aiming to evoke felt sensory corporeal experiences for the viewer.

 

The chosen medium for this project is interactive video installed in the neutral space of an art gallery. Video is able to evoke the transitory, dynamic nature of sensory experiences. It has been installed in a gallery and used to surround the viewer visually and aurally. Interactivity is applied to increase the viewer’s engagement with the work.

 

Bill Viola when writing about the nature of the moving image in his article Video Black – The Morality of the Image says ‘the entire phenomenon began to resemble less the material objects depicted and more the process of the mind that was moving them.’ [4] For Viola the moving image becomes less about representation and instead is a medium that, through its temporal qualities, has a connection to human consciousness and perceptual experience. Sensory experiences are perceptual, subjective and of the mind; video is an effective medium for suggesting these experiences.

 

Video is not just a visual medium, it has both visual and aural elements. Sean Cubitt when discussing sound design for the moving image states ‘Sound is physical: it can only be heard. It occupies, and in occupying it creates spaces.’ [5] Later he goes on to say ‘skin produces and receives sound; it is the intimacy of body on body.’ [6] Hearing is a tactile sense and sounds are spatial. This means sound is a potent medium to evoke the experience of environments that have physical qualities.

 

Digital media technology means the process of producing and post-producing high quality video and sound is faster, simpler and cheaper. Time-based media become highly malleable, they can be layered, manipulated and highly controlled. They are not technically difficult to work with which means the focus can be concentrated on how they are being applied. I have used these possibilities in Liquid Sensations to be able to rapidly process and rework the recorded media so that they are manipulated to give an impression of the environment being evoked.

 

The viewers are not totally immersed in the media spaces created in Liquid Sensations, instead they are surrounded by the media. They are able to move around the spaces and look at the media from multiple viewpoints and they remain conscious of being in a gallery. Liquid Sensations does not attempt to totally transpose the perceptual experience of the viewer to another location in the way technologies such as virtual reality do. Mine is an approach that accepts that humans are spatial and physical beings. The technologies of multi-channel video projection and stereo sound are a viable means for artists to be able to surround the viewer.

 

Interactive video is not a new area: during the 1980s it was pioneered by artists using computer-controlled video laser discs. Newer digital video technology allows for more flexibility and a greater level of control over the video segments. Interactive video helps to break video out of being a linear narrative medium, into being a spatial and active medium. It allows the media to change and vary depending on the viewer’s responses and this creates an intimate level of engagement between the viewer and the works.

 

The sensations that have been chosen relate to being in and near the water because they can provide a corporeal experience that has a personal resonance for me and are difficult to evoke in a medium that does not involve multiple senses. Immersion in water is where I escape to find solace. This may be in, near or on the water; lap swimming or snorkelling, walking on the beach or sailing [7] . Roger Deakin writes of swimming:

 

You see and experience things when you’re swimming in a way that is completely different from any other. You are in nature, part and parcel of it, in a far more complete and intense way than on dry land and your sense of the present is overwhelming [8] .

 

When entering water the body seems to meld into the substance surrounding it. ‘When you swim, you feel your body for what it mostly is, water’ and your body ‘begins to move with the water around it’. Swimming is not an activity where the surroundings disappear, it is an activity where the environment is the focus, the corporeal sensations of it all-encompassing. It is also an environment in which swimmers are isolated and alone. They are unable to communicate normally and are separated from people outside the water and other swimmers in the water. They alone must act and move to survive. Charles Sprawson in his book Haunts of the Black Masseur: The Swimmer as Hero suggests of a swimmer that ‘so intense and concentrated are his conditions that he becomes prey to delusions and neuroses beyond the experiences of other athletes’ [9] .

 

Human bodies are mainly water, but are not ideally adapted to aquatic environments; there is a constant risk of drowning. We have an uncertain connection with submersion in water. Even for all my preoccupation with and pleasure derived from being in, near or on the water, it is an environment in which I am always conscious of the inherent danger. Submersion in water is a strong bodily sensation and man’s lack of adjustment to it can generate unusual states of mind [10] . These feelings are almost beyond language and exist in their own perceptual category. This odd unsettling experience is difficult to suggest with media that is not transitory, that does not surround and interact with the viewer.

 

Human bodily encounters with the aquatic world are broad-ranging and involve all five senses. Water has many associations and metaphorical meanings including its use as a medium of healing and our growing concern about humankind’s impact on the natural environment. It creates tactile feelings of cold or warmth on our skin. It has a density and weight and has its own fragrance. The movement of water generates sounds; when submerged in it our hearing is altered. Water can be encountered in many varied forms and places including showers, baths, swimming pools, the sea, rivers and lakes and each of these can induce different sensory responses.

 

Liquid Sensations specifically deals with the sensations that are related to entering the water from the sea shore, shallow breath diving and floating under the surface of the water. The location is the sea because it has its own distinct range of sounds and imagery and is relatively untouched by human influence compared to aquatic environments such as swimming pools. This project concentrates on these bodily responses as first person subjective experiences and draws on personal experiences of aquatic environments throughout my life.

 

Liquid Sensations has been a studio-based project where I have worked on all aspects of the project. This is a different method from a project-based production methodology in which an idea is outlined, planned, developed and then implemented by a team of experts working collaboratively [11] .

The studio-based methods used have included:

·       a personal journal to track progress and to reflect on the process

·       regular setting up and testing of the installations [12]

·       video documentation of the outcomes at each stage

 

A parallel to this ‘hands-on’ method has been a focus on the viewing and studying first hand of other artists’ works. Working with new media technologies is an interdisciplinary activity, which can make issues of context complex, because influences and areas of interest are diverse. The field for this research is contemporary arts. This project has not developed new technologies or explored the effects new technology has on society and has not proven nor shown theoretical and technological possibilities. The research has focused on the challenges that develop from applying interactive video installations to the creation of evocative experiences.

 

The context for the research has been other artists’ practices that evoke experiences related to submersion in water and some related artists who have used video installation. Within the wider field of contemporary arts, there are artists who attempt to suggest some of the unsettling corporeal experiences of aquatic environments. One approach is to depict people involved in the aquatic environment. Other projects have used the sensual properties as a starting point for the interface for virtual reality works and for hybrid media and built environments. Video technology has been applied by other artists as a means to surround the viewer and interactivity used to add new levels of engagement to the viewer’s experience of art. Progressively, technologies are being seen as a way to construct experiential art. These relevant art practices are discussed further in Part Three: Related Art Practices.

 

‘Technical risk’ is the term used to describe research fields which investigate novel applications of existing technologies. This project required alternative uses of much the technology involved. This aspect of the technical risk research is discussed in Part Four: How the Project was Pursued and in the details of the final solutions are provided in Appendix Two and Three.

 

During the evolution of this project the installations were constantly refined and reworked. My research evolved from seeing the potential for specific technologies, to an understanding of the types of experiences that the technologies help to produce. The case study focuses on the corporeal sensations that relate to the activity of being submerged in the water. The installations that have developed out of my research project are:

 

·       Wake, which evokes the sensations of water wrapping around the body that occur when entering the water from a beach and the relative calm once past the surf.

·       Dropping, which evokes the sensory experiences of shallow breath diving, which are to descend under the water, to experience a slowing down and then to surface again for air.

·       Under, which suggests the bodily feelings of breathing out while looking up to the surface from under the water.

 

These installations are described in detail in Part Two: A Description of the Works.

 

During the development of these installations a number of challenges needed to be resolved. I have divided these challenges into three interrelated research questions:

 

·       How can video installations be used to evoke sensory experience?

·       How can unobtrusive interactivity be used to enhance the viewer’s engagement with the works?

·       What are the methods for a solo artist to develop interactive video installations?

 

The contribution Liquid Sensations makes to the field of interactive video installation art, as a medium to evoke experience, encompasses both the documented methodologies and the outcomes.

The Research Questions

How can video installations be used to evoke sensory experience?

The moving image is linked to both cinema and narrative. Two components of the cinematic experience are the emotive and corporeal effects of imagery and sound. Many cinematic devices exist for evoking emotional and physical responses related to narrative, but this project does not utilise narrative; it is instead an exploration specifically of the sensory realm. This leads to the question of how a medium based on recording and that is most often used to create a narrative can be utilised to go beyond the representation of an environment and, instead, be applied to evoking the physical feelings of being in that environment.

 

This project achieves the following:

 

·       By using imagery and sound, it has suggested what the viewer might see and hear when immersed in the water. Subjective first person viewpoints of being hit by a wave, diving under the water and looking up at the surface of the water have been used.

·       By digitally manipulating the imagery and sounds recorded from in, near and above the water the works have not just represented the aquatic environment. The media have been manipulated through processes such as changes to the tonality and colours of the imagery, the pitch of the sounds and layering of these sounds and imagery. These manipulations mean that the recorded sounds and imagery suggest a bodily experience of the water.

·       By using video projection and how the images have been positioned in the gallery, the bodily nature of submersion in water has been heightened. This has been accomplished by the use of projections that are not related to a cinematic style of viewing an image. The imagery has been placed on the floor or ceiling or is hanging in mid-air.

How can unobtrusive interactivity be used to enhance the viewer’s engagement with the works?

In art the introduction of active interaction in the process of viewing artworks by utilising technology has been emerging for over thirty years. Interactivity can create another layer of dynamic engagement with artworks. The challenge with applying interactivity to create environments that aim to evoke sensual responses is the need for conscious interaction and any interface to be intuitive. Interfaces where the viewer uses gestures, touches or manipulates objects would reduce the seamless effect of installations in the context of a gallery. In order to interact with works that use this type of interface the viewer has to learn or modify their behaviour. What viewers commonly do in a gallery is walk around, look, pause and then move again. These assumptions are based on my personal experience with other interactive works and observations of other viewers interacting with works. When in the water our connection with it is seamless; the way we interact with water is an integral component of submersion. Actions have an effect on our sensory perception of the environment. When dealing with the sensory feelings of submersion in water, the focus needs to be on actions in the environment that have a meaningful relationship to the activities of pausing and moving.

 

The following solutions were developed:

 

·       The reactions of the installation are based on movement and stillness in the room, which is usually part of encountering an art work in a gallery. To be submerged in water is to move through the water. The activities of pausing in the surf, diving down under the water and breathing out are linked to the viewer’s actions of moving or being still. These simple intuitive interactions are part of an encounter with the installation and add to the immersive intent of these works.

·       A control system that takes into account how long a viewer has been moving or for how long they have been still was developed. The system then decides if a viewer’s actions are unusual based on probability data. This system develops a memory and understanding of past and present activity levels in the gallery. The media played reflect these levels of activity. The amount of movement in an aquatic environment affects the related experiences. In the case of Wake the waves that hit when the level of activity is less than normal are harder and when the level of activity is more than normal the images are softer. This reflects the experience where if swimmers stand still in the surf they are pushed over, but if they move and try to find a balance with the energy of the wave the experience of the environment is less turbulent. This technique of temporal interactivity increases the viewer’s level of engagement with the work through an unobtrusive method.

·       The project has resulted in three installations, each of which explores a different sensory aspect of submersion in water. The works have been orchestrated together using sound. The overlapping of the sound has been considered and the underwater sounds are controlled by a system that reacts to the current activity level in all three installations. This adds a layer of integration between all the works, creating an interrelated environment where the viewer has a subtle effect on all of the works and there exist multiple levels of connection between the works.

What are the methods for a solo artist to develop interactive video installations?

To construct these interactive video installation a number of different fields of specialised knowledge needed to be accessed. These include video and audio production; computer programming; electronics; architecture and audio visual installation. But within these areas there does not exist a methodology that could be applied successfully to the specific problems of how to develop interactive video installations. Out of this research project a production flow has been designed (see Appendix One for a detailed outline of this process). The final interrelated, holistic methodologies involve:

 

1. Planning

·       balancing of: technology available; expertise with that technology; and the exhibition space;

·       sensory experience to be evoked and the installation space; and actions that are involved and what interaction may evolve out of these actions.

2. Prototype development

·       recording and manipulation of media;

·       development of software;

·       3D models and sketches of the gallery.

3. Testing

·       set up in a room or gallery;

·       attaining viewer feedback.

4. Refinement

·       looking at ways of working on only one aspect of the work;

·       re-recording of media.

5. Completion

·       The intended sensory experience may have been evoked or another experience may have arisen: if the latter is the case, Stage 1 needs to be revised.

 

The technologies used in the installations are relatively simple. The works have been developed using commonly available multimedia software (see Appendix Two for an outline of the final technical methodologies). Recorded video and sound have been manipulated and processed digitally as this allows detailed attention to be applied to the imagery and sounds and for them to be manipulated with a high degree of accuracy and control. The hardware, display technology and sound playback systems that have been used are becoming more prevalent and are relatively accessible compared to developing custom hardware. The cost of the technology employed is relatively low compared to many of the other solutions used to develop immersive environments.

The significance of the outcomes

Liquid Sensations is an exhibition of works that contributes to the area of interactive video installation art as a medium to evoke sensory experiences. The use of technology is a developing area and one way in which this medium is being employed by artists and designers is to suggest experiences. Liquid Sensations has focused on suggesting elements of an environment. A series of experiential artworks that evoke sensory experiences related to being in aquatic environments has been developed. A number of the techniques and processes that have been used and developed in these artworks could be applied to evoke sensory experiences connected with other environments.

 

These techniques and processes include:

·       The digital manipulation of the imagery and sounds so that they become evocative and suggestive;

·       The use of imagery recorded from a first person viewpoint, rather than imagery of people experiencing an environment;

·       How imagery and sounds have been positioned in a gallery to increase the sensory effect for the viewer;

·       The use of seamless, indirect, physical interactions where moving and being still are part of the interaction;

·       The integration of the viewer’s engagement based on a system using temporal sampling techniques where the temporal aspects are an important facet of the interaction with it;

·       The co-ordination of multiple works by sound and a network-based system for message passing where multiple installations are placed in a gallery;

·       Using the process outlined in Appendix One: The methodologies of interactive video installation to develop interactive installations; and

·       The use of commonly accessible technologies to develop multi-sensory environments.

 

If technologies are being utilised to create evocative experiences then ways to suggest, construct and apply these technologies need to be developed. Liquid Sensations has made a contribution to this understanding of how to evoke sensory experiences by developing new methods to suggest elements of an environment.

next

 

 

 

---

[1] Before starting this research my recent commissions had included community arts and multimedia work as part of a collaborative theatre production. During the previous five years, my practice had evolved around the possibilities of new media and the effects that these have on our understanding of the world. During the mid 1990s I had started to work with low end virtual reality systems. The last work in this series was Clouds Of (http://www.otheredge.com.au/prj/cloudsof). As a reaction to the degree of technical complexity involved in this last virtual reality work, I developed the simpler web-based project Archiving Imagination. Archiving Imagination is an array of online projects exploring the process of collaboration between myself and writer/web-author Diane Caney. It is an assemblage of video segments, sound, images and words, all of which are traces of the boundaries operating across word-image interactions (http://www.archiving.com.au/). The intent in my past works has always been to create immersive sensory experiences, by subtle means. In the virtual reality work this was literal, in the web-based work it was attempted by constructing poetic works where the whole was greater than the sum of the parts.

[2] Davis, Erik, ‘Experience Design.’ Archadia: Writing on Theology and Technology, Samara Mitchell, ed., Australian Network for Art and Technology, Adelaide, 2001, p 27.

[3] Vibeke Sorenson and Mark Beam call it sensual media in the abstract of From Rich Media to the Sensorium: How to Understand Pervasive Computing. They see sensual media partly as a development out ‘of the dream of technologists and science fiction writers’ for environments ‘that closely resemble physical experiences’. They define sensual media as

 

a means of interacting with information through the use of and interconnection between many senses, the extension of the human body across time and space, and the new communication structures and processes that arise from the use of digital technology (available online at : http://www.store.yahoo.com/creativedisturbance/fromricmedto.html).

 

Joseph Pine and James Gilmore in The Experience Economy call experience the ‘fourth economic offering’. They state ‘offering of experiences occurs whenever a company intentionally uses services as the stage and goods as props to engage an individual’. Nathan Shedroff in Information Interaction Design: A Unified Field of Design and later in his book Experience Design uses the term ‘experience design’ in terms of multimedia design.

[4] Viola, Bill, Reasons for Knocking at an Empty House: Writings 1973-1994, MIT Press, Cambridge, 1995, p 204.

[5] Cubitt, Sean, ‘Footprints in the Air: Mechanical Perception, the Media Arts, Diaspora and Sound’, Art & Design, 11.7–8 (1996), 72–9, p 74.

[6] Ibid., p 78.

[7] One of my first video works as an undergraduate student included images of water. These images and sounds have been an important element of my art practice since that time.

[8]  Deakin, Roger, Waterlog: a Swimmer’s Journey through Britain, Vintage, London, 2000, p 4.

[9]  Sprawson, Charles, Haunts of the Black Masseur: The Swimmer as Hero, U of Minnesota Press, Minneapolis, 1992, p 7.

[10] Is this uncertainty a manifestation of the sublime? Jonathan Raban in The Oxford Book of the Sea quotes Edmund Burke’s definition of the sublime from On the Sublime and Beautiful

 

Whatever is fitted in any sort to excite the ideas of pain and danger, this it to say, whatever is in any sort of terrible, or is conversant about terrible objects, or operates in a manner analogous to terror, is a source of the sublime;that is, it is productive of the strongest emotion which the mind is capable of feeling. I say the strongest emotion, because I am satisfied the ideas of pain are much more powerful that those which enter on the part of pleasure.(p 8)

 

Raban states this definition evokes the ‘tactility involved in the power of the ocean’. The sublime is an extreme state and this idea has influenced many artists and writers over time. Particularly in Burke’s writing the term sublime has been applied to any endeavour attempting to find an extreme state. There are many experiences of the ocean and water in which these types of experiences might be found. As we start to understand the natural world by scientific and technical means the natural environment may no longer hold this same fear and danger. What this project seeks to evoke is an unusual state of physical experience related to being in the water. These are not the extreme mental states of fear and danger that Burke defines as the sublime.

[11]

The interdisciplinary nature of art-making involving technology often demands a type of project-based collaborative work group.

[12] The media and scripts were developed in a computer laboratory situation and then refined while the works were in situ. The viewers that tested installations were largely other research students and academic staff. Where possible an ‘episodic first person narrative’ was used where viewers or myself would talk through their experience of the piece.

next

 

 

This page is part of the web version of 'Liquid Sensations: Evoking sensory experiences with interactive video installation art' written by Robin Petterd as part of a PhD by studio practice at the Tasmanian School of Art, The University of Tasmania.

robin@otherdge.com.au