The Use of Bio-interfaces in Interactive Multimedia Works : Two Examples

This research is about the relationship between the human body and digital devices, particularly the interaction between real-time media (audio and visual) and the bio-data from performers or from an audience. This research belongs to the field of biofeedback, based on research pursued by the psychologist Neal E. Miller in the 60s. As a result of this research, the paper introduces two interactive performances of my authorship using bio-interfaces that have been produced in the last six years. The results herewith are used for artistic purposes and not to reduce stress, as it is the case in biofeedback methods such as Miller’s.

This method was developed during the 60s by the psychologist Neal E. Miller, which consists of the process of measuring physiological parameters from a subject (for example, the heartbeat, the brainwaves or the breathing), sending this data to a computer to be analysed and afterwards translating these parameters to sound and video, feeding them back to the subject.This treatment technique was originally created to improve health, helping tense and anxious people to learn how to alter these functions by relaxing.The two bio-interfaces introduced in this paper are the EMG (electromyogram) interface used for the Butoh Performance-Installation: Seed/Tree and the EEG (electroencephalogram) interface used for the performance INsideOUT.Both performances were programmed in real time media with the software MAX/MSP-Jitter.
This installation-performance consists on a forest environment created by some panels projected by haptic images from tree cortex and human skin.By haptic images I mean: HAPTIC from (the Greek word: HAPTΌS -tactile): the sensation "to feel": the feeling that one can 'touch' with the eyes.There are three Butoh dancers in the space performing the process of a seed growing to become a tree.Each performer moves in his/her own way representing the same subject a-synchronically.In the performance, the inner impulse of every dancer emerges on the stage; the dancers have each their own movements, which are the product of their own imagination.
The installation runs for three hours.The dancers develop the main subject in twenty minutes, then they lie for ten minutes on the floor and afterwards they repeat the process from the beginning.During this time, the performers have the necessary time to experience their own imaginary world combined with the outer space created by the sound and video projection.There is a continuous feedback between the dancers and the media; the translation of emotional physiological parameters to sound and video, however, gives feedback not only to the dancers but also to the spectators.Feelings, associations, mental images and spontaneous impulses are the starting point for the creation of stories and choreography.In Seed/Tree the dancers produce and transform the sound and the audience control the video projections.The results are instantaneous creations, expressions of the moment, with image, movement and music forming living signs in space.
There are two types of interactivity in this performance.The first one is the interaction between dance and sound: the performers have microphones and EMG electrodes attached to their bodies.The breathing and the heartbeat of two of the performers produce sounds that are continuously modified by the muscular tension of a third dancer.The second type of interactivity is that between the installation space and the visitors.During the performance, visitors can walk freely around the virtual forest.There is a video observing the installation space and human presence influences the video projections; these interactions create subtle differences of the video on the panels.The installation space itself becomes aware of the visitors and reacts according to their movements.In this way the visitor is invited to be part of the environment.A video excerpt of Seed/Tree can be downloaded at: http://icemserv.folkwang-hochschule.de/~robles/works_media/seed_tree/seed_tree.mov

INsideOUT (2009) Performance/Live Electronics
This project was created during an "artist in residence" program at the KHM (Academy of Media Arts in Cologne, Germany).It is about the materialization of the performer's thoughts and feelings on the stage.'The stage is a place for the appearance of the invisible.Yasu Ohashi says: the actors aim at our senses, our body and our unconscious and not at our intellect.Their gestures try to envision THE INVISIBLE WORLD'.(Haerdter and Kawai, 1998: 25) The performer interacts with the sound and images using an EEG (electroencephalogram) interface, which measures the performer's brain activity.Those sounds and images -some already stored in the computer and some produced live-are continuously modified by the values from two electrode combinations via MAX/MSP-Jitter.Hence, the performer determines how those combinations will be revealed to the audience.Images are projected to a screen and also onto the performer, while sounds are projected in surround.The German psychiatric doctor Hans Berger was the first to produce electroencephalograms of human subjects and also the first to observe the characteristic alpha-rhythm.The EEG project was meant at the beginning as the expression of the self, about turning the subject's imagination from the inside to the outside.
For this performance, I used an open source EEG interface from Olimex, which measures the brain activity and consists of two assembled boards: one analogue and the other digital.Each board has two electrode combinations (or two EEG channels).It is possible to connect up to three boards (i.e. six EEG channels).I use only two for this piece: the frontal and the occipital channels.The rubber cap and the contact electrodes of the interface are those typically used in medical applications.In Processing, a Fast Fourier Transform (FFT) is applied to that data and extracts the bins for the frequencies 0-50 Hz.From these, the median of the frequencies for the alpha channel (8-13 Hz) is extracted, smoothed using a low-pass filter and transmitted via OSC, which is received once again by the OSC-route object in the MAX program.This frequency band between 8-13 Hz correspond to the Alpha frequency range that is accentuated during relaxation.A small part of the Beta range is taken, which corresponds to an alert or an anxiety state.
For the performance of INsideOUT I have tried to train my brain in order to control the media combinations on the stage, putting in evidence different emotional and mental states.However, this conscious control is not completely attained due to the enormous and uncontrollable stream of feelings that generally appear surprisingly under such circumstances.My aim herewith is to be able to control and to be aware of the body and its functions and at the same time, to allow for an empty space that could be populated by the invisible or the imperceptible.This research and its artistic results aim to raise awareness of the human body and its functions (e.g.muscle tension, breathing, etc) as a means to manipulating the media by controlling those functions consciously on the stage.This shall create an environment that invites the visitor to perceive the body in a different way and to reflect on the relation human/machine.

Figure 2 .
Figure 2. Seed/Tree at the ZKM Centre Karlsruhe (Germany)©, 2004 For Seed/Tree I used a wireless EMG (electromyogram) interface developed by Frieder Weiss, as shown in Fig. 3.

Figure 5 .Figure 6 .
Figure 5.The Frieder Weiss program sending the values as a continuous OSC packet

Figure 8 .
Figure 8.The analogue and digital boards from Olimex.2