Introduction

2016 saw the first release of virtual reality (VR) aimed at widespread commercial use (O’Brien, 2016). These products are beginning to gain large-scale traction in what was until recently a niche industry, in part because mainstream computer processing power has reached a point where it can achieve ‘presence’. This term is given to a combination of low latency, visual fidelity and frame rates vital for a convincing VR experience (Abrash, 2014), where the mind can be tricked into believing it is in a space as opposed to seeing a 3D environment on a screen.

Most initial development has focused around games (Robertson, 2016), with notable experimentation into 3D art and design – such as Google’s Tilt Brush Artist in Residence programme (Tilt Brush Artist in Residence, 2017). Our research aimed to take these tools and to investigate what opportunities these could offer for those interested in the fields of dance and movement research. Led by experimentation, focus was first placed on the nature of the experience itself, especially in relation to our sensory understanding of space while within VR (Figure 2). This led to experimentation in creating interactive and sculptural spaces that invited the audience/performer to interact with this simulated reality (Figure 1). The final area of exploration involved using physical objects within the virtual space to explore our aesthetic, kinaesthetic and perceptual understanding of this technology. This work will place these explorations within a range of existing research to probe the complexity of the relationship between the moving body and virtual space.

Figure 1

Ben Skinner creating an interactive space in VR.

Figure 2

Jennifer-Lynn Crawford exploring virtual space early in the research process.

The practical investigation was undertaken by James Else (Head of Creative and Contextual Studies at the Northern School of Contemporary Dance), Jennifer-Lynn Crawford (independent dance artist and body work practitioner) and Ben Skinner (independent dance and visual artist).

Initial Experiences

This project chose to use the HTC Vive, as this offered the only consumer priced room-scale experience available at the time of the research. Not only did room-scale seem crucial to exploring movement, but as Paul James (2016) states:

There’s something intangibly special about moving around VR with your whole body in tow. Engaging your physical self in virtual worlds seems to unlock new opportunities for achieving presence that felt out of reach with seated experiences.

Our initial experiences were more pronounced due to the lack of widespread exposure to virtual reality technology either through an iterative development process (such as smart phone development) or through acclimatisation while growing up (like television). Everyone involved in this project experienced similar feelings of wonder with their first exposure. It is a non-translatable experience that divorces a computerised environment from its usual physical constraints to a setting where you ‘take your whole body in with you’ to engage in realistic and functional movement within an unreal space.

VR, in its current form, virtually reproduces some objects from the real world, such as motion controllers, but omits almost all others, including the body of the person in VR. The disjunction of the senses caused by this produced very different initial experiences. Jennifer-Lynn immediately felt profoundly the lack of her body – of being, but also not being, within this virtual space:

Nothing was familiar and there were literally no objects to handle in any familiar way, including my closest horizon of almost-object, my body … Projecting myself in to the virtual space felt impossible – there was no space for ‘me’ there, as I feel myself to be on a daily level (Crawford, personal correspondence, 16^th Jan, 2017).

Ben experienced a similar disconnection from his body. However, in his case, this enabled him to get lost in this virtual world, encouraging him to act without higher-level thinking:

I was unable to see my hands, feet and therefore became immersed in a new world. I was hesitant to step into the image but my hands drew me in. Nothing was fixed and the fluidity was disconcerting, the stability of seeing something permanent on a piece of paper was no longer relevant as I became the paper but also the looker/analyser (Skinner, personal correspondence, 30^th Jan, 2017).

Both of these experiences suggest that first experiences within VR have such a strong impact because the relationships between sight, touch and self-awareness are disrupted. These thoughts mirror those of Susan Kozel’s during her time in Paul Sermon’s 1992 installation Telematic Dreaming. In this installation performer and audience occupied different physical spaces, but had their actions mirrored through filming and projection rather than through VR technology (Kozel, no date). Kozel found the more she:

Ventured into the visual, virtual world the more my non-virtual body called attention to itself like an anchor, like ballast. I seemed to be pulled between the two extremes of an imaginary spectrum: the abjection of flesh and the sanitization of technology (ibid).

As with Ben, Kozel found this also opened up new possibilities, ‘the experience was one of extending my body, not losing or substituting it’ (ibid).

For myself my initial experiences were of total immersion on the border of sensory overload when immersed in VR. It was also an experience that drew a sharp contrast between the ‘real’ and ‘virtual’ worlds:

What I was most unprepared for was putting the headset down, and returning from the stark modernist mega-warehouse [of the tutorial application] to the subtle dark post-Autumn colours of reality, and having a split-second moment of genuinely wondering which is reality (Else, research notes, 6^th Jan 2017).

During the course of the research this highlighted how frequently we accept disconnection from our senses. In a similar way humans commonly present cognitive dissonance by seeing ourselves as both part of nature while nature also excludes human interference (Vining, 2008, p. 10), VR creates dissonance in our understanding of the space we inhabit:

The body becomes an inter-media surface, the field for a dual experience between real space and virtual space which thereby acquires a new single dimension. And this dislocation of the corporeal experience can open the way to a new interrogation of the world and ourselves and, consequently, the possibility of imagining other possible kinds of space, other possible ways of being a body-that-becomes-space (Palumbo in Morie, 2008).

In this Palumbo is questioning the binary distinction between body and world. Even without technology, our body cannot occupy a space without also affecting that space, while equally we cannot inhabit a space without it having an effect on ourselves. The body has become part of the space. VR draws our attention to this, and blurs these lines further, questioning the physical body, your experience of your body, and how you perceive your body in space. In doing so VR invites us to explore the relation of body and space in new ways.

A state of wonder

In the same way that much of contemporary art of the last century has drawn attention to the process of its creation as an intrinsic part of the artwork, this software invites us to reconsider where the art resides in this medium. A lot of our investigations focused around using Google Tilt Brush (an application in which the user ‘paints’ with a motion controller to leave their paint strokes suspended in the 3D world). When creating work, even without skill or experience, the newness and limitations of this experience enhance the feeling of ultimate auteur and consumer simultaneously. You experience both at once, and you are the only person who can.

Parallels can be drawn to the ideas of Barbara Bolt where she considers a visual art work (such as a painting) to no longer be a passive object, but something that transcends the notion of being a thing – ‘painting no longer merely represents or illustrates reading. Instead, it performs’ (Bolt, p. 1). This perhaps suggests that VR is an ‘actual’ virtual container for a ‘perceptual’ virtual experience. Just as Bolt sees painting as a multi-layered activity where painter and painting cannot easily be separated, the medium of VR is not so much defined by the equipment, but by the activity that takes place between the user and the spaces created.

As with our discussion of the impact of VR on the senses, this highlights an existing misunderstanding that we are normally happy to accept. Watching a dancer in the studio, or a pianist on the stage, we equate our sensory knowledge to an understanding of their experience, even though such an understanding is only approximate conjecture. The physical barrier of the headset forces us to reconsider the performer-audience relationship in all but the most passive of experiences. In the case of Tilt Brush, by inviting us to consider ‘How do you draw space?’ (Skinner, personal correspondence, 30th Jan, 2017), it makes it not only a program in which you can paint, but also a program about the act of painting.

Instruction-based work

In an attempt to explore the act of painting as art we experimented with instruction-based works. In these one person would devise a set of instruction for another to follow. My initial experiments tended towards linear works to be experienced sequentially along a predetermined path (Figure 3). These pathway pieces owe much to similar artworks that put the emphasis on guiding the audience through a narrative experienced via a physical and interactive journey.

Figure 3

An instruction based experiment, which involved a sequence of interactive experiences.

In contrast Ben’s experiments took a more installation-based approach often characterised by an unravelling of the work as you decipher the creator’s intent. In these you would enter the virtual space (such as Figure 4) without a clear understanding of what you needed to do, and through interaction with the piece gradually reveal the work through your understanding of it. One of the most satisfying of these was using instructions written just above ground level which required the audience/performer to correctly deduce the right angle to read the instructions from and then manoeuvre their body into this position.

Figure 4

A sculptural interactive experience.

As is common with instruction-based work the most engaging instructions proved to be the ones that gave the right amount of freedom of expression for the person enacting them, whilst being sympathetic to the nature of the medium they inhabit.

The VR medium encourages the participant to move around, in, and through (Figure 5), in contrast to even the most active of interactive performances where the audience, through social convention and herding instincts, gravitate towards a more familiar static spectator role whenever they can. One opportunity therefore in VR is a more natural inclination to physical movement, perhaps reminiscent of Robert Morris’s bodyspacemotionthings. In this work, Morris’s subverts the traditional nature of the gallery space. Instead of viewing sculptures as simply something to be looked act (including often ignoring an instinctual desire to touch), his choice and position of 3D objects led the audience to have a spontaneous desire to interact with them through a range of different movements. This mirrors the way that participants in this project felt invited to naturally and unselfconsciously reorientate themselves in the space in a variety of ways (Figure 6).

Figure 5

Part of a work devised to invite different movement interactions in the virtual world.

Figure 6

Jennifer-Lynn Crawford absorbed in VR.

Exploring the boundaries between virtual and real worlds

The final area of investigation we undertook was looking at what happens when we blur the boundary between the real and virtual worlds to create a composite reality. Already there have been some high-profile experiments with the blurring of realities, for example rollercoasters on which you wear a VR headset which transposes the movement of the ride to an imagined world. Our investigations focused on exploring the implications and interactions when playing with dissonance between the virtual and real world, leading to our senses contradicting or overlapping. Unlike augmented reality experiences, which most frequently see extra visual information mapped onto the real world, we were particularly interested in creating situations where aspects of the real world impact on the virtual world and vice versa.

Our first idea involved placing a chair within the space, and then crudely drawing around it in virtual reality. This created the paradox that you were looking at something that appears to not be real, but can in fact be sat on and interacted with.

As we played around with a number of objects – a stepladder, a trolley, a bin – the interaction and range of possibilities they offered were very much dictated by the shape, function and social context of the object.

Misrepresenting objects created compelling tension between how they were seen in VR and the physical object. This ranged from false/exaggerated layers of texture (such as a trolley – Figure 7) through to recontextualising and subverting social understanding (like adding baroque detail to a utilitarian table, or playing an outline sketch of a bin as a drum). The facsimiles of these objects were real enough to be recognisable, but unlike the image of the VR controllers, whose likeness you accept on a functional level, the distortion of these objects was sufficient that the viewer is never able to resolve their ambiguity, leading to an unresolved state of tension:

The everyday objects, when they were inserted inside the virtual environment, were alien – they didn’t match their virtual signposts in a way that touch and movement would normally reveal them to me if I just had my eyes shut or was watching a film at cinema … (Crawford, personal correspondence, 16^th Jan, 2017).

Figure 7

A visual representation of a trolley created in VR.

Physical objects could also be used to alter the virtual space. For example, a bench was used to create an elevated tunnel (Figure 8). Within VR, where the content creator is able to control everything we see, one of the most important certainties we have is the ground we are standing on. We can physically feel it, and almost all VR applications replicate it as visual information. By placing such a large physical object in the space and having this represented virtually subverts an emerging convention of VR.

Figure 8

A doorway drawn in VR leading to an elevated tunnel around a physical bench.

Given the nature of the medium, it is hardly surprising that these VR experiments easily became explorations of perception and reality (Figure 9). Consider the chair investigation. When we first see an object, we perceive its shape, form and texture before we identify it as a specific object (Stasko, 2016). Therefore, if we see a chair in real life, we first see the shape and form of a chair, before then identifying it as a chair. This normally corresponds with an awareness of social function and expectation of environmental context.

Figure 9

James Else exploring 3 objects recreated in the virtual environment.

In comparison, with our VR chair, we first see the shape and form of the outline of a chair, we identify it as the outline of a chair, but without the density, structure or supporting capability of a chair. This contradicts our contextual (and potentially kinesthetic) knowledge that a physical chair is actually there, leaving us very little understanding of function or environmental context.

Further details in the environment serve to cloud our understanding further. For example, the objects within virtual reality have highly realistic shadows in sharp contrast to the non-existent shadow of our absent body. Details such as these ensure that, when challenged, our understanding of ourselves is as contradictory as our understanding of the chair, and potentially even more complex and disconcerting. As Noë states, ‘Perceptual experience is transparent: To reflect on experience is, of necessity, to reflect on the world around us that we perceive’ (2004: 179).

It is possible to draw an abstract comparison here to the Pelli-Robson Contrast Sensitivity Chart designed to investigate elongated processing time in information at a visual threshold, to which Pelli has drawn links to understanding aesthetic preference (Pelli, 1997). In his experiments he describes a sense of limbo that exists in the 30-second or longer gap between a participant being able to see a shape and identify it as a letter. Our VR research often generated a similar ‘intense experience, tantalizing or frustrating’ (Pelli, 1997). However, unlike Pelli’s experiments which became uninteresting as ‘the categorization erases … the ineffable pre-categorical experience of the stimulus’ (Schmidt in Pelli, 1997), in our VR experiences the cognitive paradox was often impossible to resolve within our more limited contextual understanding of this medium, leading to an unresolved state of limbo.

Limitations of current VR

It was in exploring placing physical objects within the space that we found ourselves pushing up against the technical limitations of the technology. Placing multiple objects in the 3D space did cause some problems with tracking of the VR sensors. We also found our experiments were governed by the nature of the equipment. The weight and cabling, combined with the unfamiliarity of the environment discouraged complete freedom of movement, with the controllers corralling the use of the hands into functional tool-based movement vocabulary. Ironically this meant that these controllers, which were our primary means of interacting with the space, themselves became barriers to experiencing it fully, with the controllers often being placed down during explorations. For these reasons, our experiments into tracing patterns in 3D space proved less immediately successful then those experiments described above (Figure 10).

Figure 10

The outcome of an experiment tracing movement in VR.

Closing Thoughts

In analysing VR for the movement-based artist, it is impossible to avoid being drawn into considering the relationship between those involved in generating and those experiencing these new artworks. Since only facsimiles of physical bodies can occupy the virtual space, a completely connected experience becomes impossible. Consequently, not only can the performer and audience member no longer occupy the same physical space, no one involved in the work can even have a cohesive sensory experience with themselves.

Unlike the majority of VR game experiences, where there is a pre-existing acceptance of disconnection from human movement (for example, press ‘space’ to jump), for those seeking to exploit the physical possibilities of VR such a vocabulary does not exist. Therefore, just as early TV producers had to define the syntaxes and structures of television, the early artists working in VR will have to resolve the meaning behind the wide range of possible relationships and interactions between creator and participant. While elements of site-specific and interactive art theory can be applied, it means that VR is about an audience experience which is removed from any pre-existing models and understanding.

Our experiments into interactive spaces and composite realities demonstrated the emphasis placed primarily on the visual (and to a lesser extent auditory) senses by VR. Our somatic understanding of the space was far less coherent, yet just as important a part of the experience. Ultimately this means that when you are exploring movement within VR you are exploring the body interacting with a type of space that we simply have not encountered before.

By subverting physical and virtual objects in VR, this research has highlighted how easily our perception and understanding is challenged in this medium. This is particularly relevant to an understanding of our body within this environment, and how VR destabilises an understanding of our ‘self’ in space. Equally, it has shown that a successful understanding of VR must acknowledge that the activity of person interacting with the virtual space cannot be considered separately to an understanding of the space itself.

Funding Information

This project was jointly funded by research money from the Conservatoires of Dance and Drama and the Northern School of Contemporary Dance.

Competing Interests

The author has no competing interests to declare.

Author Information

James is a York based lecturer, composer and filmaker. He currently works as Head of Creative and Contextual Studies at the Northern School of Contemporary Dance, as well as being a freelance TV producer creating programmes for the BBC. Prior to this he completed a PhD in composition at the University of York with Nicola LeFanu.