- Few of us have actually donned an HMD (head-mounted
display) and DGs (data-gloves), and entered a computer-generated,
three-dimensional landscape in which all of our wishes can be
fulfilled: wishes such as experiencing an expansion of our physical
and sensory powers; getting out of the body and seeing ourselves
from the outside; adopting a new identity; apprehending immaterial
objects with most of our senses, including touch; being able to
modify the environment through either verbal commands or physical
gestures; seeing creative thoughts instantly realized without going
through the process of having them physically materialized. Yet
despite the fact that virtual reality as described above is still
largely science-fiction, still largely what it is called --a virtual
reality--there is hardly anybody who does not have a passionate
opinion about the technology: some day VR will replace reality; VR
will never replace reality; VR challenges the concept of reality; VR
will enable us to rediscover and explore reality; VR is a safe
substitute to drugs and sex; VR is pleasure without risk and
therefore immoral; VR will enhance the mind, leading mankind to new
powers; VR is addictive and will enslave us; VR is a radically new
experience; VR is as old as Paleolithic art.
- This flowering of opinions is fanned by
the rhetoric of the gurus of the technology:
Worldwide, VR is happening in protected
pockets of technology; inside giants
corporations, universities, and small
entrepreneurial start-ups; in Berlin and North
Carolina; covering Japan and especially in the
San Francisco Bay Area. . . . A rare excitement is
in the air, an excitement that comes from
breaking through to something new. Computers
are about to take the next big step--out of
the lab and into the street--and the street
can't wait. (Pimentel and Texeira, 7)
This sense of anticipation permeates all
books about virtual reality. They are less concerned with what has
been achieved so far than with what will be available in the (we
hope or fear) very near future. We may have to wait until the year
2000 to see VR become an important part of our lives, but since it
is depicted so realistically by its prophets, and since it exists
very much in the popular imagination, we don't have to wait that
long to submit the claims of its developers to a critical
investigation. In this paper I propose to analyze VR as a semiotic
phenomenon, to place it within the context of contemporary culture
and to explore its theoretical implications.
- My point of departure is this definition
by Pimentel and Texeira:
In general, the term virtual reality refers to
an immersive, interactive experience generated
by a computer. (11)
While "computer generated"
accounts for the virtual character of the data, "immersive"
and "interactive" explain what makes the computer-assisted
experience an experience of reality. To apprehend a world as real is
to feel surrounded by it, to be able to interact physically with it,
and to have the power to modify this environment. The conjunction of
immersion and interactivity leads to an effect known as telepresence:
Telepresence is the extent to which one feels
present in the mediated environment, rather
than in the immediate physical environment. . . .
This [mediated environment] can be either a
temporally or spatially distant real
environment . . . or an animated but nonexistent
virtual world synthesized by a computer.
(Steuer 76)
- Far from being restricted to VR, the
features of immersion and interactivity can be regarded as the
cornerstones of a general theory of representation and communication.
The purpose of this paper is to explore the problematics of their
textual implementation and to assess their significance for
contemporary literary theory.
Immersion
- Since immersion depends on the vividness
of the display, its factors are closely related to the devices that
lead to realism in representation. A factor that comes immediately
to mind is the projection of a three-dimensional picture. The
introduction of perspective in painting took a first step toward
immersion by creating a sense of depth that integrated the spectator
into the pictorial space. But because the medium of painting
simulates depth on a flat surface the spectator cannot break through
the can vas and walk into the pictorial space. In the visual
displays of VR the barrier disappears--there is no material plane of
projection--and the user feels surrounded by a virtual world which
can be freely "navigated" (as a standard metaphor of
networking describes movement in cyberspace).
- The creation of a 3D effect falls under
a more general category that Steuer (81) calls "depth of
information." This depth is a function of the resolution of the
display, i.e. of the amount of data encoded in the transmission
channel. As the other main source of immersion Steuer mentions the
"breadth of information," a category defined as "the
number of sensory dimensions simultaneously presented." Breadth
of information is achieved through the collaboration of multiple
media: image, sound, olfactory sign als, as well as though the use
of technical devices allowing tactile sensations. VR is not so much
a medium in itself, as a technology for the synthesis of all media.
- Sheridan (58) proposes another factor of
telepresence which stands halfway between immersion and
interactivity: control of the relation of sensors to the environment.
In order to feel immersed the user must be able to move around the
virtual space and to apprehend it under various points of view. The
computer tracks his movements and generates the sensory data
corresponding to his position in a continuously shifting display.
The control of sensors can go as far as a leaving the body,
relocating the center of consciousness into foreign objects and
exploring in this way places and objects normally inaccessible to
humans, such as the inside of a molecule, or the geography of a
distant planet.
- Insofar as immersion is "the
blocking out of the physical world" (Biocca 25), it cannot be
experienced if the user remains aware of the physical generator of
the data, namely the computer. The "virtual reality effect"
is the denial of the role of signs (bits, pixels, and binary codes)
in the production of what the user experiences as unmediated
presence. It is significant that Pimentel and Texeira title their
first chapter "the disappearing computer": as in the trompe-l'oeil
of illusionist art, the medium must become transparent for the
represented world to emerge as real. VR represents in this respect
the refutation of a popular myth: the personification of the
computer as an autonomous mind (a myth fostered by artificial
intelligence and its attempt to endow machines with creative
thinking). As Brenda Laurel declares in a book stressing the need
for aesthetic concerns in the design of software: "Throughout
this book I have not argued for the personification of the computer
but for its invisibility" (143). Jaron Lanier, a leading
developer of VR systems, echoes: "With a VR system you don't
see the computer anymore--it's gone. All that's there is you" (Lanier
and Biocca 166). The disappearance of the computer--which
constitutes the culmination of the trend toward increasing user-friendliness
in computer design--requires the replacement of arbitrary codes with
natural modes of communication. Binary coded machine instruction
once gave way to the mnemonic letter-codes of assembly languages;
assembly languages were in turn translated into high-level languages
with a syntax resembling that of natural languages. Then arbitrary
words were supplanted by the motivated signs of icons on the screen.
When machines are enabled to respond to spoken commands, the
keyboard will become superfluous. Next to go will be the screen and
the sight of the machine: visual displays should occupy the entire
field of the user's vision, rather than forming a world-within-the
world, separated from reality by the frame of the monitor. Last but
not least, language itself must disappear, at least in those areas
where it can be more efficiently replaced by physical actions. In
the ideal VR system the user will be able to grab and move objects,
to mold them through the touch of the hand, or to change their
colors with the stroke of a virtual paintbrush. In this mode of
communication there will be no need for the user to translate her
vision into sets of precise instructions. Purely visual thinking
will be implemented by means of practical, non-symbolic gestures. As
Pimentel and Texeira put it:
Simply, virtual reality, like writing and
mathematics, is a way to represent and
communicate what you can imagine with your
mind. But it can be more powerful because it
doesn't require you to convert your ideas into
abstract symbols with restrictive semantic and
syntactic rules, and it can be shared by other
people. (17)
The mystics of ages past (such as
Swedenborg, an esoteric philosopher of the XVIIIth century) had a
term for this radically anti-semiotic mode of communication. They
called it "the language of the angels."
Immersion and Literary Theory
- Through its immersive dimension, VR
inaugurates a new relation between computers and art. Computers have
always been interactive; but until now the power to create a sense
of immersion was a prerogative of art. It is significant that when
attempting to describe the immersive quality of the VR experience,
the advocates of the technology repeatedly turn toward a metaphor
borrowed from the literary domain:
For centuries, books have been the cutting
edge of artificial reality. Think about it:
you read words on a page, and your mind fills
in the pictures and emotions--even physical
reactions can result. (Wodaski 79)
The question isn't whether the created world
is as real as the physical world, but whether
the created world is real enough for you to
suspend your disbelief for a period of time.
This is the same mental shift that happens
when you get wrapped up in a good novel or
become absorbed in playing a computer game.
(Pimentel and Texeira, 15)
- The concept of immersion promoted by
virtual reality bears thought-provoking affinities to recent
theories of fiction based on the notions of possible worlds and of
game make-believe. The possible-world theories of fiction come in
many varieties (i.e. David Lewis, Umberto Eco, Lubomir Dolezel,
Thomas Pavel) and I cannot account for all of them; the following
discussion is mainly a synopsis of my own approach. Common to all
theories, however, is a reliance on the semantic model of a set of
possible worlds in which a privileged member is opposed to all
others as the one and only actual world. The distinction
actual/non-actual can be characterized absolutely, in terms of
origin, or relatively, in terms of point of view. In the absolute
characterization, the actual world is the only one that exists
independently of the human mind; merely possible worlds are products
of mental activities such as dreaming, wishing, forming hypotheses,
imagining, and writing down the products of the imagination in the
form of fictions. VR adds to this catalog of "accessibility
relations" a mode of apprehension that involves not only the
mind, but also the body. For the first time in history, the possible
worlds created by the mind become palpable entities, despite their
lack of materiality. The relative characterization of the concept of
actuality--advocated by David Lewis--regards "actual" as
an indexical predicate: the actual world is the world from which I
speak and in which I am immersed, while the non-actual possible
world s are those that I look at from the outside. These worlds are
actual from the point of view of their inhabitants. Among the modes
of apprehension that enable us to contemplate non-actual possible
worlds, some function as space-travel vehicles while others function
as telescopes. In the telescope mode--represented by expressing
wishes or forming conjectures about what might have been--consciousness
remains anchored in its native reality, and possible worlds are
contemplated from the outside. In the space-travel mode, represented
by fiction and now by virtual reality technology, consciousness
relocates itself to another world, and recenters the universe around
this virtual reality. This gesture of recentering involves no
illusion, no forgetting of what constitutes the reader's native
reality. Non-actual possible worlds can only be regarded as actual
through Coleridge's much quoted "willing suspension of
disbelief." The reader of a fiction knows that the world
displayed by the text is virtual, a product of the author's
imagination, but he pretends that there is an independently existing
reality serving as referent to the narrator's declarations.
- The notion of pretense and the related
concept of game of make-believe forms the core of Kendall Walton's
theory of fiction. According to Walton, a fictional text--as well as
any type of visual representation--is a "prop in a game of make-believe"
(11). The game consists of selecting an object and of regarding it
as something else, usually in agreement with other players (author/reader,
in the case of fiction.) Just as a stump may stand for a bear in a
children's game of make-believe, the picture of a ship is taken for
a ship, and the text of a novel is taken for an account of real
facts (an account which may or may not be regarded as accurate, as
the case of unreliable narration demonstrates). Players project
themselves as members of the world in which the prop is a bear, a
ship or a text of nonfiction, and they play the game by "generating
fictional truths." This activity consists of imagining the
fictional world according to the directives encoded in the prop.
Some of the fictional truths concern the players themselves, or
rather their fictional alter ego. The reader of a fiction does not
simply generate truths of the type "p is fictional" but
also "it is fictional that I believe p." And if p relates
the pitiful fate of a character, it will be fiction al that the
reader's alter ego pities the character. The emotions experienced in
make-believe in the fictional world may carry over to the real world,
causing physical reactions such as crying for the heroine. The
affinity of Walton's theory of fiction with virtual reality and its
concept of immersion thus resides in his insistence on the
participation of the appreciator in the fictional world. It is truly
a theory of "being caught up in a story."
- Like computer-generated VR, possible-world
and make-believe theories of fiction presupposes a relative
transparency of the medium. The reader or spectator looks through
the work toward the reference world. If the picture of a ship is
experienced as the presence of a ship located in the same space as
the viewer, it is not apprehended as "the sign of a ship."
If readers are caught up in a story, they turn the pages without
paying too much attention to the letter of the text: what they want
is to find out what happened next in the fictional world. This
reading for the plot focuses on the least language-dependent
dimension of narrative communication. And if readers experience
genuine emotions for the characters, they do not relate to these
characters as literary creations nor as "semiotic constructs,"
but as human beings.
- The literary devices which create a
sense of participation in fictional worlds present many parallelisms
with the factors leading to telepresence. One of the factors
mentioned above was the projection of a three-dimensional
environment. The literary equivalent of three-dimensionality is a
narrative universe possessing some hidden depth, and populated by
characters perceived as round rather than flat. By hidden depth I
mean that the sum of fictional truths largely exceeds the sum of the
propositions directly stated in the text. In a virtual world
experienced as three-dimensional, the user knows that reality is not
limited to what what can be seen from a given position: the outside
conceals the inside, the front conceals the back, and small objects
in the foreground conceal large objects in the background. Similarly,
in a narrative world presenting some hidden depth (let us call it a
"realistic world") there is something behind the narrated:
the characters have minds, intents, desires, and emotions, and the
reader is encouraged to reconstruct the content of their mind--either
for its own sake, or in order to evaluate their behavior. The
procedures of inference relating to inner life would be inhibited in
the case of the referents of human names in lyric poetry or in some
postmodern novels where characters are reduced to stereotypes,
actantial roles or allegories. When the reader feels that there is
nothing beyond language, inference procedures become largely
pointless.
- As is the case in VR systems, the
reader's sense of immersion and empathy is a function of the depth
of information. It is obvious that detailed descriptions lead to a
greater sense of belonging than sketchy narration. This explains why
it is easier be be caught up in a fictional story than in a
newspaper report. But in purely verbal communication--in contrast
with the visual or auditory domains--depth of information may reach
the point of saturation and create an alienating effect: the length
and minute precision of the descriptions of a Robbe-Grillet, as well
as their restriction to purely visual information, constitute a
greater deterrent to immersion than the most laconic prose.
- Breadth of information is not literally
possible in fiction, since we are talking about writing and not
about multi-media communication. But insofar as it relays sensations
through the imagination, literary language can represent the entire
spectrum of human experience. This ability of language to substitute
for all channels of sensation is what justifies the comparison of
literature with a multi-media mode of communication such as VR.
- Another factor of immersion that seems
at first glance impossible in textual communication is the control
of the sensors. The reader only sees (hears, smells, etc.) what the
narrator shows. But to the extent that the narrator's sensations
become the reader's, fiction offers a mobility of point of view at
least as extensive as that of VR systems. The development of a type
of narrator specific to fiction---the omniscient, impersonal
narrator--has freed fictional discourse from the constraints of real
world and pragmatically credible human communication. The
disembodied consciousness of the impersonal narrator can apprehend
the fictional world from any perspective (external observer point of
view or character point of view), adopt any member of the fictional
world as focalizer, select any spatial location as post of
observation, narrate in every temporal direction (retrospectively,
simultaneously, even prospectively), and switch back and forth
between these various points of view. Fiction, like VR, allows an
experience of its reference world that would be impossible if this
reference world were an objectively existing, material reality.
- The ultimate freedom in the movement of
the sensors is the adoption of a foreign identity. As Lasko-Harvill
observes, "in virtual reality we can, with disconcerting ease,
exchange eyes with another person and see ourselves and the world
from their vantage point" (277). This "exchanging eyes
with another person" is paralleled in fiction by the
possibility of speaking about oneself in the third person, or of
switching between first and third when speaking about the same
referent. (Cf. Max Frisch, Montauk.) But there is an even
more fundamental similarity between the role-playing of VR and the
nature of narrative fiction. As authors strip themselves of their
real world identity to enter the fictional world, they have at their
disposal the entire range of conceivable roles, from the strongly
individuated first person narrator (who can be any member of the
fictional world) to the pure consciousness of the third person
narrator.
- Both VR and fiction present the ability
to transcend the boundaries of human perception. Just as VR systems
enable the user to penetrate into places normally inaccessible to
humans, fiction legitimates the representation of what cannot be
known: a story can be told even when "nobody lived to tell the
tale." Of all the domain represented in fiction, no one
transcends more blatantly the limits of the knowable than foreign
consciousness. As Dorrit Cohn observes: "But this means that
the special life-likeness of narrative fiction--as compared to
dramatic and cinematic fiction--depends on what writers and readers
know least in life: how another mind works, how another body feels"
(5-6).
- The effacement of the impersonal
narrator and his freedom to relocate his consciousness anywhere, at
any time and in whatever body or mind conveys the impression of
unmediated presence: minds become transparent, and events seem to be
telling themselves. The mobility of the sensors that apprehend
fictional worlds allow a degree of intimacy between the reader and
the textual world that remain unparalleled in nonfiction.
Paradoxically, the reality of which we are native is the least
amenable to immersive narration, and reports of real events are the
least likely to induce participation. New Journalism, to the scandal
of many, tried to overcome this textual alienation from nonvirtual
reality by describing real-world events through fictional techniques.
In the television domain, the proliferation of "docu-drama"
bears testimony to the voyeuristic need to "be there" and
to enjoy fiction-like participation, not only in imaginary worlds,
but also in historical events.
Against Immersion
- Theories of fiction emphasizing
participation in fictional worlds represent a somewhat reactionary
trend on the contemporary cultural scene. Immersion in a virtual
world is viewed by most theorists of postmodernism as a passive
subjection to the authority of the world-designer--a subjection
exemplified by the entrapment of tourists in the self-enclosed
virtual realities of theme parks or vacation resorts (where the
visitor's only freedom is the freedom to use his credit card).
According to Jay Bolter, immersion is a trademark of popular culture:
"Losing oneself in a fictional world is the goal of the naive
reader or one who reads as entertainment. Its is particularly a
feature of genre fiction, such as romance or science fiction"
(155).
- As we have seen above, the precondition
for immersion is the transparency of the medium. But we live in a
semiotic age, in an age that worships signs. Contemporary theories
such as deconstruction teach us that the freedom of the mind must
originate in a freedom from signs. So does virtual reality, in some
respect, but while VR seeks this freedom in the disappearance of
signs, contemporary cultural theories regard signs as the substance
of all realities and as the prerequisite of thought. Freedom from
signs cannot be achieved through their disappearance but only
through the awareness of their omnipresence, as well as through the
recognition of their conventional or arbitrary character. The
aesthetics of immersion is currently being replaced--primarily in
"high culture" but the tendency is now stretching toward
popular culture--by an aesthetics of textuality. Signs must be made
visible for their role in the construction of reality to be
recognized. A mode of communication that strives toward transparency
of the medium bereaves the user of his critical faculties. The
semiotic blindness caused by immersion is illustrated by an anecdote
involving the XVIIIth century French philosopher Diderot. According
to William Martin, "he tells us how he began reading Clarissa
several times in order to learn something about Richardson's
techniques, but never succeeded in doing so because he became
personally involved in the work, thus losing his critical
consciousness" (Martin 58). According to Bolter, this l oss of
critical consciousness is the trademark of the VR experience: "But
is it obvious that virtual reality cannot in itself sustain
intellectual or cultural development. . . . The problem is that
virtual reality, at least as it is now envisioned, is a medium of
percepts rather than signs. It is virtual television" (230).
"What is not appropriate is the absence of semiosis"
(231).
- In reducing VR to passive immersion,
however, Bolter ignores the second component of the VR experience.
If contemporary art and literature are to achieve an enhancement of
the reader's creativity, it should be through the emulation of the
interactive aspect of VR, and not through the summary condemnation
of its immersive power.
Interactivity
- Interactivity is not merely the ability
to navigate the virtual world, it is the power of the user to modify
this environment. Moving the sensors and enjoying freedom of
movement do not in themselves ensure an interactive relation between
a user and an environment: the user could derive his entire
satisfaction from the exploration of the surrounding domain. He
would be actively involved in the virtual world, but his actions
would bear no lasting consequences. In a truly interactive system,
the virtual world must respond to to the user's actions.
- While the standard comparison for
immersion derives from narrative fiction, the most frequently used
metaphor of interactivity invokes theatrical performance. The simile
captures a largely utopian dream of dramatic art: putting spectators
on stage and turning them into characters:
As researchers grapple with the notion of
interaction in the world of computing, they
sometimes compare computer users to theatrical
audiences. "Users," the argument goes, are
like audience members who are able to have a
greater influence on the unfolding of the
action than simply the fine-tuning provided by
conventional audience response. . . . The users of
such a system are like audience members who
can march up onto the stage and become various
characters, altering the action by what they
say and do in their roles. (Laurel 16)
- Whereas immersion may be a response to a
basically static form of representation, interactivity requires a
dynamic simulation. A simulative system does not simply respond to
the user's actions by displaying ready-made elements, it creates its
data "in real time" according to the user's directions.
Like movies and narratives, a simulative system projects a world
immersed in time and subjected to change, but while these media
represent history retrospectively, fashioning a plot when all events
are in the book, simulation generates events prospectively, without
knowledge of the outcome. Taken as a whole, a simulative system does
not reproduce a specific course of events, but like a "Garden
of Forking Paths"--to parody the title of a short story by
Borges--it is open to all the histories that could develop out of a
given situation. Every use of the system actualizes another
potential segment of history. The simulative system is like an
alphabet containing all the books on a given subject, while the
simulation itself is the writing of a potential book (except that
there is no book left when the writing in completed). In a flight
simulator, for instance, the user enacts the story of one particular
flight out of a large set of possibilities by operating the keys
that represent the control panel of the airplane.
- The degree of interactivity of a VR
system is a function of a variety of factors. Steuer enumerates
three of them, without claiming that the list is exhaustive:
speed, which refers to the rate at which
input can be assimilated into the mediated
environment; range, which refers to the
number of possibilities for action at any
given time; and mapping, which refers to the
ability of a system to map its controls to
changes in the mediated environment in a
natural and predictable manner. (86)
The first of these factors requires little
explanation. The speed of a system is what enables it to respond in
real time to the user's actions. Faster response means more actions,
and more actions mean more changes. The second factor is equally
obvious: the choice of actions is like a set of tools; the larger
the set, the more malleable the environment. A VR system allowing an
infinite range of actions would be like real life, except that in
real life our choice of actions in a concrete situation is limit ed
by pragmatic considerations. The factor of mapping imposes
constraints on the behavior of the system. Insofar as
"mapping" is defined in terms of natural response, it
advocates the disappearance of arbitrary codes. Far from being
associated with passive immersion, semiotic transparency is
conceived by VR developers as a way to facilitate interactivity. The
predictability of the response demonstrates the intelligence of the
system. The user must be able to foresee to some extent the result
of his gestures, otherwise they would be pure movements and not
intent-driven actions. If the user of a virtual golf system hits a
golf ball he wants it to land on the ground, and not to turn into a
bird and disappear in the sky. On the other hand, the predictability
of moves should be relative, otherwise there would be no challenge
nor point in using the system. Even in real life, we cannot
calculate all the consequences of our actions. Moreover,
predictability conflicts with the range requirement: if the user
could choose from a repertory of actions as vast as that of real
life, the system would be unable to respond intelligently to most
forms of input. The coherence of flight-simulation programs stems
for instance from the fact that they exclude any choice of activity
unrelated to flying. Meaningful interactivity requires a compromise
between range and mapping and between discovery and predictability.
Like a good narrative plot, VR systems should instill an element of
surprise in the fulfillment of expectations.
Interactivity and Literary Theory
- Increasing the reader's participation in
the creative process, and thereby questioning such distinctions as
author/reader, actor/spectator, producer/consumer, has been a major
concern of postmodern art. This does not mean that without these
efforts reading would be a purely passive experience: theorists such
as Iser or Ingarden have convincingly demonstrated that a world
cannot emerge from a text without an active process of construction,
a process through which the reader provides as much material as sh e
derives from the text. But the inherently interactive nature of the
reading experience has been obscured by the reader's proficiency in
performing the necessary world-building operations. We are so used
to playing the fictional game that it has become a second nature: as
quasi native readers of fiction we take it for granted that worlds
should emerge from texts. This explains why postmodernist attempts
to promote active reader involvement in the construction of meaning
usually take the form of self-referential demystification. As Linda
Hutcheon writes: "The reader of fiction is always an actively
mediating presence; the text's reality is established by his
response and reconstituted by his active participation. The writer
of narcissistic fiction merely makes the reader conscious of this
fact of his experience" (141). The price of this consciousness
is a loss of membership in the fictional world. In the narcissistic
work, the reader contemplates the fictional world from the outside.
This world no longer functions for the imagination as an actual
world--this is to say, as an ontological center--but is expelled
toward the periphery of the modal system, where it acquires the
status of a non-actual possible world. The metafictional gesture of
de-centering thus inverts a paradox inherent to fiction. Insofar as
it claims the reality of its reference world, fiction implies its
own denial as fiction. By overtly recognizing the constructed,
imaginary nature of the textual world, metafiction reclaims our
"native reality" as ontological center and reverts to the
status of nonfictional discourse about non-actual possible worlds.
In order to enhance participation in, or at least awareness of the
creative process, the metafictional gesture thus blocks
participation in the fictional world.
- But the reader's interest is difficult
to maintain in the absence of make-believe. The most efficient
strategy for promoting an awareness of the mechanisms of
fictionality is not to block access to the fictional world, but to
engage the reader in a game of in and out: now the text captures the
reader in the narrative suspense; now it bares the artificiality of
plots; now the text builds up the illusion of an extratextual
referent; now it claims "this world is mere fiction."
Shuttled back and forth between ontological levels, the reader comes
to appreciate the layered structure of fictional communication, a
layered structure through which he is both (in make-believe)
narratorial audience in the fictional world, and authorial audience
in the real world. One of the most successful examples of this game
of in-and-out is John Fowles' The French Lieutenant's Woman.
The fictional world may be eventually demystified as a textual
construct, yet the text succeeds in creating an immersive
experience. At times the reader regards the characters as human
beings and invests an emotional interest in their fate; at other
times he is made to acknowledge their status as literary creations.
It is the memory of the immersive power of the text that engages his
critical faculties during the self-reflexive moments. The object the
reflexive activity is as much the phenomenon of immersion as the
artificiality of fictional worlds. But if immersion alternates with
an "interactive" stance toward the fictional world and the
plural ontological levels embedded in the textual universe, the two
experiences cannot occur at the same time. They imply mutually
exclusive perspectives on the reference world.
- When applied to traditional forms of
text--that is, preserved and transmitted in book
form--"interactivity" remains a largely metaphorical
concept. It stands more for the reader's awareness of his
collaboration with the text in the production of meaning than for
personal initiative and decision making. Not surprisingly, the
textual mode in which the ideal of interactivity comes closest to
literal fulfillment is hypertext, a form of writing made possible by
the electronic medium. The idea of hypertext is well-known and I
will do no more than summarize it. Organized as a network of
paragraphs connected by electronic links, hypertext offers at given
points a choice of directions to follow. Each choice brings on the
screen a different chunk of text, to which are attached new
branching possibilities. Rather than consuming the text in a
prescribed sequential order, the reader determines her own path of
traversal through the textual network.
- Through the initiative given to the
reader, hypertext realizes a very basic form of interactivity. As
Bolter observes: "The reader participates in the making of the
text as a sequence of words" (158). If we equate
"text" to one particular traversal of the network, then
indeed every reading session generates a new text, and the reader
takes an active part in this writing. In this view, "text"
is not a static collection of signs but the product of a dynamic
encounter between a mind and a set of signs. If the concept of text
is indissoluble from the act of reading, the physical interactivity
of hypertext is a concrete metaphor for the mental activity required
by all texts. While every particular path of navigation through a
hypertextual network brings to th e screen different chunks of text,
every particular reading of a non-electronic text highlights
different episodes, links different images, and creates a different
web of meaning. The difference between the experiences of hypertext
and of traditional text s is mostly a matter of intensity, of
awareness and of having no other source of pleasure than what
Nabokov calls "combinational delight" (69). In the absence
of the distraction created by a dominating storyline, it is hoped
that the reader will devote all of his attention to the tracking of
links.
- Alternatively, the concept of
"text" could be equated to the sum of possible readings,
or rather to the written signs forming the common source of these
readings. In the case of hypertext, this would mean that the text is
the entire network of links and of textual nodes. According to this
view, the interactivity of hypertext is not a power to change the
environment, as is the case in VR systems, but merely a freedom to
move the sensors for a personalized exploration. The reader may
choose in which order she visits the nodes, but her choices do not
affect the configuration of the network. No matter how the reader
runs the maze, the maze remains the same. Far from relinquishing
authority (as Bolter has claimed), the author remains the hidden
master of the maze. The reader's actions could only modify the
environment if the hypertextual system generated text in real time,
as an intelligent response to the reader's decisions."1
As I have argued above, this is what happens in simulative systems.
The computer calculates the position of the plane according to the
user's input, rather than displaying a pre-calculated position. This
will not happen in hypertext until it joins forces with AI--and
until AI sharpens its story-generating capabilities. In the
meantime, the closest to a hypertextual system operating in real
time will be for the user to get on line with the author herself.
- The fullest form of interactivity occurs
when the reader is invited to contribute text to the network. "2
This invitation may take one of two forms. The first possibility is
the user adding text and links which become permanent parts of the
system. When this input concerns a specific character, the user is
less playing the role of the character in question than taking over
authorial responsibilities for the writing of his fate. In other
words, the user manipulates the strings of a puppet, playing its
role from the outside. The other conceivable form of interactivity
is more like playing a game of make-believe such as cops and
robbers. The system defines a cast of characters by specifying their
attributes. The user selects an identity from this repertory, and
plays the role from the inside. She encounters other users playing
other characters, and they engage in a dialogue in real time. This
dialogue does not count as description of the actions of the
character b ut as performance of these actions: the character's
freedom to act is a freedom to select speech acts. Of these two
modes of contribution, only the second constitutes an immersive
experience. The first may be addictive--as any game, any activity
might be--but it maintains a foreign perspective on the fictional
world.
Immersion or Interactivty: The Dilemma of Textual Representation
- Whether textual interactivity takes the
weak form of a deliberate play with signs leading to a production of
meaning, or the strong form of producing these signs, one
consequence appears unavoidable: in literary matters, interactivity
conflicts either with immersion or with aesthetic design, and
usually with both. The strong forms of interactivity run most
blatantly into the design problem: how can the contributions of the
reader-turned-author be monitored by the system, so that the text as
a whole will maintain narrative coherence and aesthetic value? An
interactive system may be an alphabet for writing books, but the
user should be prevented from producing nonsense. As Laurel argues:
"The well designed [virtual world] is, in a sense, the
antithesis of realism--the antithesis of the chaos of everyday
life" (quoted by Pimentel and Texeira 157). Howard Rheingold
stresses the need for "scenario control": "They [VR
developers] want a world that you can walk around in, that will
react to you appropriately, and that presents a narrative structure
for you to experience" (307). The control of a pre-determined
narrative script imposes severe limits on the user's freedom of
moves (think of the narrow range of choices in the children's books
"Choose Your Own Adven tures," where all the branches
constitute a coherent story); but without this control the
hypertextual network would turn into a multi-user word processor. In
the worst case scenario, interactive fiction will be reduced to a
bunch of would-be authors e-mailing to each other the fruits of
their inspiration.
- In the weaker forms of interactivity,
design is easier to control, but immersion remains problematic. The
reader of a classical interactive fiction--like Michael Joyce's Afternoon--may
be fascinated by his power to control the display, but this
fascination is a matter of reflecting on the medium, not of
participating in the fictional worlds represented by this medium.
Rather than experiencing exhilaration at the freedom of
"co-creating" the text, however, the reader may feel like
a rat trapped in a maze, blindly trying choices that lead to
dead-ends, take him back to previously visited points, or abandon a
storyline that was slowly beginning to create interest. The best way
to prevent this feeling of entrapment, it seems to me, would be to m
ake the results of choices reasonably predictable, as they should be
in simulative VR, so that the reader would learn the laws of the
maze and become an expert at finding his way even in new territory.
But if the reader becomes an expert at running the maze, he may
become immersed in a specific story-line and forget--or deliberately
avoid--all other possibilities. He would then revert to a linear
mode of reading and sacrifice the freedom of interactivity.
- It would be preposterous to pass a
global judgment on the intrinsic merit of hypertext: whether the
maze is experienced as a prison or as the key to freedom depends on
the individual quality of the text and on the disposition of the
reader. But I would like to advance one general pronouncement
concerning the immersive power--or lack thereof--of the genre: a
genuine appreciation of a hypertextual network requires an awareness
of the plurality of possible worlds contained in the system; but
this plurality can only be contemplated from a point of view
external to any of these worlds.
- The various attempts by contemporary
literature to emulate the interactivity of VR thus involve a
sacrifice of the special pleasure derived from immersion. The more
interactive, the less immersive the text. The texts that come the
closest to combining both types of pleasure are those that
orchestrate them in round-robin fashion through a game of
in-and-out. The textual incompatibility of immersion and
interactivity can be traced back to several factors. While immersion
depends on the forward movement of a linear plot, interactivity
involves (and creates) a spatial organization. While immersion
presupposes pretended belief in an solid extratextual reference
world, interactivity thrives in a fluid environment undergoing
constant reconfiguration. While immersion looks through the signs
toward the reference world, interactivity exploits the materiality
of the medium. Textual representation behaves in one respect like
holographic pictures: you cannot see the worlds and the signs at the
same time. Readers and spectators must focus beyond the signs to
witness the emergence of a three-dimensional life-like reality.
- In computer-generated VR, immersion and
interactivity do not stand in conflict--or at least not necessarily.
Immersion may offer an occasional threat to interactivity"3,
but the converse does not hold. The more interactive a virtual
world, the more immersive the experience. There is nothing
intrinsically incompatible between immersion and interactivity: in
real life also, the greater our freedom to act, the deeper our bond
to the environment.
- An obvious reason for this difference in
behavior is the above-mentioned difficulty for texts to integrate
the reader's input into a coherent narrative macro-structure. VR
also experiences this type of problem when it attempts to turn users
into the characters of a multi-media dramatic production. It is in
very restricted domains regulated by narrowly defined
"narrative" scripts--flight simulators, golf, paddle-ball,
etc.--or in areas not subjected to the requirements of narrative
logic--visual displays, or systems combining visual data with sound
and dance--that VR systems achieve the most complete fusion of
immersion and interactivity.
- But there is a more fundamental reason
for VR's ability to combine the two types of experience. In a
textual environment, the tools of interactivity are signs. But signs
are not the only mean of action. In the real world we can act with
the body by pointing at things, manipulating them, and working on
them with tools. We can also use the body as an instrument of
exploration by walking around the world and moving the sensors.
Virtual reality, as its developers conceive it, reconciles immersion
and interac tivity through the mediation of the body. "Our body
is our interface," claims William Brickemp in a VR manifesto
(quoted in Pimentel and Texeira, 160). When the reader of a
postmodern work is invited to participate in the construction of the
fictional world she is aware that this world does not exist
independently of the semiotic activity; hence the loss in immersive
power. But the user of a VR system interacts with a world that is
experienced as existing autonomously because this world is
accessible to m any senses, particularly to the sense of touch. As
the story of Saint Thomas demonstrates, tactile sensations are
second to none in establishing a sense of reality. The bodily
participation of the user in virtual reality can be termed
world-creative in the same sense that performing actions in the real
world can be said to create reality. As a purely mental event,
textual creation is a creation ex nihilo that excludes the
creator from the creation: authors do not belong to the world of
their fictions. But if a mind may conceive a world from the outside,
a body always experiences it from the inside. As a relation
involving the body, the interactivity of VR immerses the user in an
world already in place; as a process involving the mind, it turns
the user's relation to this world into a creative membership. The
most immersive forms of textual interactivity are therefore those in
which the user's contributions, rather than performing a creation
through a diegetic (i.e. descriptive) use of language, count as a
dialogic and live interaction with other members of the fictional
world. I am thinking here of children's games of make-believe, and
of those interactive hypertextual systems where users are invited to
play the role of characters. These modes of interactivity have yet
to solve the problem of design, but they point the way toward a
solution of the conflict between immersion and interactivity: turn
language into a dramatic performance, into the expression of a
bodily mode of being in the world.
|