1529 Armacost Ave #9
Los Angeles, CA
ashes@zennet.com

The following is a transcript of the video documentation for my MFA thesis project in design and media art. The video is approximately six minutes in length and is intended as a basic introduction to the first implementation of this approach towards interactive storytelling. Copies of the video are available on request.

[Begin transcript of Dr. K---]

Office. A bench. A coat rack. Some furniture.

Office. A bench. A coat rack. A desk.

A character enters.

"A place. Some scenery." So begins the narrative of Dr.K---, a unique virtual reality experience that lures the audience into the captivating world of early Nineteenth Century medical practices.

Envisioned as an investigation into a user responsive environment, the Dr. K--- project draws upon powerful ideas of improvisational theater and narrative theory. Looking through a special viewing apparatus, a participant has access to the virtual story interface. By manipulating a pointer within a page of text, the participant is allowed to manipulate the story in a straightforward fashion.

When an active area of the story is selected, a storytelling engine regenerates the scene in relation to the selected element.

In addition the virtual environment, a recreation of an anatomist's office exists within the gallery space. This setting serves several functions: as a lounging area, before and after the "examination", and as a stage for live performance.

[excerpt #1 from live performance]:

Armory. A wall. The wall is old and warped. A curtain. A weapon rack. The sound of a barking dog.

A character. An area/location. The character is in a relation with the area/location. A victim. A man.

The boy interacts with the sound of a barking dog.

A place. Some scenery.

A place. A curtain

A place. A curtain. Some scenery

A place. A curtain. A portal.

[end of excerpt #1]

There are several technological aspects to Dr. K---: a story database [diagram], a client interface [diagram], the viewing device [diagram], and a large Victorian mouse.

The Victorian mouse is a primitive but effective method for navigating through a virtual environment. Two independent levers control horizontal and vertical motion of a display pointer. By moving the pointer over an active area of the display, the scene is modified, and the story advances.

[excerpt #2 from live performance]:

A place. Sounds of people. The floor. Some bones. Some stairs. The ground. Some scenery.

A place. Sounds of people. The floor. Some bones. Some stairs. The ground. Some litter.

Street. Sounds of people. The floor. Some bones. Some stairs. The ground. Some litter. A couch.

Street. Sounds of people. The floor. Some bones. Some stairs. The ground. Some litter. The litter is scattered all over. A couch.

presented a part of the UCLA MFA exhibition for design and media art, june 26-july 3 1998.

[End transcript of Dr. K---]

The particular approach to the creation of narrative in this project revolves around – as demonstrated by the excepts above – a continual recreation of textual scenes in an ongoing dialogue between the storytelling engine and the user. Activity is focused on the actual construction of the story. It is not a story developed in "real time", but in a lurching and discontinuous fashion, as if the audience were continually asking questions to a storyteller while the storyteller performs.

When the user makes a selection, the engine suspends the present exposition:

Engine: A character enters.

User: Who is the character?

and backtracks to resolve the element of interest:

Engine: A man enters.

User: Okay.

This form of interaction is in sharp contrast to the branching dialogue used in many computer games, where an explicit choice of user responses is displayed and the dialogue is typically between the user’s character and a character within the story. Here the choices are implicit – any fragment of the story can be selected, and the dialog, such as it is, takes place outside the story. The user does not control a character within the story. Additionally, there is little user knowledge about how a selection will influence the story. The "character" above may turn out to be "the victim", or "Mrs. I---", or the issue of character identity may be ignored by the story engine entirely:

User: Tell me more about "a character".

Engine: A character enters from the stairs.

As a result of this interaction, elements which are initially generic in form – some scenery – tend to become more clearly focused – some litter – when the user prompts for more detail, and tend towards more explicit detail – the litter is scattered all over - with continued prompting.

Not only do elements come into focus, they can lose focus through neglect. Elements revert to more archetypal forms, "a desk" becomes "some furniture". The scene is in continual flux.

Because some elements must be in focus for other elements to lose focus, there is a gradual buildup of more-or-less focused elements as the story progresses. Highly focused elements are assumed to be of interest to the user, are considered to be of more consequence in the story, and are thus less likely to lose detail through neglect.

The flux of the story occurs on three levels. First level changes occur on a kind of object/detail level: characters, props, scenery. The second level is flux within a scene: the accumulation of details or objects which eventually define a named location, relationships between objects, and actions involving the objects. Finally there is the flux of the overall story: changes from one scene to another, recurring characters and locations. [The current implementation of the story level flux is still very rudimentary.]

Weighted choices are made when elements flux into more/less focused states. Participant input is weighted depending on the current state of the story: if there are a quantity of details in a scene, new actions are introduced into the scene, otherwise the state of current details will be modified or new details added. When a certain quantity of actions has been executed in the scene, the engine resets the scene and the user returns to exploring the first level of narrative.

Visually, the flux of elements is represented by the opacity of the text in the interface. 80% black text indicates a very focused element, while 20% black text is almost invisible and indicates that the element is a candidate for change. Flux is measured by a combination of variables: the age of the element (how long since the element was last selected), how many times the object has been selected, the relevance of the element to the current scene (a table is relevant in a kitchen, a tombstone is relevant in a graveyard).

A simulation operates on a mechanical model of the universe. A simulation may incorporate some randomness as a substitution for unknown or undefined variables in the system, but is otherwise made up of a collection of determinate properties: objects, physics, continuity, &c. Because simulations are frequently modeled on the "real" world, and because objects in the real world are not spontaneously generated or destroyed (above the quantum level), the simulated world exists in a knowable state at any given moment. In other words, someone has built all of the virtual walls, floors, and lemon trees of the simulated world ahead of time. Simulation lends itself to highly representational presentation – immersive 3D virtual reality.

A fabrication, such as that proposed by the Dr. K--- project, operates on a potential model of the universe. There is randomness as a means to add diversity to the world, which would otherwise remain a homogenous world of potential. Exploring a fabrication creates an artifact, a fixed collection of objects and events, and the world is not otherwise knowable except by exploration.

Briefly, some characteristics that influence the quality of experience within simulated/fabricated worlds:

Repeatability: The same experience can be repeated multiple times by many users with a simulation. This is a scientific quality; repeating the experience is a useful way to test that the system "works". The fabricated experience is not prone to repeatability, each experience is intended to be unique for a particular user across repeated interactions.

Direct user control: The user can have control over one or more agents within a simulation. User control in a simulation will often "break" the simulation, much like tinkering with the innards of a clock may prevent it from working properly. The user has only limited control of agents within a fabrication. When the potential of the fabrication is exhausted, agent actions become determinate and thus out of control of the user.

Viewpoints: Because the simulated world is well-defined, there are a potentially unlimited number of viewpoints into the world. Virtual reality often conflates the high quantity of viewpoints with the notion of "interactivity". Because fabrication does not allow the user an unrestricted view of the world, a fabrication has only a fractional number of viewpoints.

Diversity of landmarks: A landmark is a memorable element in the experience. It can be a character, a location, a special event. In a simulation, the user may experience the same situation repeatedly. Only highly unique situations stand out from all others, resulting in a low number of landmarks. Fabrication is concerned with the direct construction of these landmarks; fabrication fails if there are no distinguishable landmarks.

Representation: Simulation is dominated by high-fidelity representation. The higher the desired quality of representation, the longer the production cycle will be. If the simulated world is unclear or ambiguous there is a gap in user comprehension. With fabrication, ambiguity is the vehicle for exposition; to represent the world piece by piece is to tell the story of that world.

In the worst case scenario, the characteristics of a simulation result in a guided tour experience. So much effort is invested in creating the world, testing the world, and making the world look as good as possible, the author must ensure that the user does not miss any of the "good stuff". Indeed, the highly designed areas are the most aesthetically pleasing, given the otherwise mundane experiences that make up a majority of the simulated world.

There is one major pitfall to avoid when creating a fabricated world: the repeated generation of un parseable nonsense. There are few guarantees that the world won't revert to some more potential form, a more random collection of elements, at the wrong moment.

Dr. K--- uses a few different approaches to maintain some level of comprehensibility within the narrative.

The first approach is the choice of focused subject material for the story. Dr. K--- is constructed around the historic account of William Burke and William Hare in 1820's Edinburgh. The elements of the project were drawn from this particular story, elements that when viewed in quantity create a underlying setting and mood for the story. (Burke and Hare were notorious criminals – they made a living by killing people and selling the bodies to an anatomy school.)

Second, the project is presented within a theatrical setting. The elements of the set – a desk, a bench, a tea chest – echo the elements within the story engine, reinforcing the place and mood of the story. Even when incongruous elements are presented by the story engine, the theatrical setting helps to emphasize the hidden relationships between those elements.

Finally, the story engine itself contains some rudimentary code that attempts to steer the story in a logical direction.

The Dr. K--- engine contains a number of author-defined structures, an ontological collection of elements placed into non-disjoint classes. For example, detail elements are placed in these classes:

~floor: [ ~floor, ~rug ]

~cloth: [ ~cloth, ~rug, ~curtain ]

~wall: [ ~wall, ~curtain ]

A ~rug belongs to both the ~floor and the ~cloth class. Should a ~rug instance revert to an instance of ~cloth, it may subsequently resolve from the ~cloth instance into a ~curtain instance. An element that was once the "floor" is now the "wall"! A secondary consequence is that a character formerly "lying on" the rug is now "standing next to" the curtain!

Actions are also classified according to their properties:

~character: [ ~character_action, ~character_enter,

~character_attack ]

~enter: [ ~character_enter, ~character_enter_via_portal ]

~portal: [ ~character_enter_via_portal,

~character_exit_via_portal ]

Characters are introduced to the scene as a consequence of being the object of an action. A character is not required to "enter" the scene, the character's presence can simply be implied by their reference within an action. The ~enter action only establishes a character's presence after that action has been executed by the engine, thus:

[time 0] Kitchen. A table. A man enters.

[time 1] Kitchen. A table. A man.

The sequential order of actions is determined by their order of execution by the engine:

[time 1] Kitchen. A table. A man. The man attacks a

character. The man leaves.

[time 0] Kitchen. A table. A boy. A man enters.

[time 1] Kitchen. A table. A boy. A man. The man

attacks the boy. The man leaves.

Should the ~exit action get executed next, the ~character_attack action will have to change, either by reverting to a ~character action or by replacing the obsolete "a man" reference to "a character" – a character that doesn't yet exist in the scene.

This project is documented and presented as an artistic exploration into interactive storytelling. As an author and artist I feel that the current emphasis on simulation and photo-realistic representation within the discipline of "interactivity" needs to be critically reevaluated.

The main source of "intelligence" in Dr. K--- is the cognitive ability of the participant herself. The ability of a human mind to recognize and understand symbols, even when those symbols are distorted and occluded, is key to the operation of a minimal representation. This has led me to a difficult question:

Can one convey meaning [have connotation] without creating symbols [without denotation]?

Where is meaning generated within a narrative? One can manipulate the associations of objects without denoting the objects themselves: "There exists an entity that performed an action at some moment." This entity can be perceived as being threatening/compassionate without any change in representation, if the action is resolved as being threatening/compassionate. Alternatively, actions can become threatening/compassionate without any change in representation if the objects associated with them have threatening/compassionate characteristics. But these subtle interactions are not possible with the typical models of representation – in particular with the current model of virtual reality and its goals of photorealistic visualization. Virtual reality presents all associations at once, from any viewpoint. This leaves little room for new or alternate associations.

If these associations are made to be mutable, as has been attempted in Dr. K---, one can exploit that critical moment of interactivity when the unresolved associations are firmly realized – when a collection of objects becomes a place, when a character becomes a villain – and one can discover the moment in which is contained the seed of drama and the creation of narrative.