MERL - A Mitsubishi Electric Research Laboratory

201 Broadway

Cambridge, MA 02139 USA

This is a preliminary design document for narrative systems in which multiple users interact in a manner reminiscent of the chorus in ancient Greek theater. The work extends premises originally described in Working Notes of the 1995 AAAI Spring Symposium on Interactive Story Systems (held at Stanford University), and represented in a prototype for a single-user platform (Strohecker et al. 1999). The current work constitutes a "what-if?" sort of inquiry, considering multiple simultaneous interactors with a narrative system in which tangible or otherwise palpable computational objects both facilitate and reflect actions and conversation.

Introduction

This note stems from an experiment in which the theatrical device of Greek chorus serves as a model for interactions with a computational narrative (Strohecker et al. 1999). That experiment is largely structural: players’ interactions unfold progressively finer detail about the story, but do not change the course of events or modify the characters. Players interact mainly by querying representations of chorus members, who comment on the narrative from different perspectives. Here I consider how the chorus model might be pushed further, such that viewers become chorus members in some more direct sense, thereby contributing to the choral aspect of the narrative system as it grows over time. My musings include a discussion of the psychological process of introjection and how it could guide thinking about systems that change based on players’ interactions. The usefulness of considering introjection (and its counterpart process of projection) with regard to systems development has to do with humans’ affinity for manipulating objects and imbuing them with meanings that can be shared or personal.

An important assumption is that the psychoanalytic tradition has helped to articulate processes of human thinking that need not, and often should not, be considered pathological. Many of the cases reported in the literature describe reasonable responses to unusual situations. Furthermore, (Winnicott 1971a, b) and (Fairbairn 1963) are careful to note that they believe the particular behaviors and thinking patterns cited here (i.e., the use of transitional objects and the phenomenon of object-splitting) to be among normal human developmental processes.

Another assumption concerns the use of physical objects as devices facilitating interactions with computational systems. It would be problematic to assert that a physical object can be designed so particularly as to be obviously suited to a specific purpose: (Norman 1988)’s thesis is countered by discussions of the bricoleur who collects objects and adapts them to various purposes as the need arises; the purpose may very well differ from that for which the object was originally designed (Levi-Strauss 1966, Pirsig 1984 [1974], Turkle and Papert 1990). This human capability to reinvent mandates allowance of a range of creative, unpredictable uses for any given object, but does not deny that an object may indeed also be useful for whatever purpose its designer may have intended or desired. The rudimentary design principles considered here rely on variability of taste and concept as different people choose and use objects for interactions with narrative systems. Nevertheless the range may also include commonalities of interpretation among interactors. Both results would be useful and potentially interesting.

Forms of dramatic chorus appear in theater works of various cultures. In performances by Ladysmith Black Mambazo, for example, the chorus address audience and actors, providing both narrative continuity and musical entertainment. In the film, Little Shop of Horrors (Oz 1986), a chorus of singer/dancers appears from time to time, amplifying story events and characters’ emotional states. Another film, Annie Hall (Allen 1977), embodies a chorus-like function in textual commentary rather than personifications. In the balcony scene, Annie and Alvy chat about wine and tennis while a literal subtext is displayed at the bottom of the screen, comically revealing inner monologues replete with worries that the other person will not find the speaker attractive. The role of the chorus can be so diverse that it provides a rich base of potential functions for computational narrative systems.

In ancient Greek theater, the chorus evolved through several eras and incarnations. Initially the chorus were separate from the actors in role, location, and appearance, but over time they merged more and more with the actors and the action. At first a collective, singing, dancing, and speaking together, chorus members gradually emerged as individual speakers. Often, a chorus leader spoke for the group or provided coherence for their diverse expressions. Originally anonymous, their uniform identities accentuated by masks, chorus members gradually appeared as personalities with particular views on issues and events. And, beginning as a theatrical device whose function was to express mood or tone, and whose contribution was to witness, comment, or clarify, the chorus gradually shared responsibility for delivering pieces of the narrative.

Thus the chorus may personify, magnify, subdue, transpose, interpret, retell, frame, or give perspective to the narrative action. Chorus members may fill in holes in the narrative, foreshadow the action, and reflect the action by re-enacting it in other modalities. The chorus may also act as a kind of buffer, an intervening layer that protects the audience from incidents too frightening to experience directly, such as the multiple deaths in Antigone.

Proposing the chorus as a model for interactions with computational narratives recalls the notion of "computers as theatre," a phrase coined through consideration of the use of metaphor, so well understood in theatrical contexts, in development of graphical user interfaces and other experiential aspects of computational systems (Laurel 1991). In later work, Laurel experimented more explicitly with theatrical spaces supported by computation (Bates 1992, Laurel et al. 1994). My discussion assumes similar contexts but focuses on users as a collective whose interactions help shape the system as it grows over time, and whose interactions are supported by tangible or otherwise palpable objects. The design premises are supported by literature on the history of theater (Strohecker et al. 1999), as well as theories of psychological development (Strohecker 1991).

Psychologists describe the phenomenon of "introjection" (also called "incorporation," "internalization," or "identification") in various ways.

In Sigmund Freud's model, our minds are in a particular state as we come into the world. They are like bundles of energy, instincts, drives — what he calls the "id." Gradually, through interactions with the environment, an infant begins to form a sense of itself and of others. The infant's mind is changing in the process: the id is giving rise to the "ego," and for several years the two will co-exist as the mind of the child. In Freud's theory, the mind undergoes another fundamental change at the time of the oedipal conflict (at about age six). The ego gives rise to the "super-ego," which will supervise the ego as it is influenced by the unchecked urges of the id. The child now has a "conscience," an internal model of the parents' values and castigations. The superego develops through a certain kind of interaction between child and parent, and mind and environment:

Two aspects of this explanation later became important to a group of psychoanalysts in what became the "object-relations" school: these included the concept of identification, and extensions of the idea of the so-called "split" that occurs in the ego as the superego develops.

The object-relations theory of human development is concerned with the importance of Freud's concepts of identification and object choice to the early relationship between infant and mother (or other caregiver), and to later relationships in the life of the individual. The theory makes use of several senses of the word "object," which sometimes refers to an actual person (or part of that person), sometimes to a representation of a person (which exists in the mind and has developed through a process of identification), and sometimes to an external object that substitutes temporarily for a person (while the process of identification is occurring).

W. R. D. Fairbairn, one of the foundation thinkers for the direction of psychoanalytic theory toward object-relations, makes use of the concepts of identification and object choice, but explains a process of ego-splitting that differs from Freud's. In Fairbairn's model, there is no id. An ego is present from birth, and early interactions with the caring figure play a crucial role in how the infant's ego develops. Fairbairn's discussion of the "internalization of the object" is concerned with the infant's way of coming to understand the alternating presence and absence of the caring figure. The frequency and duration of the absences increase, of course, as the child grows.

When alone, the infant is thought to keep in mind some image or internal representation of the caregiver. This representation develops when the infant is with the other person: their interactions include the infant’s process of internalizing the person or part of the person. The internal representation that results is called an "object," which at first is relatively simple. However, given that the external version of the object sometimes satisfies the infant but sometimes does not, the object is perceived as having a double nature. Both the external and internal versions of the object are said to be both "exciting" and "frustrating" (or "rejecting"). As these two aspects are acknowledged, the internalized object becomes more complicated: its exciting and rejecting aspects "split off from the main core of the object" (Fairbairn 1963, p. 224). Then, because these aspects are accompanied by powerful, problematic feelings with which the infant struggles, they are repressed by the ego. Thus Fairbairn sketches a scenario of progressive splitting of internal objects as the external separation is enacted.

Melanie Klein, another influential theorist of the object-relations school, posits

Experiences with caregivers lead to the infant's formation of internal objects that represent the figures, as well as to specific relationships with these inner objects.

D. W. Winnicott also describes the formation of emotionally charged inner objects (Winnicott 1971a, b). For Winnicott, the infant's process of separating from the mother occurs gradually and involves the progressive distinction of "me" and "not-me" objects. Here again, the "objects" are internal conceptions — of the mother and of the infant's own "self." The model is one in which the infant moves slowly from a sense of being merged with the mother (then understood as a "me" object), to a sense of autonomy as a separate individual (when the mother is understood as a "not-me" object). Winnicott sketches the dynamic using distinct spaces that are meant to represent concepts or states of mind — one for the "me," one for the "not-me." They are mediated by a space that lies between them, in which play occurs. Through play, the distinction of "me" and "not-me" comes to be understood — that is, the process of separation occurs.

"Transitional objects" assist in the process. The child substitutes something tangible (such as a blanket, teddy bear, or favorite toy) for the physical closeness to another person, which by necessity lessens as the child grows. The substituted object is a sign of a process of psychological incorporation of the person. This incorporation is what enables the child to tolerate realities of the loss of the external relationship. What develops in its stead is an internal relationship, with a representation of the loved one. In other words, not only does an internal object develop, but some quality of relationship with that object is internalized as well.

When a physical object plays such an important role in such a highly charged emotional process, it is likely to have been carefully chosen, based on inherent properties that resonate with some sense of the person for whom it is meant to compensate, or the situation in which it is used.

Objects can play a comparable role in situations that do not involve a significant loss. Papert describes the importance of gears in his early thinking:

In Papert's language, structures "grow out of one another"; in Freud's and Fairbairn's, one structure "splits off from" another. Papert's description is more constructive in tone, but the notions of derivation are strikingly similar. Klein and Fairbairn describe how internal objects become infused with affect, emotional charge, as they are formed; Papert describes how they "acquire both logical and emotional form." The person’s sense of the nature of the external object influences the interaction.

Piaget describes the importance of actions — observable, overt activities — as the operational "glue" that holds together certain structured understandings.

Thus actions with objects — doing things to and with objects — may constitute a crucial aspect of learning and other kinds of emotionalized thinking. (Turkle and Papert 1990) assert that many people of all ages prefer thinking with objects, moreso or rather than with abstractions. Objects may facilitate transitions from one thought to another, or one emotional state to another, at any age; furthermore the thinking may pertain to any topic. Perhaps those who prefer this style of thinking would be especially inclined toward commentary on narratives through the use of manipulable objects.

Game and film manufacturers sometimes produce supplemental media to extend their audiences’ experience of narratives. For example, Nintendo publishes magazines that include solutions, character descriptions, accounts by skilled players, and so on. The experience of a game is not limited to sessions in which the player is actually engaged with the machine, but extends through reading and discussion at other times. Relevant media include game cartridges and magazines, even dolls or other facsimiles of characters and objects within the game. Similarly, the producers of Toy Story (Lasseter 1995) released a line of dolls and other toys to augment the cultural impact of the film. Involvement with the narrative includes not just the suspension of disbelief, identification with characters, and other processes of film-watching, but creative processes involved in play with objects that can be held, moved, and transformed.

These strategies for extending a narrative context through use of a range of media tacitly acknowledge people’s use of objects to mediate thought. They also pave the way for development of objects that augment a narrative context not just in players’ minds, but also within a computational system, through communication back to the system and the narrative it presents.

The theatrical chorus is concerned primarily with commentary: the collective enriches the audience’s experience of the narrative through verbal, often musical, articulations of surprises, fears, reconciliations, and other responses to dramatic events. We could imagine objects associated with a narrative system as being recorders, transmitters, and/or players of spoken or chanted commentary. However, filtering and editorial functions would likely soon be needed in order to manage the accumulating data, and despite progress in the field of computational linguistics, the effort could be daunting and the results disappointing. Another approach could be to designate a live coordinator, like an emcee or editor, to orchestrate or constrain players’ interactions (e.g., Laurel et al. 1994). Indeed, this role bears some resemblance to that of the traditional chorus leader. Using objects as mediating devices suggests still another approach, which would distribute control of the systems’s growth among the interactors-as-chorus-members.

Choral comments typically occur at a meta-level: the narrative proceeds as characters enact their roles within the structure of the plot; the chorus observes the enactments and comments on meanings, implications, and outcomes. This function requires an alternation of time or space, and lends itself to re-tellings of stories that the audience already knows. The chorus doesn’t change the course of events, but enriches understanding of it through multiple perspectives. The chorus may be best suited to stories in which many truths are possible, such as the moral dilemmas of tales like Antigone and the arrest of Rosa Parks (Strohecker et al. 1999).

Discussion of many interpretations of such stories is essential to appreciating them. Sharing objects that pertain to the story could facilitate discussion among participants (Bellamy et al. 1994), as well as providing means through which they interact with the narrative system. For example, coins, a bus driver’s hat, and a policeman’s badge are objects that could facilitate changes of point of view within the Rosa Parks story. An interactor holding the police badge may shift focus from the morality of personal rights to the morality of civic responsibility. An interactor with the coins may take the opportunity to deposit them at the front of the bus, only to be told to exit and re-enter from the back. Of interest in developing the choral aspect of the system are the responses that different interactors may have to such prompts. These responses may register in forms other than verbal: for example, the system may associate specific facial expressions or manual gestures with certain emotions (e.g., Pinhanez 1999, Wilson and Bobick 1999, Wren 1999).

Multiple interactors could convene in a real-world "smart room" sort of theatrical space, in a local-net multiuser virtual domain, or via a web-based graphical environment. Each of these milieus implies a different scale in terms of the number of simultaneous participants, and each scale would necessitate particular strategies for coordinating the multiple inputs, but the basic modes of interaction could apply to each. For example, chorus members could be represented as masks that the interactors don as they approach or enter the computational domain. While not mandating any particular comment, the representational quality of each mask could suggest a view or mood to which the player would respond. The role would be interpretable by each player. Interactions related to the mask could accumulate through successive enactments, in turn modifying the appearance of the mask to register the various expressions. For example, a player’s speech prosody may affect the facial expression of the mask (e.g., Cahn 1999). It may even cause a new mask to spin off, as a prop for future chorus members. Ultimately, the narrative system would reflect a plurality of influences, and the choral aspect would become much richer than the original set of masks. The cumulative feedback would create a kind of collective construction, perhaps reminiscent of efforts like the AIDS quilt.

Objects handed from one player to another could act as secondary registers, supplementing the choral masks. The form and function of the object would contribute to its evocative qualities, and interactors would respond in kind. The timing of an object’s appearance could create shared moments of interest that further coordinate interactions. Objects would become "conversational props" that enliven the experience of the narrative and emphasize the role of the chorus (Bellamy et al. 1994).

Objects associated with the computational system could include physical, manipulable props that collect and transmit information about who is using them, how, when, for how long, etc. (e.g., Brave et al. 1998, Resnick 1998, Ullmer and Ishii 1999). Representations within a web-based graphical interface could refer to the physical objects, or to other facets of the system; in any case, computational structures associated with objects could keep track of aspects of their use (Gamma et al. 1977), informing development of the choral aspect of the system. Less frequently handled objects could be made more accessible, etc. Displays of computational objects could be supported by a range of audio-visual material, including film, video, illustrations, photographs, and/or computer-generated graphics. Each modality suggests techniques by which granularity of media can be controlled, enabling participants’ access to aspects of the narrative and the choral contents (e.g., Agamanolis and Bove 1997, Casey 1998, Pinhanez 1999).

As interactors considered the props with respect to characters and events, their voices would feed back in various forms to develop the commentary on the narrative. This notion of multiple voices shaping the system reflects contemporary models of mind (Bakhtin 1981, McGoldrick 1982, Minsky 1986, Mondykowski 1982, Wertsch 1991).

Thanks to Warren Sack for the helpful discussions. This work is supported by MERL — A Mitsubishi Electric Research Laboratory.

Agamanolis, S., and Bove, V.M. 1997. Multilevel Scripting for Responsive Multimedia. IEEE Multimedia, Oct.-Dec., pp. 40-50.

Allen, W., dir. 1977. Annie Hall. United Artists.

Bakhtin, M. 1981. The Dialogic Imagination: Four Essays by M. M. Bakhtin. M. Holquist, ed., Caryl Emerson and Michael Holquist, trans. Austin: Univ. of Texas Press.

Bates, J. 1992. Virtual Reality, Art, and Entertainment. Presence 1:1, 133-138.

Bellamy, R. K. E., Cooper, E. B. W., and Borovoy, R. D. 1994. Supporting Collaborative Learning through the Use of Conversational Props. In Proceedings of the East-West Conference on Human-Computer Interaction. Berlin: Springer-Verlag.

Beth, E. W., and Piaget, J. 1966. Mathematical Epistemology and Psychology. W. Mays, trans. Dordrecht-Holland: D. Reidel.

Brave, S., Ishii, H. and Dahley, A. 1998. Tangible Interfaces for Remote Collaboration and Communication. Proceedings of CSCW '98, ACM Press, pp. 169-178.

Cahn, J. 1999. A Computational Memory and Processing Model for Prosody. Doctoral dissertation, Media Laboratory, Massachusetts Institute of Technology.

Casey, M. A. 1998. Auditory Group Theory with Applications to Statistical Basis Methods for Structured Audio. Doctoral dissertation, Media Laboratory, Massachusetts Institute of Technology.

Fairbairn, W. R. D. 1963. Synopsis of an Object-Relations Theory of the Personality. International Journal of Psycho-Analysis 44:224-25.

Freud, S. 1965 [1933]. Lecture XXXI: The Dissection of the Psychical Personality. New Introductory Lectures on Psychoanalysis. J. Strachey, trans., ed. New York: Norton.

Gamma, E., Helm, R., Johnson, R., and Vlissides, J. 1977. Design Patterns: Elements of Reusable Object-Oriented Software. Reading, MA: Addison-Wesley.

Lasseter, J. 1995. Toy Story. Walt Disney Productions and Pixar Animation Studios.

Laurel, B. 1991. Computers as Theatre. Reading, MA: Addison-Wesley.

Laurel, B., Strickland, R., and Tow, R. 1994. Placeholder: Landscape and Narrative in Virtual Environments. Computer Graphics 28:2, 118-126.

Levi-Strauss, C. 1966. The Savage Mind. Chicago: Univ. of Chicago Press.

McGoldrick, M., Pearce, J. K., and Giordano, J., eds. 1982. Ethnicity and Family Therapy. New York: Guilford Press.

Minsky, M. 1986. The Society of Mind. New York: Simon and Schuster.

Mondykowski, S. M. 1982. Polish Families. In McGoldrick, M., and Pearce, J. K., and Giordano, J., eds., Ethnicity and Family Therapy. New York: Guilford Press.

Mulligan, R., dir. 1962. To Kill a Mockingbird. Universal International Pictures; Pakula-Mulligan, Brentwood Productions.

Norman, D. A. 1988. The Psychology of Everday Things. New York: Basic Books.

Oz, F., dir. 1986. Little Shop of Horrors. The Geffen Company.

Papert, S. 1980. Mindstorms: Children, Computers, and Powerful Ideas. New York: Basic Books.

Pinhanez, C. 1999. Representation and Recognition of Action in Interactive Spaces. Doctoral dissertation, Media Laboratory, Massachusetts Institute of Technology.

Pirsig, R. M. 1984 [1974]. Zen and the Art of Motorcycle Maintenance: An Inquiry into Values. New York: Bantam Books.

Resnick, M. 1998. Technologies for Lifelong Kindergarten. Educational Technology Research and Development, vol. 46, no. 4.

Strohecker, C. 1991. Why Knot? Ph.D. diss., Epistemology and Learning Group, Media Laboratory, Massachusetts Institute of Technology.

Strohecker, C., Brooks, K. M., and Friedlander, L. 1999. Tired of Giving In: An Experiment in Narrative Unfolding. Available as Technical Report 99-16. Cambridge MA: MERL - A Mitsubishi Electric Research Laboratory.

Sutherland, J. D. 1989. Fairbairn's Journey into the Interior. London: Free Association Books.

Turkle, S., and Papert, S. 1990. Epistemological Pluralism: Styles and Voices within the Computer Culture. Signs 16(1).

Ullmer, B., and Ishii, H. 1999. mediaBlocks: Tangible Interfaces for Online Media. Extended Abstracts of Conference on Human Factors in Computing Systems (CHI '99), ACM Press, pp. 31-32.

Wertsch, J. 1991. Voices of the Mind: A Sociocultural Approach to Mediated Action. Cambridge, MA: Harvard Univ. Press.

Wilson, A. D., and Bobick, A. F. 1999. Realtime Online Adaptive Gesture Recognition. Proceedings of the International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real Time Systems '99.

Winnicott, D. W. 1971a. Playing and Reality. London: Tavistock Press.

Winnicott, D. W. 1971b. Therapeutic Consultations in Child Psychiatry. London: Hogarth Press.

Wren, C. R. 1999. Understanding Expressive Action. Doctoral dissertation proposal, Media Laboratory, Massachusetts Institute of Technology.