05-830, User Interface Software, Spring, 1999
Lecture 13,   March 31, 1999
Copyright © 1999- Brad Myers

Amulet's Input Model:
Interactors and
Command Objects

code and pictures

Overview

• Try to provide more support so input handling isn't so difficult
• Make easy things simple and complex things possible
• Based on the "Model-View-Controller" architecture from Smalltalk
  • True separation of graphics (view) and input handling (controller)
  • Also uses idea from Foley&Wallace of identifying types of input handlers:
    • move
    • grow
    • rotate
    • text edit
    • gesture
    • select (pick)
• Innovations:
  • Identifying primitive "Interactor" objects and correct parameterizations so most direct manipulation UIs can be constructed by re-using built-in objects.
  • Only a few kinds of behaviors, and standard parameters
  • Real separation between input and output handling
  • Handles all input
    • insides of widgets
    • and for application programs
  • + First successful separation of View from Controller in Smalltalk MVC
  • + Integration of gestures with conventional interaction.
  • + Easier to code because substantial re-use
  • + Built-in support for multi-window dragging
• General idea: attach interactor objects to a set of graphical objects to handle their input.
  • Graphical objects don't handle input
    • No "event methods" in objects
  • Instead, define invisible "Interactor" objects and attach them to graphics
  • Interactors can operate on multiple objects
  • Strategy: pick the right type of Interactor, attach to the objects to be moved, fill in necessary slots of interactor
  • Widgets use interactors internally
  • Can have multiple interactors on an object (e.g., different mouse buttons)
  • Interactors directly set slots of objects,
    • constraints can be used to map those slots into behaviors:
  • Details of input events and event processing is hidden
• Interactor directly sets slots of objects using a standard protocol
• Used first in Garnet, refined in Amulet.

Types of Interactors

• Am_Choice_Interactor : select one or more of a set of objects
• Am_One_Shot_Interactor - single action, like Choice
• Am_Move_Grow_Interactor : move or grow objects with the mouse
• Am_New_Points_Interactor: to create new objects by entering points while getting feedback "rubber band" objects
• Am_Text_Edit_Interactor : mouse and keyboard edit of text
• Am_Gesture_Interactor: interpret freehand gestures

Parameters for all Interactors

• set of objects to operate on:
  • To be active, Interactor must be attached to an object which is (recursively) attached to the screen
  • Equivalent to visibility of graphical objects
  • Unlike graphical objects which can only be added as parts of windows or groups, interactors can be added as parts of any object:
    rect.Add_Part(my_inter);
  • Default: operates on the object attached to
  • But also common to operate on any member of a group.
  • Controlled by the Am_Start_Where_Test slot, which should contain a method.

SP_Ship_Mover =
   Am_Move_Grow_Interactor.Create("SP_Ship_Mover")
    .Set (Am_START_WHEN, "LEFT_DOWN")
    .Set (Am_START_WHERE_TEST, Am_Inter_In_Part);
ship_group.Add_Part(SP_Ship_Mover);

• Multiple groups
  • interactor can span multiple windows
• start, stop and abort events
  • - single key, mousebutton, "any" mousebutton, modifiers, (shift, meta...), double click, click vs. drag, etc.
• active?
• priority levels

Parameters for specific types of Interactors

• For buttons (Choice Interactors)
  • how many objects to select: set, toggle, list-toggle
• For move-grow:
  • interim feedback object (while the mouse moves)
    • if missing then object itself is modified

Am_Object feedback_circle = moving_circle.Create()
  .Set (Am_LINE_STYLE, Am_Dashed_Line)
  .Set (Am_VISIBLE, false);
my_win.Add_Part (feedback_circle);
// The definition of the interactor
Am_Object objs_grower = Am_Move_Grow_Interactor.Create ()
  .Set (Am_START_WHERE_TEST, Am_Inter_In_Part)
  .Set (Am_GROWING, true)   // grow instead of move
  .Set (Am_FEEDBACK_OBJECT, feedback_circle);
objs_group.Add_Part (objs_grower);

• gridding
• move or grow
• where-attach
  • center, n, ne, nw, w ... , where-hit
• flip if change sides
• minimum size
• For New_Point:
  • interim feedback object (while the mouse moves)
  • gridding
  • minimum size
  • abort if too small
• Text-interactor:
  • editing translation table (to map keystrokes and mouse into editing functions)
• Gesture-interactor:
  • gesture recognizer table
  • If missing, can use this to get freehand drawings

Simple Example

• To make an object movable with the mouse:

Am_Object rect = Am_Rectangle.Create()
  .Set(Am_LEFT, 40)
  .Set(Am_TOP, 50)
  .Set(Am_FILL_STYLE, Am_Red)
  .Add_Part(Am_Move_Grow_Interactor.Create());

Standard Behavior

Choice Interactor

• By default, sets the Am_INTERIM_SELECTED and Am_SELECTED slots of the affected objects
  • Constraints that depend on these slots

Am_Define_Style_Formula (line_from_selected) {
  if ((bool)self.Get(Am_INTERIM_SELECTED))
	return Am_Red;
  else if ((bool)self.Get(Am_SELECTED))
	return Am_Black;
  else return Am_Blue;
}

Am_Object my_prototype = Am_Line.Create()
	.Set(Am_LINE_STYLE, line_from_selected);
my_group = Am_Group.Create()
	.Add_Part(Am_Choice_Interactor.Create()
		.Set(Am_START_WHERE_TEST, Am_Inter_In_Part);

• Now add instances of my_prototype to my_group
• Also collects a list of the selected objects in the Am_VALUE slot of the command object in the interactor

Debugging: Tracing

• Can turn on tracing and get print out of which Interactors run and what they do
• Options: trace all, trace only setting of slots, trace a particular interactor, short trace (only which interactors run), etc.
• Inspector just toggles inspecting all or none
• Printout to console window

<><><><><> LEFT_DOWN x=180 y=289 time=3114329169

Enter GO for <grow_inter_in_handle_185>, state=0...
Checking start event against wanted = LEFT_DOWN * SUCCESS
Checking start where..  ~~SUCCESS=<Am_Rectangle_650>
Move_Grow starting over <Am_Rectangle_650> translated coordinates 169,268
Calculated attach point for non-line is Am_ATTACH_S
++Object <grow_inter_in_handle_185> setting Am_VISIBLE of <Sel_Rect_Feedback_197> to true
++Object <grow_inter_in_handle_185> setting obj=<Sel_Rect_Feedback_197> setting obj=<Sel_Rect_Feedback_197> LEFT=90 TOP=142 WIDTH=182 HEIGHT=148

<><><><><> LEFT_UP x=179 y=326 time=3114329838 drawonable=Amulet Test Selection Widget(0x4015b848)

Enter GO for <grow_inter_in_handle_185>, state=1...
Checking abort event against wanted = CONTROL_g * FAILED
Checking running where..  ~~SUCCESS=<window>
Checking stop event against wanted = ANY_MOUSE_UP * SUCCESS
Move_Grow stopping over <Am_Rectangle_650>
++Object <grow_inter_in_handle_185> setting Am_VISIBLE of <Sel_Rect_Feedback_197> to false
++Object <grow_inter_in_handle_185> setting obj=<Am_Rectangle_650> setting obj=<Am_Rectangle_650> LEFT=79 TOP=121 WIDTH=182 HEIGHT=185

Advanced Feature: Priorities

• If two interactors want to run, priorities used to determine which
• Am_PRIORITY slot contains a number. Default = 1
• When running, 100 added to it
• Inspector interactors use 300.0
• If multiple with same priority, runs the one attached closer to the leaf

Advanced Feature: Slots of Interactor

• In addition to value set into Command, a number of slots are set into the Interactor itself, which might be useful.
• Can get the interactor as the Owner of the command passed to the DO_METHOD
• Am_START_OBJECT, Am_START_CHAR, Am_FIRST_X, Am_FIRST_Y, Am_WINDOW, Am_CURRENT_OBJECT

Doing something with results: Command Objects

• Each Interactor and Widget has a Command object as a part in the Am_COMMAND slot
• Instead of executing a callback function, interactors and widgets create a "Command Object" and execute its "Do" method
• Slots of command object:
  • DO_METHOD
  • UNDO_METHOD
  • REDO_METHOD (undo the undo)
  • SELECTIVE_UNDO
  • SELECTIVE_REPEAT and SELECTIVE_REPEAT_On_NEW
  • HELP: for "balloon" or status line
  • LONG_HELP
  • ACTIVE (enabled) -- usually contains a constraint
  • VALUE, OLD_VALUE
  • etc.
• Like Command Objects in MacApp, but hierarchical
• Customize by overriding default methods

Innovations

• + Better reuse
  • Many commands are in the library and are usually used without change
  • Cut, Copy, Paste, Create-Object, Delete-object, To-Top, etc.
  • Usually can be used without change
  • Easier to customize when necessary
• + Better modularization
  • Hierarchical means each level only has to deal with its own actions
• + New form of selective undo, repeat
  • Includes undo/repeat of selections and scrolling
• + Single place for enabling, help, label, etc.
  • Can be computed by constraints
• + Makes scripting easier
  • End-User programming by demonstration
  • Parameterization possible without help from application
• + May help with CSCW
  • Executing the command on multiple machines

Operation

• Interactors do 3 things
  • modify objects directly
  • set Am_VALUE slot of their command
  • call the command's Am_DO_METHOD

• Can just have a constraint from an object or widget to the Am_VALUE of the widget
• Method to override for application-specific operations
• Fill in the UNDO_METHOD to support undo

Commands in Widgets

• Various kinds of button panels:
  • Take list of items in the Am_ITEMS slot as an Am_Value_List
  • Contents of list can be strings, graphical objects, or instances of Am_Command
  • If commands, then the Am_LABEL field is used, and the DO_METHOD of that command is called
  • If not commands, then DO_METHOD of the top-level widget is called
  • For menu_bar, Am_ITEMS slot of each top-level command contains Am_Value_List of the sub-commands

Values

• Value of Command in Choice Interactor set to object selected
• Value of Command in button-type widgets set to Label of command, or Am_ID field of the command
• Remember, label can be a string or a graphical object

Handling Undo

• Add an undo-handler to the window
• Each command will be registered with the undo handler
• Built-in types of undo handlers
  • One undo then redo, like Mac
  • Infinite undo, one redo
  • Selective undo mechanism
• Each DO method saves necessary information for undo in command object
• Implementing selective undo not much harder than regular undo:
  • Allocates a new command object and adds to top to history list
  • Semantics is based on what the user would want
  • Undo the operation in a new context means to set the object back to its previous value
  • Enabled if object is still available
  • Undo of create is delete
• Redo the operation means to set the value of the object again; create a new object
• Also, redo on new object

Scripting

• New feature (CHI'98)
• Select set of commands and specify that in a program
• Can parameterize actions
• Moving selection handles
• • Forwards, backwards, left, right, up, down, in, out
  • Search for object of a particular type or value
• Little or no change to application if it supports Selective Repeat

Back to 05-830 main page