05-830, User Interface Software, Spring, 2000
Lecture 15,   April 21, 2000
Copyright © 2000 - Brad Myers

UIMS Techniques:
Menu Trees, Transition Networks, Grammars; Event Languages,
HyperTalk & Production Systems;
Declarative Languages

---

Overview

• The term "User Interface Management System" associated with older systems that represent the dialog of the interface in a formal language.
• Thus, most of the following UIMSs define a special (textual or graphical) language for programming the UI.
• This is primarily only the input from the user to the computer, not the reverse.
  • Human-human dialog is symetric, but not computer-human dialogs
• Original forms were: menu trees, transition networks, grammars, and events.
• Issues: Formal range vs. effective range
  • What can conceivably be represented vs. what is convenient and understandable.

Menu Trees

• Also called "Dialog Trees"
• Hierarchy of menus
• picture, Olsen  UIMS book, p. 116  (Fig 7:1) (Dan R. Olsen, Jr., User Interface Management Systems: Models and Algorithms, Morgan Kaufmann, San Mateo, CA, 1992.)
• Used in Kasik's "TIGER" UIMS, which was used for years to create Boeing's CAD tools (1982)
  • Interface is "command-line style" with menus to choose the keywords
    • Prefix
    • Operator defines the parameters that will come
  • Very large number of commands, e.g. 10 ways to draw a circle
  • Special facilities for jumping around the tree, and doing the right sematic actions
  • Aborting commands

Transition Diagrams

• Set of states and set of arcs.
• Probably the earliest UIMS:
  • William Newman's "Reaction Handler" in 1968
• Simplest form, arcs are user input events.
  • arcs can be extended by listing feedback (output) and semantic actions on the arcs
  • actions usually written in a conventional language (e.g. C)
  • picture, Olsen, p. 37 (Fig 3:1)
• Often, represented textually:
  • picture, Olsen p. 38
• Sub-diagrams
  • To help modularize and simplify large networks
  • if call themselves, then "recursive transition network"
    • Picture Olsen, p. 41(Fig 3:4)
  • Problem: when to enter and leave the sub-dialog:
    • don't want to use up a token
• "Pervasive states" to handle help, abort, undo, etc.
  • "Escape" transitions to abort (permanently leave) a dialog
    • picture, Olsen p. 53 (Fig 3:11)
  • "Re-enter" sub-dialogs for temporary excursions that return to same place. E.g., help, use calculator, etc.
    • picture, Olsen p. 55 (Fig 3:12)
  • Transitions are taken if no specific arcs from node
• "Augmented transition networks"
  • local variables
  • function on arcs can determine transitions
  • "guards" determine whether transition is legal
  • "conditional transitions" calculate where to go
    • picture, Olsen p. 57 (Fig 3:14)
  • upgrades the power to context-free-grammar
• Transition Networks, in general, used in various UIMSs:
  • Research: Newman's "reaction handler", Jacob's RTN, Olsen's "Interactive Pushdown Automata", etc.
  • Commercial: IDE's RAPID/USE, Virtual Prototypes's VAPS
  • VAPS uses spreadsheet interface to the ATN
    • Pictures
• Evaluation:
• Good
  • Make explicit the interpretation of all events in each state
  • Emphasize the temporal sequence of user and system actions (unlike grammars)
  • Natural and easily understood if small
    • easy to teach, learn, and read
  • Appropriate for some parts of GUIs: widget behaviors, dialog box transitions, etc.
• Bad
  • Does not scale: 150 commands with 3 or 4 states each
    • explosion of lines and states for normal interfaces: "maze of wires"
  • unordered inputs
    • picture, Olsen p. 91 (Fig 6:1)
  • Textual form is like GOTO-based assembly language
  • Communication through global variables
  • Doesn't handle GUI mode-free style well
  • What to do with un-specified input? crash, ignore input
  • Doesn't address output

Grammars

• Context-free grammars good for describing textual dialogs which have a formal syntax.
  • alphabet of symbols
  • sentences - legal sequences of symbols
  • language - set of all legal sentences
  • grammar - set of terminals, set of non-terminals, set of productions, start symbol
  • context-free-grammar: all productions of the form
    <a> ::= B where <a> is a single nontermimal, and B is a non-empty string of terminals and/or non-terminals
• Example: syntax of Lisp floating point numbers:
  • picture, Lisp book, p 17
• Non-terminals, terminals. Listed in order.
• Iteration by recursion, unless support *, +
• Essentially the same power as STNs, certainly ATNs.
• Eliminates the need to specify a lot of states as in STN
• Attach semantic actions to the parsing of non-terminals
  • But not clear when processed: bottom-up or top-down
• Used in only a few UIMSs:
  • Olsen's "SYNGraph"
    • special features for Undo and Cancel (Rubout)
  • others using YACC+LEX
• Evaluation:
• Good
  • Handles complex dialogs
  • Handles both lexical and syntactic levels
• Bad
  • Unintuitive for developer
  • Hard to know where to attach semantic actions
  • Bad error recovery: can't push forward looking for ";"
  • Doesn't address output
  • Doesn't handle GUI mode-free style well

Event Languages

• Since get a stream of events from window manager, design a language where receiving events is the central paradigm.
• Central theme: interface designed in terms of input events and visual objects, not internal states (modes) and syntax
• Designed to handle multiple processes:
  • Multiple input devices at the same time
  • Multiple interaction techniques
• Event Languages have
  • Events: any action from the user or program
    • input events: leftdown
    • synthetic to control the program: modeDrawingLine
    • usually have parameters, such as (X,Y) of event
  • Event handlers: procedures to process events
    • have an event that starts them and code that is executed.
    • Code can compute, interface with the application, generate new events, change internal state, etc.
    • Computation is arbitrary, so event languages have full power.
• Sometimes called "production systems" since are similar to AI systems with "if -- then --" rules.
  • Distinction: Where events are qualified with the object performed over.
• Examples:
  • Cardelli's Squeak (C)
    • textual language for communicating with mice
  • Flecchia & Bergeron's ALGAE (Pascal) 1987
  • Hill's SASSAFRAS & ERL (lisp) 1986
  • HyperCard's HyperTalk
  • Martin Frank's EET
• Event Response Language (ERL), in "Sassafras"
  [ACM Trans. Graphics, July 86]
  • list of rules: condition -> action
  • conditions: name of an event + list of flags or just a list of flags.
  • action: flags to raise |   events to send !    assignments <-
  • example rules:
    1. MouseDown waitingForLine ->
   StartLine.X <- MouseDown.X
   StartLine.Y <- MouseDown.Y
   StartLine!
   lineDragging |
2. MouseDown waitingForCircle ->
   CircleCenter.X <- MouseDown.X
   CircleCenter.Y <- MouseDown.Y
   CircleCenter!
   circleDragging |
3. MouseUp circleDragging ->
   EnterCircle.X <- MouseUp.X
   EnterCircle.Y <- MouseUp.Y
   EnterCircle!
   waitingForCircle |
  • Note that CircleCenter! stores the center for EnterCenter
  • Another example (unordered inputs):
    Command getCmd -> cmd |
Argument getArg -> arg |
- - - cmd arg -> Process!
waitProcessing |
DoneProcessing waitProcessing ->
   getCmd |
   getArg |
  • Note that more than one rule may fire
• HyperTalk in HyperCard
  • Define scripts to be executed on events
  • Attempts to be "English-like" but is a real programming language
    on mouseUp
   global PreviousName
   put field "Name" into PreviousName
   PurgeName
end mouseUp
on PurgeName
   put "???" into field "Name"
end PurgeName
  • "real" and synthetic events like Sassafras
  • Always interpreted, so slow
  • Scripts can be associated with buttons, fields (text editing), cards, backgrounds, stacks, and global. Events given to the most specific first:
    
• Martin Frank's "Elements, Events & Transitions" (EET) Model
  • PhD, 1996, Georgia Tech
  • http://www.isi.edu/~frank/
  • Define scripts to be executed on events
  • Per object, rather than global, handlers (like HyperTalk)
  • "Transitions" happen on events:
    Transition PropertiesButton.pressed()
{
  PropertiesWindow.mapped := 1;
  PropertiesButton.enabled := false;
}
Transition PropertiesCancelButton.pressed()
{
  PropertiesWindow.mapped := 0;
  PropertiesButton.enabled := true;
}
  • Can be conditional on values of events:
    Transition (*.isSelectable=="true").pressed()
{
  self.color := "red";
}
    • this transition invoked whenever a "press" event occurs on an element which has an "isSelectable" attribute with value equal "true"
    • operates on all items of set that it applies to
  • "Implicit" events generated for parameter values changing, lists change size, etc.
    • for objects created, deleted or attributes changed
      Transition bb.changed(string attr == "selectedPins",                         int NewLength != 2)
{
  connectButton.enabled := false;
}
Transition bb.changed(string attr == "selectedPins",                          int NewLength == 2)
{
  connectButton.enabled := true;
}
    • button is enabled whenever the list in the bb element's selectedPins attribute is have length = 2
    • like "active variables"
    • can implement a form of constraints
  • Programmer can "throw" events explicitly
    Transition ((*.isAButton=="true") &&
            (*.status =="enabled")).released()
{
  throw self.invoke();
}
  • + Advantage over ERL: events on objects, like HyperTalk
  • - Difficult to figure out meaning of the transitions (when they will fire)
  • - Flow of control hard to program and read

Event Languages, in General

• Implementation: can be expensive if test every rule
  • Can hash off input events
  • Bit vector for flags so test if applicable is fast
• Evaluation
  • + Seems more natural for GUIs
  • + Can handle multi-threaded dialogs
  • + Greater descriptive power [Green86]
    • - because conditions match any values for the unnamed states
  • + More direct since deals with graphics and their events
  • - Difficult to write correct code since flow of control not localized
  • - Hard to specify syntax: need to use synthetic events or flags
  • - Need to learn a new programming language
  • - Hard to understand code when it gets large
    • Where is the handler for this message?
  • - Little (no?) separation UI code from application code
  • - Still communicating through global variables and semantics has to store temporary information
  • - Often no modularization: events and flags are global
• Combined approach: Jacob's event+ATN system
  [Robert J.K. Jacob, "A Specification Language for Direct Manipulation Interfaces," ACM Transactions on Graphics, Oct, 1986, vol. 5, no. 4, pp. 283-317.]
  • Interfaces are highly moded, in that mouse button will have different actions in different places, and depending on global state
    • but modes are highly visible and very quick
  • Conventional ATN for DM would be large, regular and uninformative
    • also, remembered state in each widget, e.g. partial file name, item awaiting confirmation, etc.
    • so "co-routines" are appropriate: suspended and resumed with remembered state
  • Define "interaction objects" that internally are programmed with state machines
  • "Events" are high level tokens, including location
  • All active interaction objects define event handlers and "executive" chooses the correct handler based on input event.
  • Evaluation: fundamentally, programmers generally prefer to program using textual languages.

Declarative Languages

• For dialog boxes, forms and menus, just list the contents
• But need to also list a lot of properties for each field
• Used by CMU's COUSIN system (~1985), and
• Apollo's Domain/Dialog (later HP/Apollo's Open Dialog) ~1985-87.
• Goals:
  • less overall effort
  • Better separation UI and application
  • Multiple interfaces for same application
  • Consistency since UI part uses same package
• UI and application interface through "variables"
  • Either can set or access
  • Typed by the kind of data transmitted: string, integer, enumerated
• "Domain Dialog" ("Open Dialog")
  • Call-back attached to each variable
• Motif's UIL is basically a modern-day example of this
  • - Can be edited by users (in theory) for customization
• Cousin (1985):
  • Cousin supports push buttons, cycle buttons, text, table, windows
  • Cousin sends to application a single event encoding the user's interaction, application must do dispatch
    • different from callbacks on widgets

[
FormName: "File System Manager"
Purpose: "For browsing and managing files"
]
[
Name: "Sort by"
ValueType: String
DefaultValue: Name
InteractionMode: CycleButton
ChangeResponse: Active
EnumeratedValues: (Name, "Last Change Date", Size)
Purpose: "How the files should be sorted"
Location: 10, 40
]
[
Name: Files
ValueType: String
DefaultSource: NoDefault
MinNumber: 1
MaxNumber: 1000
InteractionMode: Table
NumColumns: 4
ChangeResponse: Passive
Purpose: "List of files found matching the spec"
Location: 10, 40
Size: 400, 100
]

• Evaluation:
  • + Specify "what" not "how" = declarative
  • + Good for "coarse grain" interaction (forms)
  • + Programmer doesn't have to worry about time sequence of events
  • + Separate UI from application -- communicate through variables
  • - Fixed interactions
  • - Can't deal with non-widgets: need to use "graphic areas" and program by hand
  • - Can't dynamically change interface based on application
  • - Can't express dependencies: this widget is enabled when that widget has specific value.
  • - Interactive tools are easier to use for widgets and their properties
    • Motif's UIL or Microsoft Resource files designed to be written by such tools
• Next step: "Model-based" tools that take a specification of the contents and automatically decide on a layout, so fewer parameters have to be specified.

Back to 05-830 main page