From Theory to Practice
Scheduling and Graphical User Interfaces
Computer Music Systems and Information Processing

Roger B. Dannenberg
Jan 2018

Introduction

We've seen a lot of concepts and pseudo-code:

events
timestamps
schedulers
logical time systems
threads

All of these of course must be implemented in order to use them (e.g. for Project 2).

We're going to look at implementations and libraries as follows:

Scheduling in serpent64
Graphical User Interfaces
Scheduling in wxserpent64
Forward synchronous scheduling with MIDI messages

A scheduler is implemented in lib/sched.srp, and this can be used with both serpent64 and wxserpent64, but the details are a little different.

Recently, Serpent has been extended with co-routine-like threads. All of the scheduling examples in this document have been implemented using Serpent threads, and these examples are described on this page on using threads.

Scheduling in serpent64

Simple Example

(Code available here.) Let's make a simple program to print a message every second.

require "debug"
require "sched"

def activity(i):
    display "activity", i, sched_rtime, time_get()
    sched_cause(1, nil, 'activity', i + 1)

sched_init()
sched_cause(1, nil, 'activity', 101)
sched_run()

Notes:

activity plays the role of a process or thread -- its action stretches across time.
The last line of activity schedules another call to itself -- this is sometimes called temporal recursion.
When a function or method is invoked by a scheduled event, sched_cause uses sched_current, which is set to the scheduler that invoked the function or method.
Here, sched_current will always be rtsched, the real-time scheduler.
The parameters to the sched_cause method are:

time delay in seconds, measured from the ideal "now"
the object containing the method to be called, or nil to call a function
the function name, which must be a symbol (usually a literal denoted by single quotes -- not double quotes which would denote a literal string rather than a symbol)
any number of parameters (only positional parameters are allowed, but the method or function to be called can have optional parameters.)

sched_init must be called to initialize the scheduler
We "prime the pump" by scheduling the initial call to activity -- otherwise, activity would never run.
sched_run invokes the "scheduling executive" -- a fancy name for an infinite loop that runs events according to schedule.
sched_run does not return unless sched_stop() is called (or the global sched_running is set to false).
Source code for sched.srp is in lib.

Here's what some output looks like:

activity: i = 101, sched_rtime = 1, time_get() = 1
activity: i = 102, sched_rtime = 2, time_get() = 2.002
activity: i = 103, sched_rtime = 3, time_get() = 3.001
activity: i = 104, sched_rtime = 4, time_get() = 4.001
activity: i = 105, sched_rtime = 5, time_get() = 5
activity: i = 106, sched_rtime = 6, time_get() = 6
activity: i = 107, sched_rtime = 7, time_get() = 7.001
activity: i = 108, sched_rtime = 8, time_get() = 8
activity: i = 109, sched_rtime = 9, time_get() = 9.001
activity: i = 110, sched_rtime = 10, time_get() = 10.001
^C

Notice that the ideal event times, accessed as sched_rtime are precisely 1 second apart, as if the computer is infinitely fast and time is not quantized. The actual time, accessed as time_get() is close to the ideal time, but sometimes a millisecond or two late.

Two Activities

(Code available here.) Scheduling is most useful when there are many "threads" whose events are interleaved in time.

We use temporal recursion, i.e. functions (re)scheduling themselves instead of true threads or processes.

The next example shows two activities. activity_1 runs every second; activity_2 runs every 3 seconds.

require "debug"
require "sched"


def activity_1(i):
    display "act1    ", i, sched_rtime, time_get()
    sched_cause(1, nil, 'activity_1', i + 1)


def activity_2(i):
    display "    act2", i, sched_rtime, time_get()
    sched_cause(3, nil, 'activity_2', i + 1)


sched_init()
sched_cause(1, nil, 'activity_1', 101)
sched_cause(1, nil, 'activity_2', 201)
sched_run()

Here is some output generated by the program. Notice that when 2 events are scheduled for the same time, they run sequentially, but it is not obvious which will run first. (In this case, ordering is deterministic, but order depends on the implementation of the priority queue in the scheduler.)

Notice also that activity 1 ("act1") runs every second, while activity 2 ("act2") runs every 3 seconds.

    act2: i = 201, the_sched.time = 1, time_get() = 1.001
act1    : i = 101, sched_rtime = 1, time_get() = 1.001
act1    : i = 102, sched_rtime = 2, time_get() = 2
act1    : i = 103, sched_rtime = 3, time_get() = 3.001
act1    : i = 104, sched_rtime = 4, time_get() = 4.001
    act2: i = 202, sched_rtime = 4, time_get() = 4.002
act1    : i = 105, sched_rtime = 5, time_get() = 5
act1    : i = 106, sched_rtime = 6, time_get() = 6
act1    : i = 107, sched_rtime = 7, time_get() = 7
    act2: i = 203, sched_rtime = 7, time_get() = 7
act1    : i = 108, sched_rtime = 8, time_get() = 8.001
act1    : i = 109, sched_rtime = 9, time_get() = 9
act1    : i = 110, sched_rtime = 10, time_get() = 10
    act2: i = 204, sched_rtime = 10, time_get() = 10
act1    : i = 111, sched_rtime = 11, time_get() = 11.001
act1    : i = 112, sched_rtime = 12, time_get() = 12.002
^C

Multiple ”Threads“ Using One Function

(Code available here.) Let's go back to the first example and initialize the schedule with two calls to activity.

require "debug"
require "sched"


def activity(i):
    display "activity", i, sched_rtime, time_get()
    sched_cause(1, nil, 'activity', i + 1)


sched_init()
sched_cause(1, nil, 'activity', 101)
sched_cause(1.1, nil, 'activity', 201) 
sched_run()

Here is some output generated by the program. Notice that there are two independent "threads" or sequences of calls to activity, even though there in only one function. The events are separated by 1 second, with one set of events starting at 1 and the other starting at 1.1.

activity: i = 101, sched_rtime = 1, time_get() = 1.002
activity: i = 201, sched_rtime = 1.1, time_get() = 1.101
activity: i = 102, sched_rtime = 2, time_get() = 2.001
activity: i = 202, sched_rtime = 2.1, time_get() = 2.102
activity: i = 103, sched_rtime = 3, time_get() = 3
activity: i = 203, sched_rtime = 3.1, time_get() = 3.1
activity: i = 104, sched_rtime = 4, time_get() = 4
activity: i = 204, sched_rtime = 4.1, time_get() = 4.1
activity: i = 105, sched_rtime = 5, time_get() = 5.001
activity: i = 205, sched_rtime = 5.1, time_get() = 5.101
^C

The Stopping a “Thread” Pattern

(Code available here.) How can we stop a function that keeps rescheduling itself? The simplest way is to set a flag that causes the function to return immediately. This is not a good solution. To restart the sequence of calls, you have to unset the flag. If you unset before a scheduled event wakes up and notices the flag, then the flag will have no effect, and you end up with two "threads" as in the previous example.

A good solution uses a counter rather than a flag, and every sequence of calls (every "thread") has an identifier that must match the counter.

require "debug"
require "sched"

activity_id = 0 // used to kill "threads"
 
def activity(id, i):
    if id != activity_id:
        display "activity terminates", id, sched_rtime, time_get()
        return
    display "activity", id, i, sched_rtime, time_get()
    sched_cause(1, nil, 'activity', id, i + 1)

def activity_start(i):
    // kill any old running activity 
    activity_id = activity_id + 1
    activity(activity_id, i) // start a new activity "thread"

sched_init()
sched_cause(1, nil, 'activity_start', 101)
sched_cause(4.5, nil, 'activity_start', 201)
sched_run()

Here is some output generated by the program. Notice that the first "thread" is effectivly killed at time 4.5 when activity_id is incremented, but the thread does not "know" this until it wakes up at time 5, "sees" activity_id has changed, and terminates (returns without performing any action or scheduling any future calls).

activity: id = 1, i = 101, sched_rtime = 1, time_get() = 1.001
activity: id = 1, i = 102, sched_rtime = 2, time_get() = 2.002
activity: id = 1, i = 103, sched_rtime = 3, time_get() = 3.001
activity: id = 1, i = 104, sched_rtime = 4, time_get() = 4.001
activity: id = 2, i = 201, sched_rtime = 4.5, time_get() = 4.5
activity terminates: id = 1, sched_rtime = 5, time_get() = 5.001
activity: id = 2, i = 202, sched_rtime = 5.5, time_get() = 5.501
activity: id = 2, i = 203, sched_rtime = 6.5, time_get() = 6.501
activity: id = 2, i = 204, sched_rtime = 7.5, time_get() = 7.501
^C

Spawning New Activity

(Code available here.) A "thread" can be spawned at any time. Here's a "thread" that runs every second. Each second, it spawns another thread that runs every 0.1 seconds, but only 3 times.

require "debug"
require "sched"

def activity(i):
    display "activity", i, sched_rtime, time_get()
    sched_cause(0.1, nil, 'subactivity', 0)
    sched_cause(1, nil, 'activity', i + 1)

def subactivity(j):
    display "     sub", j, sched_rtime, time_get()
    if j < 2:
        sched_cause(0.1, nil, 'subactivity', j + 1)

Here is some output generated by the program.

activity: i = 101, sched_rtime = 1, time_get() = 1.001
     sub: j = 0, sched_rtime = 1.1, time_get() = 1.102
     sub: j = 1, sched_rtime = 1.2, time_get() = 1.202
     sub: j = 2, sched_rtime = 1.3, time_get() = 1.3
activity: i = 102, sched_rtime = 2, time_get() = 2.002
     sub: j = 0, sched_rtime = 2.1, time_get() = 2.101
     sub: j = 1, sched_rtime = 2.2, time_get() = 2.2
     sub: j = 2, sched_rtime = 2.3, time_get() = 2.3
activity: i = 103, sched_rtime = 3, time_get() = 3.001
     sub: j = 0, sched_rtime = 3.1, time_get() = 3.101
     sub: j = 1, sched_rtime = 3.2, time_get() = 3.201
     sub: j = 2, sched_rtime = 3.3, time_get() = 3.302
^C

Virtual Time, Fixed Tempo

(Code available here.) Let's run activity under a virtual time scheduler, at 2x speed, so the events are every 0.5 seconds instead of every 1 second. We use the already-created vtsched, an instance of Vscheduler.

require "debug"
require "sched"

def activity(i):
    display "activity", i, sched_rtime, time_get(), sched_vtime
    sched_cause(1, nil, 'activity', i + 1)

sched_init()
sched_select(vtsched)
sched_set_bps(2) // 2 beats per second
// sched_trace = t
sched_cause(1, nil, 'activity', 101)
sched_run()

Here's the output. Notice the_sched (which is vtsched) time is not even close to real time because this is virtual time (or beats, if you prefer).

activity: i = 101, sched_rtime = 0.5, time_get() = 0.5, sched_vtime = 1
activity: i = 102, sched_rtime = 1, time_get() = 1.002, sched_vtime = 2
activity: i = 103, sched_rtime = 1.5, time_get() = 1.502, sched_vtime = 3
activity: i = 104, sched_rtime = 2, time_get() = 2.001, sched_vtime = 4
activity: i = 105, sched_rtime = 2.5, time_get() = 2.502, sched_vtime = 5
^C

Virtual Time, Variable Tempo

(Code available here.) The speed or tempo can be changed. Here, we set the tempo to 0.5 beats per second on beat 4 in the next-to-last line of code. You could also call set_bps() within activity, or elsewhere for that matter.

require "debug"
require "sched"


def activity(i):
    display "activity", i, sched_rtime, time_get(), sched_vtime
    sched_cause(1, nil, 'activity', i + 1)


sched_init()
sched_select(vtsched)
sched_set_bps(2) // 2 beats per second
// equivalent to: sched_set_period(0.5)  
sched_cause(1, nil, 'activity', 101)
sched_cause(4, nil, 'sched_set_bps', 0.5)
sched_run()

Here's the output. Notice the real times (time_get()) are every 0.5s until beat 4, then every 2s.

activity: i = 101, sched_rtime = 1, time_get() = 0.5
activity: i = 101, sched_rtime = 0.5, time_get() = 0.5, sched_vtime = 1
activity: i = 102, sched_rtime = 1, time_get() = 1.002, sched_vtime = 2
activity: i = 103, sched_rtime = 1.5, time_get() = 1.501, sched_vtime = 3
activity: i = 104, sched_rtime = 2, time_get() = 2, sched_vtime = 4
activity: i = 105, sched_rtime = 4, time_get() = 4.002, sched_vtime = 5
activity: i = 106, sched_rtime = 6, time_get() = 6.002, sched_vtime = 6
^C

Graphical User Interfaces

Basics

wxserpent automatically creates default_window (and a text output window if there is any text output).

Graphical objects -- buttons, windows, menus, canvases to paint on -- are all represented by Serpent objects.

Simple Example -- Button

(Code available here.) Create a button that prints "hello world" when pressed.

require "debug"
require "wxserpent"


button = Button(0, "Hello", 5, 5, 75, 20)
button.method = 'button_pressed'


def button_pressed(obj, event, x, y):
    print "Hello World

Notes:

require "wxserpent" to get graphics library definitions
Button is a class
To make an instance of a class, you "call" it like a function.
Parameters:

0 is the parent (window). The proper name is default_window, but you can use 0 as a short name.
"Hello" is the button label
5, 5, 75, 20 are the x, y, width, and height of the button. Coordinates are pixels relative to the upper left corner of the parent window.

Set the method field of the Button instance (stored in the variable button) to establish a handler for button events.
Handlers take 4 parameters:

obj is the object (the Button) that received the press
event is an integer code for the event, e.g. WXS_BUTTON_CLICKED is the event for a button press
x and y are sometimes used, but not for button clicked events.

Menus

(Code available here.) Menus are objects; menu items are not.

require "debug"
require "wxserpent"


file_menu = default_window.get_menu("File")
file_menu.item("Hello", "Print Hello World", false, 
               nil, 'file_menu_handler')


def file_menu_handler(obj, event, x, y):
    display "file_menu_handler", obj, event, x, y
    print "Hello World"

Notes:

Each window has a different set of menus.
The get_menu() method either retrieves an existing menu or creates a new one if it does not already exist.
The item() method makes a menu item and optionally makes it “checkable” and also allows you to specify an object (or nil) and method.
Handlers for menus are similar to those for buttons. The x parameter is the index of the selected menu item (the same value returned by the item method). The y parameter tells you if the menu item is checked (only if the menu item is checkable).

Labeled Sliders

(Code available here.) The Slider class makes a simple linear slider. A more "application ready" slider is implemented by the Labeled_slider class, which displays a label, the text value corresponding to the slider, uses floating point numbers, and can give you a linear or dB scale.

require "debug"
require "wxserpent"
require "slider"


slider = Labeled_slider(0, "Volume", 5, 5, 250, 20, 70, 0, 1, 0.5, 'linear')
slider.method = 'slider_handler'


def slider_handler(obj, x):
    display "slider_handler", obj, x
    print "Slider changed to:", x

Notes:

require "slider" loads the definition of Labeled_slider (not defined in wxserpent)
Parameters are:

0 is parent window
"Volume" is the label
5, 5, 250, 20 is the x, y, width, and height relative to the parent
70 is the width of the label
0, 1, 05 give the minimum, maximum, and initial values of the slider
'linear' selects the mapping from slider position to floating point values. Other options are 'exponential' and 'db' (which ignores minimum and maximum values)

Scheduling in wxserpent64

Simple Example

(Code available here.) This example is equivalent to our first example, only adapted to wxserpent64.

require "debug"
require "sched"

def activity(i):
    display "activity", i, sched_rtime, time_get()
    sched_cause(1, nil, 'activity', i + 1)

sched_init()
sched_cause(1, nil, 'activity', 101)

Notes:

Notice there is no call to sched_run(). Since sched_run() enters an infinite polling loop, calling it would “steal” the main thread and prevent any polling for and handling of graphical user interface events.
Use ^C on OS X or Alt-X on Windows to quit.

Play Some MIDI Example

(Code available here.) In this example, we open a MIDI output device change activity to play a MIDI note.

require "debug"
require "wxserpent"
require "sched"
require "midi-io"
require "prefs"
require "mididevice"

def activity(p):
    display "activity", p, sched_rtime, time_get()
    midi_out.note_on(0, 60 + p % 12, 0)
    time_sleep(0.1)
    p = p + 1
    midi_out.note_on(0, 60 + p % 12, 100)
    sched_cause(0.2, nil, 'activity', p)

sched_init()
prefs = Prefs("./prefs.txt")
midi_devices = Midi_devices(prefs, open_later = true)
success = midi_devices.open_midi(latency = 10, device = 'midi_out_device')
if not success
    print "PLEASE SELECT A VALID OUTPUT DEVICE AND RESTART THIS PROGRAM"
else
    sched_cause(1, nil, 'activity', 0)

Notes:

midi-io defines classes for midi input and output. See serpent/doc/serpent.htm, serpent/programs/midi-io.srp and serpent/doc/serpent-by-example.htm for more documentation and examples.
prefs is a preferences library used by mididevice to store your last-selected midi device(s).
mididevice is a library that adds midi device selection to midi-io. It only works with wxserpent64.
The object constructor Midi_devices takes a prefs parameter which is a Prefs object, and the parameter to Prefs is a filename where preferences will be stored.
The open_later keyword parameter to Midi_devices defers opening devices until later; otherwise, Midi_devices will immediately open midi devices based on previously set preferences (if any) or open default devices. By “opening later”, we can specifically open only input or only output.
open_midi() is used to open midi output. The latency keyword parameter, if non-zero, indicates that MIDI should be precisely timed (if possible) by the device driver using the logical computation time as calculated by rtsched. (If you use vtsched or some other scheduler, rtsched is still active and will have the appropriate time value from which MIDI event timestamps are derived.) Since there will always be computational delays, the timestamp used for MIDI events will be the logical time from Serpent's rtsched plus the latency value passed to open_midi.
One annoying source of latency is wxWidgets, the GUI framework used by wxserpent64, which, under OS X, adds some cute animations to some controls. The animations are synchronous, meaning that there are no timer callbacks and thus no scheduled processing for the 250 ms or so taken by the animation. Thus, you may experience some “outages” if you use the GUI while playing carefully scheduled MIDI events. Setting latency to a higher value (e.g. 250 ms) will effectively pre-compute MIDI data, push it to a higher priority process (the Core MIDI subsystem of OS X), and apply another level of scheduling that is more precise.

Slider Volume Control Example

(Code available here.) Let's add some controls. This example uses the “stopping a thread” pattern and adds a Stop button. There are also sliders to control the period of time between notes and the MIDI velocity. We could use the first parameter of sched_cause() to control the period between notes, in which case the period change would take effect on the next note. Instead, we think of “period between notes” as a continuous indicator of tempo that can change at any time. We use sched_set_period() to change the tempo immediately when the slider is moved.

wxserpent window with midi period and
volume controls

require "debug"
require "wxserpent"
require "sched"
require "midi-io"
require "prefs"
require "mididevice"
require "slider"

player_id = 0 // use "stopping a thread" pattern

def activity(id, p):
    display "activity", p, sched_rtime, time_get()
    midi_out.note_on(0, 60 + p % 12, 0)
    if id != player_id: // stop AFTER note-off
        return
    p = p + 1
    var vel = int(velocity.value())
    midi_out.note_on(0, 60 + p % 12, vel)
    sched_cause(1, nil, 'activity', id, p)

def stop(rest ignore)
    player_id = player_id + 1

Button(0, "Stop", 5, 5, 50, 20).method = 'stop'

def set_period(obj, x):
    display "set_period", obj, x
    sched_select(vtsched)
    sched_set_period(x)

period = Labeled_slider(0, "Period", 5, 30, 250, 20,
                        70, 0.05, 5, 0.2, 'exponential')
period.method = 'set_period'

velocity = Labeled_slider(0, "Velocity", 5, 55, 250, 20,
                          70, 1, 127, 100, 'linear')

sched_init() // creates vtsched and rtsched
prefs = Prefs("./prefs.txt")
midi_devices = Midi_devices(prefs, open_later = true) 
success = midi_devices.open_midi(latency = 10, device = 'midi_out_device') 
if not success 
    print "PLEASE SELECT A VALID OUTPUT DEVICE AND RESTART THIS PROGRAM"
else 
    sched_select(vtsched)
    sched_set_period(0.2)
    sched_cause(real_delay(5), nil, 'activity', player_id, 0)

Notes:

The velocity slider has no handler; we simply read the slider value with value() method when we send a MIDI note-on.
The vtsched period is set initially to 0.2 to match the initial value of 0.2 given to the period slider. (Third line from the end)
Button() returns the new instance, so we immediately set its method and do not assign the instance to any variable. Controls like Button are stored in a table automatically, so the garbage collector does not delete them.
The last line uses real_delay(5) to compute the time delay parameter for sched_cause. Normally, you would just use a number, e.g. 5, and the delay would be 5 “beats”, where the “beat” is determined by the virtual scheduler’s beate-per-second setting, which is set using one of sched_set_bps(), sched_set_bpm() or, as in this case, sched_set_period(). However, this is a logical delay. If the scheduler is behind, e.g. if it took 2 seconds to initialize MIDI (unlikely, but sometimes systems take awhile to configure MIDI), then you could be logically behind schedule and end up spewing out some notes quickly to catch up. To make sure the program gets a clean start after initialization, the real_delay() function can be used to compute a delay relative to the current real time. Thus, there will be always be a real delay (hence the name). In this case, since the scheduler period is 0.2 s, and we ask for a delay of 5 of these periods, the actual delay will be 5 * 0.2 = 1 second of real time.

Forward synchronous scheduling

Timestamps in PortMidi

The previous examples do some behind-the-scenes work in schedulers and in midi-io to pass logical times as timestamps to PortMidi, the MIDI interface library used by Serpent. This section aims to reveal how this works.

PortMidi supports forward synchronous scheduling. In brief, everything is timestamped. The goal is to compute everything slightly ahead of real time, pass timestamped messages to the device driver, and let the device driver send the messages according to the timestamp to obtain very precise timing.

Forward synchronous scheduling is supported by the midi-io library which simplifies the process. You should use it to get better timing, especially when using wxserpent (because graphical redisplays can be slow, and the timer callback will not be issued while redrawing the display), but only if you can tolerate some latency.

Make an Example Without Timestamps

(Code available here.) To explore forward synchronous scheduling, let's first make a counter-example that has noticeable timing problems. Our simple examples on modern machines are so fast you might not notice any problems, even without careful timestamping, so we'll make an artificial situation where, at the beginning of our function to play MIDI, we delay a random amount whose distribution is controlled by a slider.

require "debug"
require "wxserpent"
require "sched"
require "midi-io"
require "prefs"
require "mididevice"
require "slider"


player_id = 0 // use "stopping a thread" pattern


def activity(id, p):
    time_sleep(random() * jitter.value())
    midi_out.note_on(0, 60 + p % 24, 0)
    if id != player_id: // stop AFTER note-off
        return
    p = p + 1
    midi_out.note_on(0, 60 + p % 24, 100)
    sched_cause(0.08, nil, 'activity', id, p)


def stop(rest ignore)
    player_id = player_id + 1

Button(0, "Stop", 5, 5, 50, 20).method = 'stop'


jitter = Labeled_slider(0, "Jitter", 5, 55, 250, 20,
                          70, 0, 0.05, 0, 'linear')


sched_init() // creates vtsched and rtsched
prefs = Prefs("./prefs.txt")
midi_devices = Midi_devices(prefs, open_later = true)
success = midi_devices.open_midi(latency = 0, device = 'midi_out_device')
if not success
    print "PLEASE SELECT A VALID OUTPUT DEVICE AND RESTART THIS PROGRAM"
else
    sched_cause(1, nil, 'activity', player_id, 0)

Notice that in the call to open_midi the latency is explicitly set to zero, which means timestamps are not used by PortMidi, and messages are delivered immediately. This still works pretty well, but if you crank the jitter slider up to 25 ms, you should notice timing irregularities.

Using Forward Synchronous Scheduling

To get forward synchronous behavior in the previous program, just change latency = 0 to latency = 25.

Now, if you crank the jitter slider up to 25ms, the program will run with same timing irregularities as before, but the MIDI output timing will be close to perfect!

Let's look under the hood and see how this happens.

First, midi-io's start() method passes latency (25) to PortMidi:

    def start(keyword latency = 0, keyword midi_out_dev)
        if port != nil
            return
        var dev = midi_out_dev
        if not dev:
            dev = midi_out_default()
        if dev != -1:
            port = midi_create()
            if midi_open_output(port, midi_dev, 100, latency) != 0:
                return false
            else:
                 print "Opened MIDI out device successfully:", dev
                 print_midi_device_info(dev)
        else:
            print "No MIDI output device opened";
            if midi_out_dev == -1:
                print " by request or preference setting."
            else:
                print ". There appears to be no output device."
            return false
        return t

A non-zero latency to midi_open_output says that all output timestamps should be incremented by latency and passed on to the device driver.
As long as the MIDI message arrives at the device driver before the timestamp, the device driver will be able to output the message with high timing precision.
Example:

An event is scheduled for time = 10000 ms.
Either the entire wxserpent process or the timer_callback is delayed by 20ms,
so the event runs at time = 1002 0ms.
The timestamp, based on ideal time, is 10000 ms,
and this is incremented by latency in PortMidi to get 10025 ms
The device driver receives the message immediately, at time = 10020 ms.
The message is held for the remaining 5 ms and output at time = 10025 ms.

Where do output timestamps come from?

Must be based on ideal time.
Scheduler knows the ideal time.

Here's a simplified version of midi_output implemented in midi-io.srp:

    def note_on(chan, key, loud):
        var ticks = rtsched.get_tick()
        if midi_trace
            display "midi note_on", ticks, chan, key, loud
        chan = chan & 15
        midi_write(port, ticks, chr(midi_status_noteon + chan) + 
                                chr(key) + chr(loud))
        if loud > 0:
            midi_note_on_count = midi_note_on_count + 1
        else:
            midi_note_off_count = midi_note_off_count + 1

As you can see, the timestamp is derived from the scheduler's ideal event time, which is stored as the time instance variable in the scheduler (marked below in blue):

    # return an idealized integer count of milliseconds corresponding
    # to the timestamp used by PortMidi. Note that the timestamp used
    # by PortMidi is equal to int(time_get() * 1000)
    def get_tick()
        // note: if you are not running the scheduler, time_offset is nil
        // and this will raise an (intentional) error. It means you are
        // trying to schedule MIDI data before starting the scheduler.
        // Also, if you are not running a scheduled event, you must call
        // start_use() before get_tick(). Otherwise, the time field
        // will be the ideal time of the last event that ran, so it
        // may be a time in the distant past.
        if sched_nesting == 0
            print "ERROR: Scheduler::get_tick() called, but not in an event"
        int((time + time_offset) * 1000)

Notice the test for event_nesting. This tells us if we are computing an event dispatched by the scheduler. If not, then time is not current and probably stores the ideal time of the last event. For this reason, you should always send MIDI from a scheduled event. In particular, if you want to send MIDI when you push a button on the user interface, you should call sched_select(), which will not only select a scheduler but update the logical time to the current real time, then use sched_cause() to schedule some MIDI output. (You can set the logical delay to 0 to send the MIDI immediately.)
Notice also time_offset.

The code sets rtsched.time_offset = time_get() (third line from the end), and when time is up-to-date, it is equal to time_get() - time_offset. Thus, scheduler time = 0 can correspond to about when the scheduler starts running.
But PortMidi timestamps are based on the same system time as time_get(), without any offset.
Thus, we have to add time_offset to translate scheduler time to MIDI timestamp time.

Animation Example

(Code available here.) In this example, we revisit “Scheduling in wxserpent64,” adding some simple commands to draw shapes. This program uses the scheduler to schedule a refresh() operation that redraws a Canvas object at 30 frames per second.

require "debug"
require "wxserpent"
require "sched"


def activity(i):
    display "activity", i, sched_rtime, time_get()
    // 30 frames per second
    sched_rcause(1/30, nil, 'activity', i + 1)
    if random() < 0.03 // 3% of the time move rectangle
        animate.change_square()
    animate.refresh(t) // t means true, redraw everything


def timer_callback()
    rtsched.poll(time_get())


class Animate (Canvas)
    var ex, ey  // location of ellipse
    var sx, sy  // location of square
    var sr, sg, sb // square color (red, green, blue)

    def init(parent, x, y, w, h)
        super.init(parent, x, y, w, h) // calls Canvas's init()
        ex = 20 // initialize this subclass of Canvas
        ey = 20
        sx = 100
        sy = 100
        sr = 200
        sg = 200
        sb = 200

    def paint(x) // x == true means redraw everything
        // this code always redraws everything
        //display "Animate::paint", x
        set_brush_color("GRAY")
        draw_ellipse(ex, ey, 50, 30)
        ex = ex + 3
        if ex > get_width() - 50
            ex = 20
        set_brush_rgb(sr, sg, sb)
        draw_rectangle(sx, sy, 40, 40)

    def change_square()
        sx = random() * animate.get_width()
        sy = random() * animate.get_height()
        sr = int(random() * 256)
        sg = int(random() * 256)
        sb = int(random() * 256)


def main()
    animate = Animate(WXS_DEFAULT_WINDOW, 0, 0, 300, 300)
    sched_init()
    rtsched.cause(1, nil, 'activity', 101)
    rtsched.time_offset = time_get()
    wxs_timer_start(2, 'timer_callback')


main()

Notes:

You must subclass Canvas so that you can make a custom paint method, overriding the default paint() method in Canvas.
The init() method is the “constructor” that initializes a new object. Inside init(), you often call super.init() to invoke the constructor of the superclass. (The superclass's constructor does not run by default if you override init() as in this example.)
To make an object, you “call” the class as if it is a function, e.g. Animate(WXS_DEFAULT_WINDOW, 0, 0, 300, 300). This operation creates a new object, invokes the init() method, and returns the object.
To draw on the canvas, you call refresh(). In this example, the canvas subclass is Animate, the instance of this subclass is animate, and we call animate.refresh(t). (The parameter is “true” and directs the canvas to fill with the background color in preparation for a full redraw.)
After invoking refresh(), as soon as the program returns control to wxWidgets, wxWidgets prepares the graphics system to draw on the canvas and calls the canvas's paint() method.
Be very careful with colors: Note that the random RGB values are converted to integers using int(). Real numbers will not work for colors. Coordinates, however, are more forgiving.

From Theory to Practice Scheduling and Graphical User Interfaces Computer Music Systems and Information Processing

Introduction

Scheduling in serpent64

Simple Example

Two Activities

Multiple ”Threads“ Using One Function

The Stopping a “Thread” Pattern

Spawning New Activity

Virtual Time, Fixed Tempo

Virtual Time, Variable Tempo

Graphical User Interfaces

Basics

Simple Example -- Button

Menus

Labeled Sliders

Scheduling in wxserpent64

Simple Example

Play Some MIDI Example

Slider Volume Control Example

Forward synchronous scheduling

Timestamps in PortMidi

Make an Example Without Timestamps

Using Forward Synchronous Scheduling

Animation Example

From Theory to Practice
Scheduling and Graphical User Interfaces
Computer Music Systems and Information Processing