Hypotheses Lists from Tcl

WARNING: valid only for hypos.c version 1.12 or newer
WARNING: requires itf.c version 1.28 or newer

General Infos:

There are a couple of objects of the type HypoList as integrated part of each search object, but you can also generate them yourself. A single hyposesis of type Hypo is always part of an object of type HypoList. You cannot create unattached instances of the type Hypo. As words are stored as indecees, a list of hypotheses is always defined over an vocabulary object.

Creating a HypoList object:

HypoList hypolist myvocab

Hypo methods:

• puts
  puts returns formated output for a single hypothesis:
  optional parameters:
  • -style <"simple"|"normal"|"verbose"|"tcl">
    • simple: unfiltered string of all words in hypothesis
    • normal: normal unfiltered word string succeded by score and hypothesis-identifier in parentheses.
    • verbose: verbose hypothesis format as in JANUS-2
    • tcl: nested tcl list with utterance score, words, word begin and end points
  • -id <string>
    hypothesis-identifier -- for instance the number in an nbest list.
  example:
  • puts [hl:0 puts -style normal -id hypo0]
    ( das ist $ ein Termin ) (hypo0 | score = 0.30)

HypoList methods:

• puts return formated list of hypotheses
  puts returns formated output for a list of hypotheses:
  optional parameters:
  • -style <"simple"|"normal"|"verbose"|"tcl">
    • simple: unfiltered strings of all words in each hypothesis, hyposeses are seperated by a single \n
    • normal: normal hypotheses list preceded by %TURN line with utterance-identifier and each line succeded by score and hypothesis-identifier.
    • verbose: verbose hypothesis format as in JANUS-2
    • tcl: nested tcl list with utterance score, words, word begin and end points
  • -id <string>
    utterance-identifier -- this is wat is printed after %TURN:
• setPublish
  syntax: <hypo list> setPublish <list of commands>
  specify the tcl command list to be used for the method publish:
  As tcl user you have to use the method setPublish for each object of the type HypoList to specify what kind of output you need. If setPublish is not used for an object, the default routine for the type HypoList is used : unfiltered, normal output is made to stdout, and if a filename is given also to filename. The default behaviour can be changed by overdefining the tcl procedure publishHypoList {hl args}.
  The variable $hl and the array $arg will be defined for you to use. $hl contains the hypothesis list, $arg is an associative array with the elements of the list passed to the publish method. The general idea is very similar to providing preprocessing functions in the feature module.
  examples:
  • hl setPublish {puts [$hl puts -style normal -uttid $arg(uttid)]}
  • hl setPublish @<filename>
  • more examples at the end of this file
• publish
  make a neat output with your tcl code
  examples:
  • hl publish {{filename test.all} {uttid GBD_001}}
  • hl publish {{key1 value1} {key2 value2} {key3 value3}}
• append
  append a hypothesis to an existing list
  example: hl append hl:0

GENERAL HYPOS EXAMPLE SCRIPT:

Vocab    vocab ../env-GSST/VM_train_dict.67; # vocabulary 
HypoList hl vocab;                   # create empty object hl
hl append {
 4.0 
 {  ( 0.9  0 10 } 
 { ja 1.1 11 20 } 
 {  ) 2.0 21 40 }
};                                   # append "( ja )"
puts [hl puts];                      # tcl output
{
 4.00
 {(   0.900 0 10}
 {ja   1.100 11 20}
 {)   2.000 21 40}
}

puts [hl puts -style simple];        # simple output
( ja ) 

puts [hl puts -sty normal -id DEMO]; # corresponds to w in v2
%TURN: DEMO
( ja )  (0 | score =    4.00)

puts [hl puts -sty verbose -id DEMO];# corresponds to v in v2
%TURN: DEMO
hypo 0 (score =    4.00)
 (                         0.90    0 ->   10
 ja                        1.10   11 ->   20
 )                         2.00   21 ->   40

hl publish {};                        # using default function
%TURN: unknown
( ja )  (0 | score =    4.00)

hl publish {{uttid DEMO}};            # still using default function
%TURN: DEMO
( ja )  (0 | score =    4.00)

hl publish {{uttid DEMO} {filename t.all}}; # still default function
%TURN: DEMO
( ja )  (0 | score =    4.00)

same thing is also appended to file t.all

hl setPublish {puts "Hallo"};         # overdefine publish for hl
hl publish;                           # now it does something else:
Hallo

hl setPublish {puts "NEXT HYPO:";puts [$hl puts -style simple];}
hl publish
NEXT HYPO:
( ja )

hl setPublish {
 set  fileH  [open $arg(filename) $arg(mode)];
 puts $fileH [$hl puts -style normal -id $arg(uttid)];
 puts        [$hl puts -style normal -id $arg(uttid)];
 close $fileH;
}
hl publish {{filename t.all} {mode a} {uttid DEMO_000}}
%TURN: DEMO_000
( ja )  (0 | score =    4.00)

same thing is also appended to file t.all

hl setPublish {
 set  fileH  [open $arg(trunc)_lz$arg(lz)_lp$arg(lp).all a];
 puts $fileH [$hl puts -style normal -id $arg(uttid)];
 puts        [$hl puts -style normal -id $arg(uttid)];
 close $fileH;
}
hl publish  {{trunc hypo} {lz 0.3} {lp 0.1} {uttid DEMO_000}}
%TURN: DEMO_000
( ja )  (0 | score =    4.00)

same thing is also appended to file hypo_lp0.3_lz0.1.all

EXAMPLES FOR THINGS TO PUT AFTER hl setPublish:

#
# all these functions will only use filename and uttid and would be
# called with something like
# hl publish {filename test.all} {uttid G127/G127A001_A}
#

#
# unfiltered output to filename and stdout (corresponding to wW in JANUS-2)
#
hl setPublish {
 set fh [open $arg(filename) a];
 puts -nonewline $fh [$hl puts -style normal -id $arg(uttid)];
 puts -nonewline [$hl puts -style normal -id $arg(uttid)];
 close $fh;
}

#
# filtered output to filename and stdout (corresponding to oO in JANUS-2)
# the filter is in the regsub line.
#
hl setPublish {
 set unfilt [$hl puts -style normal -id $arg(uttid)];
 regsub -all \\(\ |\ \\)|\\$\  $unfilt "" filt;
 set fh [open $arg(filename) a];
 puts -nonewline $fh $filt;
 puts -nonewline     $filt;
 close $fh;
}

#
# the default routine provided if setPublish is not set would
# look like this:
#
hl setPublish {
  set uttid unknown;                                     
  if [info exists arg(uttid)] {set uttid $arg(uttid)};   
  puts [$hl puts -style normal -id $uttid];              
  if [info exists arg(filename)] {                       
     set fh [open $arg(filename) a];                     
     puts $fh [$hl puts -style normal -id $uttid];       
     close $fh;                                          
  }                                                      
}

#
# a real wild format to stdout
#
hl setPublish {
 set wildHypoN [$hl configure -hypoN];
 puts "here come all hypos in the list:";
 for {set i 0} {$i< $wildHypoN} {set i [expr $i+1]} {
   set wildHypoScore [$hl:$i configure -score];
   set wildHypoWords [$hl:$i puts -style simple];
   puts "  Hypo Number $i has score $wildHypoScore and the words:";
   puts "  $wildHypoWords";
 }
}