Using the Search module from TCL

Before using the search, you usually have to create quite a few objects it depends on directly or indirectly; Directly the search only depends on the vocabulary and the language model.

Creating a search object

Search _search _vocab _lm;  # this should create a search object

Search Methods

treeFwd: perform a tree forward search
flatFwd: perform a flat forward search
fullFwd: perform a flat forward search does not require a tree pass to be performed well. This is advisable for small vocabularies (< 300 words) and a must for very small vocabularies (< 50 words) because the tree-search will not work well with those.
lattice: build lattice and perform lattice rescoring

Example:

Search _search _vocab _lm;        # this should create a search object
_search treeFwd -eval test.adc;   # this does the tree search for file test.adc
_search lattice;                  # create lattice and do trigram rescoring

Search Configuration

It's mostly the submodules of the search that are configured. The search itself only has the following:

search configure \
  -phoneInsertionPenalty 0.000000 \
  -wordPenalty 0.000000 \
  -silenceWordPenalty 0.150000 \
  -fillerWordPenalty 0.000000 \
  -use3gFwd 1 \
  -verbose 1 \
  -printWidth 70

Tree Forward Configuration

The Tree Forward can be configured before and after a search object has been created: before, you can modify the creation defaults for all search objects created lateron;

TreeFwd configure \
  -beamWidth 88.00 \
  -phoneBeamWidth 88.00 \
  -wordBeamWidth 87.00 \
  -lastPhoneBeamWidth 87.00 \
  -lastPhoneBeamWidth 87.00 \
  -topN 0 \
  -hypoFormat {{other stderr unfiltered} {file hypo.T.all append filtered}}}

Alternatively, you can configure the tree forward after creating a Search object:

Search _search _vocab _lm;        # this should create a search object
  search.treeFwd configure \
  -beamWidth 88.000000 \
  -phoneBeamWidth 88.000000 \
  -wordBeamWidth 87.000000 \
  -lastPhoneBeamWidth 87.000000 \
  -lastPhoneBeamWidth 87.000000 \
  -topN 0 \
  -hypoFormat {{other stderr unfiltered} {file hypo.T.all append filtered}}

Search Beams

The beams mean (roughly):

beginners hint: if you are clueless, set the beamWidth to average score for two phonemes (10 frames) for evaluations. Set it to the average score for one phoneme for demos.
Set phoneBeamWidth and lastPhonebeamWidth to the same value as beamWidth. Set lastPhoneAloneBeamWidht and wordBeamWidth about 20% smaller than the other tree.
Set topN to 15% of your vocabulary size for evaluations and to 15 (absolute) for demos.
beamWidth: hypos with local scores that differ by more than beamWidth from the currently best local hypo will be pruned (inactive in next frame), unless they are currently in their last phoneme.
lastPhoneAloneBeamWidth: hypos in a last phoneme of a word will be pruned if their local score differs by more than lastPhoneAloneBeamWidth from the best hypo within any last phoneme of any word. The optimum value for the lastPhoneAloneBeamWidth is usually 20-50% smaller than the beamWidth. This beam only exists in the tree pass.
phoneBeamWidth: hypos that could do a phoneme transition but have local scores that differ by more than phoneBeamWidth from the currently best local hypo will not be allowed to do such a transition (unless to the last phoneme of the word); The phoneBeamWidth is usually smaller or equal compared to the beamWidth.
lastPhoneBeamWidth: hypos with local scores that differ by more than lastPhoneBeamWidth from the currently best local hypo will not be allowed ot do a transition into the last phoneme of the word. The lastPhoneBeamWidth is usually smaller or equal compared to the phoneBeamWidth.
wordBeamWidth: hypos with local scores that differ by more than wordBeamWidth from the best score in any last phoneme are not allowed to do word transitions.
topN: at any one time only the topN best word-ends are allowed to do word transitions. Only those will be stored in the backtrace and available to the next passes. For demos often values of 10 to 20 will do.

Search Info

The search stores internal bookkeeping stuff in places where you can get at it. The following are available as fmatrix in the search object (after tree pass only):

rootTrellis: number of active root trellises per frame.
nodeTrellis: number of active non root trellises per frame.
wordTrellis: number of active word trellises per frame.
leafTrellis: number of active leaf trellises per frame.
scoreCalls: number of score calls per frame.
bestScores: score of best local hypo per frame.
lPbestScores: score of best hypo in last phone per frame.

WARNING: the format of these is not 100% finalized. If you use them be ready to update your scripts every now and then.

If this page is incomplete, ask monika@ira.uka.de or finkem@cs.cmu.edu to complete it.