CMU 11-731(MT&Seq2Seq) Advanced Topics: Multi-lingual Models

• reference
  • Multi-lingual Models

Pivot Translation

• Result pivoting

  • use existing source-pivot and pivot-target systems (F is source language, G is pivoting language, E is target language)

    $$\widehat G\;=\underset G{\;argmax}\;P(G\vert F)$$
    $$\widehat E\;=\underset E{\;argmax}\;P(E\vert\widehat G)$$

• Data pivoting

  • pseudoparallel data

• Model pivoting

  • triangulation

Multi-lingual Training

• Multi-task Learning Approaches

  • multiple sources to one target
  • one source to mutiple targets
  • many sources to many targets
    • zero shot learning: testing of models on examples that do not exist in their training data

• Transfer Approaches

• Ensembling Approaches