Loading [MathJax]/jax/output/CommonHTML/jax.js

[Paper Reading]A Bit of Progress in Language Modeling

2018-08-30

11731, Machine Translation, Paper reading

reference
- A Bit of Progress in Language Modeling

Smoothing
- Simple interpolation
  $\scriptsize P_{interpolate}(w\;\vert\;w_{i-2}w_{i-1})\;=\;\lambda P_{trigram}(w\;\vert\;w_{i-2}w_{i-1})\;+\;(1-\lambda)(\mu P_{bigram}(w\;\vert\;w_{i-1})\;+\;(1-\mu)P_{unigram}(w))$
- Katz smoothing
  - Based on Good-Turing formula, if count > 0
    $n_r\;is\;the\;frequency\;of\;frequency$
  - back off, otherwise
- Backoff Kneser-Ney smoothing
- Interpolated Kneser-Ney smoothing

Posts

python trick

Some elegant elements of writing python. reference Tricks for python transpose 12345mat = [[1,2,3],[4,5,6]]transpose = zip(*mat) # unpack matprint(lis

CMU 11-731(MT&Seq2Seq) Advanced Topics: Adaptation Methods

reference Adaptation Methods Adapting sequence-to-sequence models to a particular type of problem. Ensembling $$ P(E\;\vert\;F)\;=\;\lambda P_1(E\;\ve

CMU 11-731(MT&Seq2Seq) Advanced Topics: Multi-lingual Models

reference Multi-lingual Models Pivot Translation Result pivoting use existing source-pivot and pivot-target systems (F is source language, G is pivoti

CMU 11-731(MT&Seq2Seq) Algorithms for MT 2 Parameter Optimization Methods

reference Parameter Optimization Methods Error Functions and Error Minimization error function $error(\varepsilon,\widehat\varepsilon)$ difficulty

CMU 11-731(MT&Seq2Seq) Applications 1 Monolingual Sequence-to-sequence Problems

reference Monolingual Sequence-to-sequence Problems Paraphrase Generation re-wording sentences into other sentences with the same content but diț

Pages

(Untitled)

Xia Chen’s website

Resume

Chen(Chris) XiaResumeFirst Year Master StudentLanguage Technologies Institute,Carnegie Mellon University5000 Forbes AvePittsburgh, PA 15213-3891, USAE