AI Seminar

Bhuwan Dhingra - Neural Architectures for Reading and Reasoning over Documents

Abstract: Reading and understanding natural language text is important for AI applications which need to extract information from unstructured sources. Models designed for this task must deal with complex linguistic phenomena such as paraphrasing, co-reference, logical entailment, syntactic and semantic dependencies, and so on. In this talk I will show how architectural biases, motivated from such phenomena, can be built into neural network models to boost machine reading performance.

The first half of the talk will focus on the Gated-Attention (GA) Reader model for learning fine-grained alignments between natural language queries and documents. The output of this model is a query-focused representation of the tokens in the document, which is used to extract the answer to the query. The second half of the talk will focus on extensions which utilize prior knowledge in the form of linguistic annotations to model long term dependencies in the document. Modeling long-term dependencies is the first step towards the more ambitious goal of reasoning over distinct parts of a document. Finally, I will discuss some of the key directions for future research, in terms of both improving the models and utilizing them for concrete applications.

This is joint work with Zhilin Yang, Hanxiao Liu, Russ Salakhutdinov and William Cohen.