07_Deep_NLP_p2.md

7. Deep NLP p2

[TOC]

Encoder-Decoder for machine tranlation

Encoder-Decoder

Attention Mechanism

Main problem with Encoder-Decoder:

• A sentence may have different parts with different concepts
• The whole sentence is represented as single vector

Solution :

• Use all outputs of the encoder $h_i$ to compute the outputs
• build an Attention Mechanism to determine which output to attend to

Reasonning, Attention and Memory

Question answering tasks

Memory Networks

Key-Value Networks