Advanced Seminar in Neural Machine Translation

Summary

Neural Machine Translation (NMT) is a new paradigm in data-driven machine translation. Previous generation Statistical Machine Translation (SMT) systems are built using a collection of heuristic models, typically combined in a log-linear model with a small number of parameters. In Neural Machine Translation, the entire translation process is posed as an end-to-end supervised classification problem, where the training data is pairs of sentences. While in SMT systems, word-alignment is carried out, and then fixed, and then various sub-models are estimated from the word-aligned data, this is not the case in NMT. In NMT, fixed word-alignments are not used, and instead the full sequence to sequence task is handled in one model.

Here is a link to last semester's seminar.

NEW: David Kaumanns is also organizing a Munich interest group for Deep Learning, which has an associated mailing list. See the link here: http://www.cis.uni-muenchen.de/~davidk/deep-munich/

Instructor

Alexander Fraser

Email Address: SubstituteLastName@cis.uni-muenchen.de

CIS, LMU Munich

Schedule

Thursdays 14:30 s.t., location is C105 (CIS Besprechungsraum).

Click here for directions to CIS.

If this page appears to be out of date, use the refresh button of your browser

Date Paper Links Discussion Leader
Thursday, April 28th Karthik Narasimhan, Tejas Kulkarni, Regina Barzilay (2015). Language Understanding for Text-based Games Using Deep Reinforcement Learning. Proceedings of EMNLP (Best paper honorable mention) paper and slides David Kaumanns
Thursday, May 19th Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, Samy Bengio (2016). Generating Sentences from a Continuous Space. arXiv preprint. paper Ben Roth
Thursday, June 9th Rico Sennrich, Barry Haddow, Alexandra Birch (2015). Neural Machine Translation of Rare Words with Subword Units. arXiv preprint. paper Matthias Huck
Thursday, June 23rd Junyoung Chung, Kyunghyun Cho, Yoshua Bengio (2016). A Character-level Decoder without Explicit Segmentation for Neural Machine Translation. arXiv preprint. paper Ales Tamchyna
Thursday, July 14th Carl Doersch (2016). Tutorial on Variational Autoencoders. arXiv preprint. paper Yadollah Yaghoobzadeh
Thursday, July 21st Yangfeng Ji, Gholamreza Haffari, Jacob Eisenstein (2016). A Latent Variable Recurrent Neural Network for Discourse Relation Language Models. NAACL 2016 paper Liane Guillou
Thursday, July 28th Presentation by Ivan Bilan on Bilan+Zhekova submission to the PAN shared task No Reading
Thursday, August 18th 5 favorite papers from ACL 2016, WMT 2016, etc. (Strict 1 minute per paper) Everyone (Please Bring a Handout)
Thursday, August 25th Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov (2016). Enriching Word Vectors with Subword Information. arXiv preprint. paper
Thursday, October 6th Dan Gillick, Cliff Brunk, Oriol Vinyals, Amarnag Subramanya (2016). Multilingual Language Processing From Bytes. HLT-NAACL 2016. paper Hinrich Sch├╝tze
Thursday, October 13th Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, Hang Li (2016). Modeling Coverage for Neural Machine Translation. ACL 2016 paper Tsuyoshi Okita


Further literature:

Please click here for an NMT reading list, but also see the more general RNN reading list here (scroll down).