Current Topics in Natural Language Processing (SS 2019)

Summary

Deep Learning is an interesting new branch of machine learning where neural networks consisting of multiple layers have shown new generalization capabilities. The seminar will look at advances in both general deep learning approaches, and at the specific case of Neural Machine Translation (NMT). NMT is a new paradigm in data-driven machine translation. In Neural Machine Translation, the entire translation process is posed as an end-to-end supervised classification problem, where the training data is pairs of sentences and the full sequence to sequence task is handled in one model.

Here is a link to last semester's seminar.

There is a Munich interest group for Deep Learning, which has an associated mailing list, the paper announcements are sent out on this list. See the link here.

Instructors

Alexander Fraser

Email Address: SubstituteLastName@cis.uni-muenchen.de

CIS, LMU Munich


Hinrich Schütze

CIS, LMU Munich

Schedule

Thursdays 14:45 (s.t.), location is room 115

Click here for directions to CIS.

New attendees are welcome. Read the paper and bring a paper or electronic copy with you, you will need to refer to it during the discussion.

If this page appears to be out of date, use the refresh button of your browser

Date Paper Links Discussion Leader
Thursday, April 4th Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew Peters, Noah A. Smith (2019). Linguistic Knowledge and Transferability of Contextual Representations. NAACL 2019 paper Mengjie Zhao
Thursday, April 25th Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Weinberger (2017). On Calibration of Modern Neural Networks. ICML 2017 paper Alex Fraser
Thursday, May 2nd Pankaj Gupta, Yatin Chaudhary, Florian Buettner, Hinrich Schütze (2019). textTOvec: Deep Contextualized Neural Autoregressive Topic Models of Language with Distributed Compositional Prior. ICLR 2019 paper Pankaj Gupta
Thursday, May 9th Xin Wang, Wenhu Chen, Jiawei Wu, Yuan-Fang Wang, William Yang Wang (2018). Video Captioning via Hierarchical Reinforcement Learning. CVPR 2018. paper Dario Stojanovski
Thursday, May 16th Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, Quoc V. Le (2019). Unsupervised Data Augmentation. arXiv. paper Nora Kassner
Thursday, May 23rd Tal Schuster, Ori Ram, Regina Barzilay, Amir Globerson (2019). Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing. NAACL paper Viktor Hangya
Thursday, June 13th Alexei Baevski, Sergey Edunov, Yinhan Liu, Luke Zettlemoyer, Michael Auli (2019). Cloze-driven Pretraining of Self-attention Networks. arXiv paper Marina Sedinkina
Thursday, June 27th Review of recent publications / presentations: ACL, NAACL, Google NLP Summit, arxiv etc. Please pick one publication and put a few highlights (pictures are welcome!) on a slide. Send your slide to Masoud, who will collect them. Masoud Jalili Sabet
Thursday, July 4th Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, Nathan Srebro (2017). Exploring Generalization in Deep Learning. NIPS 2017. (We read citation 30 before) nips
arxiv (extended)
Tom Sterkenburg
Thursday, July 11th Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv paper Masoud Jalili Sabet
Thursday, July 18th Junjie Hu, Mengzhou Xia, Graham Neubig, Jaime Carbonell (2019). Domain Adaptation of Neural Machine Translation by Lexicon Induction. ACL 2019 paper Matthias Huck
Wednesday, July 24th, 12:45, room 057 RESEARCH TALK: Dominik Schlechtweg, IMS Stuttgart. Second-order Co-occurrence Sensitivity of Skip-Gram with Negative Sampling
Thursday, July 25th RESEARCH TALK: Andre Cianflone, MILA (McGill University). Generalizable Adversarial Attacks Using Generative Models
Thursday, August 22nd Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv paper Nina Pörner
Thursday, September 12th RESEARCH TALK: Denis Peskov, U Maryland. Gathering Language Data Reliably At Scale. abstract
Thursday, September 19th Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, Sebastian Riedel (2019). Language Models as Knowledge Bases? EMNLP 2019 paper Nora Kassner


Further literature:

You can go back through the previous semesters by clicking on the link near the top of the page.