Size: 5958
Comment:
|
Size: 8571
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 5: | Line 5: |
'''See below for the recording and slides of the introductory session''' | '''See below for the recording and slides of the introductory session.''' === Important: Information === Not everyone is in the Daphne DeepNLP course yet, '''please join if you're taking this seminar! ''' == Important Links == Please register for the course in [[https://campus.uni-freiburg.de:443/qisserver/pages/startFlow.xhtml?_flowId=showEvent-flow&unitId=3513&termYear=2017&termTypeValueId=2&navigationPosition=studiesOffered,searchCourses|HISinOne]] '''and''' [[https://daphne.informatik.uni-freiburg.de/login/?service=/ws1718/DeepNlp/|Daphne]]. If you have questions (both general or specific to your topic), please ask a question on the [[https://daphne.informatik.uni-freiburg.de/forum/viewforum.php?f=774/|forum]] after registering for [[https://daphne.informatik.uni-freiburg.de/login/?service=/ws1718/DeepNlp/|Daphne]]. |
Line 8: | Line 16: |
Participants of the seminar will have to present one of the topics either alone or as a group of two. Each '''presentation''' will be 30 minutes for one participant or 2 * 20 minutes for two. In addition to introducing the topic each presentation must include a '''demo''' part where participants present a practical application of their topic. |
Participants of the seminar will have to present one of the topics either alone or as a group of two. Each '''presentation''' will be 30 minutes for one participant or 2 * 20 minutes for two. In addition to introducing the topic each presentation must include a '''demo''' part where participants present a practical application of their topic. |
Line 11: | Line 18: |
What exactly this demo entails depends on the topic and will be discussed with each person/team separately. While we will provide suggestions you are very welcome to bring in your own ideas. Examples for demos may include the implementation of a ''small application'', an ''interactive visualization'' or the ''demonstration of a complex existing system which you have set up on your own'' | What exactly this demo entails depends on the topic and will be discussed with each person/team separately. While we will provide suggestions you are very welcome to bring in your own ideas. Examples for demos may include the implementation of a ''small application'', an ''interactive visualization'' or the ''demonstration of a complex existing system which you have set up on your own''. |
Line 15: | Line 22: |
||1 ||Wednesday, October 18th, 2017 ||'''Introduction and Organization''', [[http://www.youtube.com/TODO|Video Recording]] ([[http://ad-teaching.uni-freiburg.de/SeminarDeepNlpWS1718/session-01.mp4|Download]]), [[http://ad-teaching.informatik.uni-freiburg.de/SeminarDeepNlpWS1718/session-01.pdf|Slides]] || | ||1 ||Wednesday, October 18th, 2017 ||'''Introduction and Organization''', [[http://www.youtube.com/watch?v=eeuwVMOH0Uk|Video Recording]] ([[http://ad-teaching.informatik.uni-freiburg.de/SeminarDeepNlpWS1718/session-01.mp4|Download]]), [[http://ad-teaching.informatik.uni-freiburg.de/SeminarDeepNlpWS1718/session-01.pdf|Slides]] || |
Line 18: | Line 25: |
||2 ||Wednesday, November 8th, 2017 ||'''Machine Learning Introduction''' (Niklas Schnelle) + topic assignment || ||3 ||Wednesday, November 15th, 2017 ||'''Deep Learning Introduction''' (Thomas Goette) || ||4 ||Wednesday, November 22nd, 2017 || || ||5 ||Wednesday, November 29th, 2017 || || ||6 ||Wednesday, December 6th, 2017 || || ||7 ||Wednesday, December 13rd, 2017 || || ||8 ||Wednesday, December 20th, 2017 || || |
||2 ||Wednesday, November 8th, 2017 ||'''Machine Learning Introduction''' (Niklas Schnelle) + topic assignment, [[https://www.youtube.com/watch?v=YWuUQK3NUqQ|Video Recording]] ([[http://ad-teaching.informatik.uni-freiburg.de/SeminarDeepNlpWS1718/session-02.mp4|Download]]), [[https://ad-git.informatik.uni-freiburg.de/ad/nlpml|Code]], [[http://daphne.informatik.uni-freiburg.de/ws1718/DeepNlp/svn/session-02.pdf|Slides]] || ||3 ||Wednesday, November 15th, 2017 ||'''Deep Learning Introduction''' (Thomas Goette), [[https://www.youtube.com/watch?v=qAnsywbo3UU|Video Recording]] ([[http://ad-teaching.informatik.uni-freiburg.de/SeminarDeepNlpWS1718/session-03.mp4|Download]]), [[http://daphne.informatik.uni-freiburg.de/ws1718/DeepNlp/svn/session-03.pdf|Slides]] || ||4 ||Wednesday, November 22nd, 2017 ||'''Standard Language Model''' (Samuel Steinegger) || ||5 ||Wednesday, November 29th, 2017 ||'''RNN Language Model''' (Ikrima Bin Saeed) || ||6 ||Wednesday, December 6th, 2017 ||'''word2vec''' (Sameed Hayat, Salih Hasan Siddiqi) || ||7 ||Wednesday, December 13rd, 2017 ||'''Paraphrasing and Synonyms''' (Christopher Krolla, Tobias Matysiak) || ||8 ||Wednesday, December 20th, 2017 ||'''Convolutional Neural Networks for NLP''' (David-Elias Künstle, Jessica Pape) || |
Line 27: | Line 34: |
||9 ||Wednesday, January 10th, 2018 || || ||10 ||Wednesday, January 17th, 2018 || || ||11 ||Wednesday, January 24th, 2018 || || ||12 ||Wednesday, January 31st, 2018 || || ||13 ||Wednesday, February 7th, 2018 || || |
||9 ||Wednesday, January 10th, 2018 ||'''Text Classification''' (Frank Schüssele, Julian Schwarz) || ||10 ||Wednesday, January 17th, 2018 ||'''Sentiment Analysis''' (Jasmin Denk, Joao Carvalho) || ||11 ||Wednesday, January 24th, 2018 ||'''Attention''' (Didier Bissel, Omar Shehata) || ||12 ||Wednesday, January 31st, 2018 ||'''Question Answering on Text''' (Dennis Koch, Philip Klein) || ||<style=""text-align:left"" |2>13 ||<style=""text-align:left"" |2>Wednesday, February 7th, 2018 ||'''Question Answering on Knowledge Bases''' (Dennis Armbruster, Patrick Bumüller) || ||'''Speech to Text''' (Mohammad Fattouh) || |
Line 36: | Line 45: |
* '''Standard Language Model''' * Modelling natural languages using unigrams, n-grams and statistics * Material: * [[https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf|Lecture Notes from Stanford]] |
* '''Standard Language Model''' * Modelling natural languages using unigrams, n-grams and statistics * Material: * [[https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf|Lecture Notes from Stanford]] |
Line 41: | Line 50: |
* Character based language modelling with Deep Neural Networks * Material: * [[http://karpathy.github.io/2015/05/21/rnn-effectiveness/|Andrej Karpathy - The Unreasonable Effectiveness of Recurrent Neural Networks]] |
* Character based language modelling with Deep Neural Networks * Material: * [[http://karpathy.github.io/2015/05/21/rnn-effectiveness/|Andrej Karpathy - The Unreasonable Effectiveness of Recurrent Neural Networks]] |
Line 45: | Line 54: |
* Representing words as vectors from a high dimensional vector space * Material: * [[https://arxiv.org/abs/1301.3781|Original Paper by Mikolov et al]] * [[https://www.youtube.com/watch?v=wTp3P2UnTfQ|YouTube Video Explanation]] * [[https://radimrehurek.com/gensim/|Gensim framework for word vector generation]] |
* Representing words as vectors from a high dimensional vector space * Material: * [[https://arxiv.org/abs/1301.3781|Original Paper by Mikolov et al]] * [[https://www.youtube.com/watch?v=wTp3P2UnTfQ|YouTube Video Explanation]] * [[https://radimrehurek.com/gensim/|Gensim framework for word vector generation]] |
Line 51: | Line 60: |
* Challenges and approaches for handling the ambiguity of natural language * Material: * [[http://www.lrec-conf.org/proceedings/lrec2012/pdf/266_Paper.pdf|CrossWikis Wikipedia extracted synonyms]] * [[http://aclweb.org/anthology/P/P13/P13-1158.pdf|Question Paraphrasing]] (advanced) |
* Challenges and approaches for handling the ambiguity of natural language * Material: * [[http://www.lrec-conf.org/proceedings/lrec2012/pdf/266_Paper.pdf|CrossWikis Wikipedia extracted synonyms]] * [[http://aclweb.org/anthology/P/P13/P13-1158.pdf|Question Paraphrasing]] (advanced) |
Line 56: | Line 65: |
* A neural network architecture from image processing applied to NLP * Material: * [[http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/|Introductory Blog Post]] * [[https://github.com/facebookresearch/fairseq|CNN based Machine Translation by Facebook]] (advanced) |
* A neural network architecture from image processing applied to NLP * Material: * [[http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/|Introductory Blog Post]] * [[https://github.com/facebookresearch/fairseq|CNN based Machine Translation by Facebook]] (advanced) |
Line 61: | Line 70: |
* Classifying text using Machine Learning: Approaches and Techniques * Material: * [[https://medium.com/towards-data-science/machine-learning-nlp-text-classification-using-scikit-learn-python-and-nltk-c52b92a7c73a|Tutorial]] |
* Classifying text using Machine Learning: Approaches and Techniques * Material: * [[https://medium.com/towards-data-science/machine-learning-nlp-text-classification-using-scikit-learn-python-and-nltk-c52b92a7c73a|Tutorial]] |
Line 65: | Line 74: |
* Using NLP for emotion extraction: Approaches and Techniques * Material: * [[https://blog.openai.com/unsupervised-sentiment-neuron/|Accidentally discovered state of the art sentiment analysis by OpenAI]] |
* Using NLP for emotion extraction: Approaches and Techniques * Material: * [[https://blog.openai.com/unsupervised-sentiment-neuron/|Accidentally discovered state of the art sentiment analysis by OpenAI]] |
Line 69: | Line 78: |
* Attending to detail for Neural Networks * Material: * [[https://talbaumel.github.io/attention/|Learning string.reverse() with Attention]] * [[http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/|Introductory Blog Post]] |
* Attending to detail for Neural Networks * Material: * [[https://talbaumel.github.io/attention/|Learning string.reverse() with Attention]] * [[http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/|Introductory Blog Post]] |
Line 74: | Line 83: |
* Finding answers in a heap of text * Material: * [[https://rajpurkar.github.io/SQuAD-explorer/|SQuad Benchmark]] |
* Finding answers in a heap of text * Material: * [[https://rajpurkar.github.io/SQuAD-explorer/|SQuad Benchmark]] |
Line 78: | Line 87: |
* Finding answers in a warehouse of facts * Material: * [[https://www.microsoft.com/en-us/research/publication/semantic-parsing-via-staged-query-graph-generation-question-answering-with-knowledge-base/|Microsoft Research QA system STAGG]] * [[http://ad-publications.informatik.uni-freiburg.de/CIKM_freebase_qa_BH_2015.pdf|Our QA system Aqqu (code on request)]] |
* Finding answers in a warehouse of facts * Material: * [[https://www.microsoft.com/en-us/research/publication/semantic-parsing-via-staged-query-graph-generation-question-answering-with-knowledge-base/|Microsoft Research QA system STAGG]] * [[http://ad-publications.informatik.uni-freiburg.de/CIKM_freebase_qa_BH_2015.pdf|Our QA system Aqqu (code on request)]] == General Additional Information == The following links are universally useful for all topics * Deep Learning for NLP at Stanford: [[https://nlp.stanford.edu/courses/NAACL2013/|older]] and [[http://web.stanford.edu/class/cs224n/syllabus.html|newer]] * Deep Learning [[http://www.deeplearningbook.org/|the book]] by Ian Goodfellow * Yoav Goldberg’s [[http://u.cs.biu.ac.il/~yogo/nnlp.pdf|Primer on NNs for NLP]] Mainly for Question Answering: * Our chair's [[http://ad-publications.informatik.uni-freiburg.de/FNTIR_semanticsearch_BBH_2016.pdf|Survey on Semantic Search]] |
Welcome to the Wiki of the seminar ''Deep Natural Language Processing'' in the winter semester 2017/2018
This seminar is organized by Prof. Dr. Hannah Bast with the assistance of Niklas Schnelle and Thomas Goette. The seminar will take place every Wednesday, 4:15 pm - 5:45 pm, in the seminar room 02-017 in building 52. There will be no session on Wednesday, October 25th, 2017, on Wednesday, November 1st, 2017 (All Hallows), on Wednesday, December 27th, 2017, and on Wednesday, January 3rd, 2018 (christmas break).
See below for the recording and slides of the introductory session.
Important: Information
Not everyone is in the Daphne DeepNLP course yet, please join if you're taking this seminar!
Important Links
Please register for the course in HISinOne and Daphne.
If you have questions (both general or specific to your topic), please ask a question on the forum after registering for Daphne.
Modalities
Participants of the seminar will have to present one of the topics either alone or as a group of two. Each presentation will be 30 minutes for one participant or 2 * 20 minutes for two. In addition to introducing the topic each presentation must include a demo part where participants present a practical application of their topic.
What exactly this demo entails depends on the topic and will be discussed with each person/team separately. While we will provide suggestions you are very welcome to bring in your own ideas. Examples for demos may include the implementation of a small application, an interactive visualization or the demonstration of a complex existing system which you have set up on your own.
Sessions
Session |
Date |
Topic |
1 |
Wednesday, October 18th, 2017 |
Introduction and Organization, Video Recording (Download), Slides |
|
Wednesday, October 25th, 2017 |
NO SESSION |
|
Wednesday, November 1st, 2017 |
NO SESSION |
2 |
Wednesday, November 8th, 2017 |
Machine Learning Introduction (Niklas Schnelle) + topic assignment, Video Recording (Download), Code, Slides |
3 |
Wednesday, November 15th, 2017 |
Deep Learning Introduction (Thomas Goette), Video Recording (Download), Slides |
4 |
Wednesday, November 22nd, 2017 |
Standard Language Model (Samuel Steinegger) |
5 |
Wednesday, November 29th, 2017 |
RNN Language Model (Ikrima Bin Saeed) |
6 |
Wednesday, December 6th, 2017 |
word2vec (Sameed Hayat, Salih Hasan Siddiqi) |
7 |
Wednesday, December 13rd, 2017 |
Paraphrasing and Synonyms (Christopher Krolla, Tobias Matysiak) |
8 |
Wednesday, December 20th, 2017 |
Convolutional Neural Networks for NLP (David-Elias Künstle, Jessica Pape) |
|
Wednesday, December 27th, 2017 |
NO SESSION |
|
Wednesday, January 3rd, 2018 |
NO SESSION |
9 |
Wednesday, January 10th, 2018 |
Text Classification (Frank Schüssele, Julian Schwarz) |
10 |
Wednesday, January 17th, 2018 |
Sentiment Analysis (Jasmin Denk, Joao Carvalho) |
11 |
Wednesday, January 24th, 2018 |
Attention (Didier Bissel, Omar Shehata) |
12 |
Wednesday, January 31st, 2018 |
Question Answering on Text (Dennis Koch, Philip Klein) |
13 |
Wednesday, February 7th, 2018 |
Question Answering on Knowledge Bases (Dennis Armbruster, Patrick Bumüller) |
Speech to Text (Mohammad Fattouh) |
Topics
The topics are going to be introduced and roughly explained in the first session. They are basically about how Deep Learning can be used in Natural Language Processing.
Standard Language Model
- Modelling natural languages using unigrams, n-grams and statistics
- Material:
RNN Language Model
- Character based language modelling with Deep Neural Networks
- Material:
word2vec
- Representing words as vectors from a high dimensional vector space
- Material:
Paraphrasing and Synonyms
- Challenges and approaches for handling the ambiguity of natural language
- Material:
Convolutional Neural Networks for NLP
- A neural network architecture from image processing applied to NLP
- Material:
Text Classification
- Classifying text using Machine Learning: Approaches and Techniques
- Material:
Sentiment Analysis
- Using NLP for emotion extraction: Approaches and Techniques
- Material:
Attention
- Attending to detail for Neural Networks
- Material:
Question Answering on Text
- Finding answers in a heap of text
- Material:
Question Answering on Knowledge Bases
- Finding answers in a warehouse of facts
- Material:
General Additional Information
The following links are universally useful for all topics
Deep Learning the book by Ian Goodfellow
Yoav Goldberg’s Primer on NNs for NLP
Mainly for Question Answering:
Our chair's Survey on Semantic Search