= Welcome to the Wiki of the seminar ''Deep Natural Language Processing'' in the winter semester 2020/2021 =
This seminar is organized at the chair of [[http://ad.informatik.uni-freiburg.de/staff/bast|Prof. Dr. Hannah Bast]] by [[http://ad.informatik.uni-freiburg.de/staff/klumpp|Theresa Klumpp]], [[http://ad.informatik.uni-freiburg.de/staff/hertel|Matthias Hertel]] and [[http://ad.informatik.uni-freiburg.de/staff/prange|Natalie Prange]]. The seminar will take place every Tuesday, 2:15 pm - 3:45 pm, in the seminar room SR 00-010/14 in building 101 and via Zoom for those of you who want to attend online (details will follow). '''Attendance in one of these two forms is compulsory.''' There will be '''no''' session on Tuesday, December 29th, 2020 and Tuesday, January 5th, 2021 (Christmas break).

== Important Links ==
Please register for the course in [[https://campus.uni-freiburg.de/qisserver/pages/cm/exa/coursemanagement/basicCourseData.xhtml?_flowId=searchCourseNonStaff-flow&_flowExecutionKey=e1s3|HISinOne]] '''and''' [[https://daphne.informatik.uni-freiburg.de/login/?service=/ws2021/DeepNlp/|Daphne]].

{{{#!html
<!-- # 
If you have questions (both general or specific to your topic), please ask a question on the [[https://daphne.informatik.uni-freiburg.de/forum/viewforum.php?f=774/|forum]] after registering for [[https://daphne.informatik.uni-freiburg.de/ws2021/DeepNlp/|Daphne]].
 -->
}}}

There is an introduction to SVN in [[http://ad-wiki.informatik.uni-freiburg.de/teaching/SVNEnglish|english]] and [[http://ad-wiki.informatik.uni-freiburg.de/teaching/SVN|german]].

== Modalities ==
Participants of the seminar will have to present one of the topics either alone or as a group of two. Each '''presentation''' will be 30 minutes for one participant or 2 * 20 minutes for two. In addition to introducing the topic each presentation must include a '''demo''' part where participants present a practical application of their topic.

What exactly this demo entails depends on the topic and will be discussed with each person/team separately. While we will provide suggestions you are very welcome to bring in your own ideas. Examples for demos may include the implementation of a ''small application'', an ''interactive visualization'' or the ''demonstration of a complex existing system which you have set up on your own''.

== Sessions ==
||Session ||Date ||Topic ||
||1 ||Tuesday, November  3rd, 2020 ||'''Introduction and Organization''' (by Prof. Hannah Bast) ||
|| ||Tuesday, November  10th, 2020 ||'''NO SESSION''' ||
||2 ||Tuesday, November  17th, 2020 ||'''Machine Learning Introduction''' (by us) ||
||3 ||Tuesday, November  24th, 2020 ||'''Deep Learning & Tensorflow Introduction''' (by us) ||
||4 ||Tuesday, December 1st, 2020 ||'''Standard Language Model''' ||
||5 ||Tuesday, December 8th, 2020 ||'''RNN Language Model''' ||
||6 ||Tuesday, December 15th, 2020 ||'''word2vec''' ||
||7 ||Tuesday, December 22nd, 2020 ||'''?''' ||
|| ||Tuesday, December 29th, 2020 ||'''NO SESSION''' ||
|| ||Tuesday, January 5th, 2021 ||'''NO SESSION''' ||
||8 ||Tuesday, January 12th, 2021 ||'''?''' ||
||9 ||Tuesday, January 19th, 2021 ||'''?''' ||
||10 ||Tuesday, January 26th, 2021 ||'''?''' ||
||11 ||Tuesday, February 2nd, 2021 ||'''?''' ||
||12 ||Tuesday, February 9th, 2021 ||'''?''' ||

{{{#!html
<!-- # 
== Topics ==
The topics are going to be introduced and roughly explained in the first session. They are basically about how Deep Learning can be used in Natural Language Processing.


'''Please note that you are supposed to present the topic, not the material listed here.''' The material is only intended as a starting point for your research.


 * '''Standard Language Model'''
  * Modelling natural languages using unigrams, n-grams and statistics
  * Material:
   * [[https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf|Lecture Notes from Stanford]]
 * '''RNN Language Model'''
  * Character based language modelling with Deep Neural Networks
  * Material:
   * [[http://karpathy.github.io/2015/05/21/rnn-effectiveness/|Andrej Karpathy - The Unreasonable Effectiveness of Recurrent Neural Networks]]
 * '''word2vec'''
  * Representing words as vectors from a high dimensional vector space
  * Material:
   * [[https://arxiv.org/abs/1301.3781|Original Paper by Mikolov et al]]
   * [[https://www.youtube.com/watch?v=wTp3P2UnTfQ|YouTube Video Explanation]]
   * [[https://radimrehurek.com/gensim/|Gensim framework for word vector generation]]
 * '''Paraphrasing and Synonyms'''
  * Challenges and approaches for handling the ambiguity of natural language
  * Material:
   * [[http://www.lrec-conf.org/proceedings/lrec2012/pdf/266_Paper.pdf|CrossWikis Wikipedia extracted synonyms]]
   * [[http://aclweb.org/anthology/P/P13/P13-1158.pdf|Question Paraphrasing]] (advanced)
 * '''Convolutional Neural Networks for NLP'''
  * A neural network architecture from image processing applied to NLP
  * Material:
   * [[http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/|Introductory Blog Post]]
   * [[https://github.com/facebookresearch/fairseq|CNN based Machine Translation by Facebook]] (advanced)
 * '''Text Classification'''
  * Classifying text using Machine Learning: Approaches and Techniques
  * Material:
   * [[https://medium.com/towards-data-science/machine-learning-nlp-text-classification-using-scikit-learn-python-and-nltk-c52b92a7c73a|Tutorial]]
 * '''Sentiment Analysis'''
  * Using NLP for emotion extraction: Approaches and Techniques
  * Material:
   * [[https://blog.openai.com/unsupervised-sentiment-neuron/|Accidentally discovered state of the art sentiment analysis by OpenAI]]
 * '''Attention'''
  * Attending to detail for Neural Networks
  * Material:
   * [[https://talbaumel.github.io/attention/|Learning string.reverse() with Attention]]
   * [[http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/|Introductory Blog Post]]
 * '''Question Answering on Text'''
  * Finding answers in a heap of text
  * Material:
   * [[https://rajpurkar.github.io/SQuAD-explorer/|SQuad Benchmark]]
 * '''Question Answering on Knowledge Bases'''
  * Finding answers in a warehouse of facts
  * Material:
   * [[https://www.microsoft.com/en-us/research/publication/semantic-parsing-via-staged-query-graph-generation-question-answering-with-knowledge-base/|Microsoft Research QA system STAGG]]
   * [[http://ad-publications.informatik.uni-freiburg.de/CIKM_freebase_qa_BH_2015.pdf|Our QA system Aqqu (code on request)]]

== General Additional Information ==
The following links are universally useful for all topics

 * Deep Learning for NLP at Stanford: [[https://nlp.stanford.edu/courses/NAACL2013/|older]] and [[http://web.stanford.edu/class/cs224n/syllabus.html|newer]]
 * Deep Learning [[http://www.deeplearningbook.org/|the book]] by Ian Goodfellow
 * Yoav Goldberg’s [[http://u.cs.biu.ac.il/~yogo/nnlp.pdf|Primer on NNs for NLP]]

Mainly for Question Answering:

 * Our chair's [[http://ad-publications.informatik.uni-freiburg.de/FNTIR_semanticsearch_BBH_2016.pdf|Survey on Semantic Search]]
 -->
}}}