#acl Niklas Schnelle:read,write Thomas Goette:read,write
= Welcome to the Wiki of the seminar ''Deep Natural Language Processing'' in the winter semester 2017/2018 =
This seminar is organized by [[http://ad.informatik.uni-freiburg.de/staff/bast|Prof. Dr. Hannah Bast]] with the assistance of [[http://ad.informatik.uni-freiburg.de/staff/schnelle|Niklas Schnelle]] and [[mailto:thomas.goette@mars.uni-freiburg.de|Thomas Goette]]. The seminar will take place every Wednesday, 4:15 pm - 5:45 pm, in the seminar room 02-017 in building 52. There will be '''no''' session on Wednesday, October 25th, 2017, on Wednesday, November 1st, 2017 (All Hallows), on Wednesday, December 27th, 2017, and on Wednesday, January 3rd, 2018 (christmas break).
== Modalities ==
Participants of the seminar will have to present one of the topics either alone or as a group of two. Each '''presentation''' will be 20 minutes for one participant or 2 * 15 minutes for two. In addition to introducing the topic 
each presentation must include a '''demo''' part where participants present a practical application of their topic.

What exactly this demo entails depends on the topic and will be discussed with each team separately, while we will provide suggestions you are very welcome to bring in your own ideas. Examples for demos may include the implementation of a ''small application'', an ''interactive visualization'' or the ''demonstration of a complex existing system which you have set up on your own''
== Topics ==
The topics are going to be introduced and roughly explained in the first session. They are basically about how Deep Learning can be used in Natural Language Processing.

 * '''Standard Language Model''' 
   * Modelling natural languages using unigrams, n-grams and statistics
   * Material:
     * [[https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf|Lecture Notes from Stanford]]
 * '''RNN Language Model'''
   * Character based language modelling with Deep Neural Networks
   * Material:
     * [[http://karpathy.github.io/2015/05/21/rnn-effectiveness/|Andrej Karpathy - The Unreasonable Effectiveness of Recurrent Neural Networks]]
 * '''word2vec'''
   * Representing words as vectors from a high dimensional vector space
   * Material:
     * [[https://arxiv.org/abs/1301.3781|Original Paper by Mikolov et al]]
     * [[https://www.youtube.com/watch?v=wTp3P2UnTfQ|YouTube Video Explanation]]
     * [[https://radimrehurek.com/gensim/|Gensim framework for word vector generation]]
 * '''Paraphrasing and Synonyms'''
   * Challenges and approaches for handling the ambiguity of natural language
   * Material:
     * [[http://www.lrec-conf.org/proceedings/lrec2012/pdf/266_Paper.pdf|CrossWikis Wikipedia extracted synonyms]]
     * [[http://aclweb.org/anthology/P/P13/P13-1158.pdf|Question Paraphrasing]] (advanced)
 * '''Convolutional Neural Networks for NLP'''
   * A neural network architecture from image processing applied to NLP
   * Material:
     * [[http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/|Introductory Blog Post]]
     * [[https://github.com/facebookresearch/fairseq|CNN based Machine Translation by Facebook]] (advanced)
 * '''Text Classification'''
   * Classifying text using Machine Learning: Approaches and Techniques
   * Material:
     * [[https://medium.com/towards-data-science/machine-learning-nlp-text-classification-using-scikit-learn-python-and-nltk-c52b92a7c73a|Tutorial]]
 * '''Sentiment Analysis'''
   * Using NLP for emotion extraction: Approaches and Techniques
   * Material:
     * [[https://blog.openai.com/unsupervised-sentiment-neuron/|Accidentally discovered state of the art sentiment analysis by OpenAI]]
 * '''Attention'''
   * Attending to detail for Neural Networks
   * Material:
     * [[https://talbaumel.github.io/attention/|Learning string.reverse() with Attention]]
     * [[http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/|Introductory Blog Post]]
 * '''Question Answering on Text'''
   * Finding answers in a heap of text
   * Material:
     * [[https://rajpurkar.github.io/SQuAD-explorer/|SQuad Benchmark]]
 * '''Question Answering on Knowledge Bases'''
   * Finding answers in a warehouse of facts
   * Material:
     * [[https://www.microsoft.com/en-us/research/publication/semantic-parsing-via-staged-query-graph-generation-question-answering-with-knowledge-base/|Microsoft Research QA system STAGG]]
     * [[http://ad-publications.informatik.uni-freiburg.de/CIKM_freebase_qa_BH_2015.pdf|Our QA system Aqqu (Code on request)]]

== Sessions ==
||Session ||Date ||Topic ||
||1 ||Wednesday, October  18th, 2017 ||'''Introduction and Organization''' ||
|| ||Wednesday, October  25th, 2017 ||'''NO SESSION''' ||
|| ||Wednesday, November  1st, 2017 ||'''NO SESSION''' ||
||2 ||Wednesday, November  8th, 2017 ||'''Machine Learning Introduction''' (Niklas Schnelle) + topic assignment ||
||3 ||Wednesday, November 15th, 2017 ||'''Deep Learning Introduction''' (Thomas Goette) ||
||4 ||Wednesday, November 22nd, 2017 || ||
||5 ||Wednesday, November 29th, 2017 || ||
||6 ||Wednesday, December  6th, 2017 || ||
||7 ||Wednesday, December 13rd, 2017 || ||
||8 ||Wednesday, December 20th, 2017 || ||
|| ||Wednesday, December 27th, 2017 ||'''NO SESSION''' ||
|| ||Wednesday, January   3rd, 2018 ||'''NO SESSION''' ||
||9 ||Wednesday, January  10th, 2018 || ||
||10 ||Wednesday, January  17th, 2018 || ||
||11 ||Wednesday, January  24th, 2018 || ||
||12 ||Wednesday, January  31st, 2018 || ||
||13 ||Wednesday, February  7th, 2018 || ||