Aqqu

Description

Question answering from Freebase as described in the CIKM 2015 publication. The code below also contains some improvements (neural network, performance) that came after the publication. The public code also contains a README that describes how to download, install, train and run the system. Below describes how to setup the demo (for which the code is not public).

Code

Public GitHub repository: https://github.com/elmar-haussmann/aqqu .

Internal git repository (contains work after publication, mainly neural net and performance improvements): https://bitbucket.org/elmar-haussmann/aqqu .

Internal git repository for the web-UI (we didn't put that public): https://bitbucket.org/elmar-haussmann/aqqu-webserver .

Demo

Aqqu instance

2016-06-30: runs under http://metropolis.informatik.uni-freiburg.de:5455

Start as follows on metropolis:

ssh metropolis
sudo su haussmae
cd /home/haussmae/demos/aqqu-demo
source venv/bin/activate   # was: activate aqqu
PYTHONPATH=$(pwd):$PYTHONPATH python webserver/translation_webserver.py

Virtuoso instance (via Elmars home)

2016-06-30: Virtuoso instance for Aqqu runs unter http://metropolis.informatik.uni-freiburg.de:9000/sparql .

Start virtuoso as follows. Server listens on port 8999. (Alternatively follow the instructions from scratch instructions below)

ssh metropolis
sudo su haussmae
cd /home/haussmae/keyword-translation
make start-virtuoso-mini-cai

Start the HTTP proxy as follows (kill old process manually, listens on port 9000):

ssh metropolis
sudo su haussmae
cd /home/haussmae/keyword-translation
make start-varnish

Virtuoso instance (from scratch)

Start virtuoso as follows. Server listens on port 8999.

mkdir virtuoso-freebase
cd virtuoso-freebase
# get dependencies
sudo apt install bison flex gperf libssl-dev
# get the source
git clone https://github.com/openlink/virtuoso-opensource.git
cd virtuoso-opensource
git checkout stable/7
./autogen.sh
./configure --prefix=$(pwd)/virtuoso/install
# check everything ran error free and that no virtuoso is already running
# on this machine. The following steps intermediately start an instance and
# fail otherwise
make -j && make install
cd ..
# get the data
wget "http://elba.informatik.uni-freiburg.de/freebase-qa/data/virtuoso.tar.gz" && tar xvfz virtuoso.tar.gz
# run in tmux
tmux
./virtuoso-opensource/virtuoso/install/bin/virtuoso-t -f +configfile virtuoso-db/virtuoso.ini
# ctrl+b+d

Parser

Aqqu uses a parser to get part-of-speech tags of query words. The parser is accessed via HTTP Api calls. To start the parser server:

ssh metropolis
sudo su haussmae
cd /home/haussmae/keyword-translation
make start-parser

The port is configured in the corenlp-frontent/build.xml. The API can be accessed like this: http://metropolis.informatik.uni-freiburg.de:4000/parse/?text=This%20is%20a%20test%20sentence.

Run the new Aqqu version (with NN)

Start on titan (requires GPU):

ssh titan
sudo su haussmae
cd /home/haussmae/aqqu-bitbucket
source activate aqqu
PYTHONPATH=$(pwd):$PYTHONPATH python webserver/translation_webserver.py

Runs on port 5454 on titan now. However, titan is not available from outside the uni network.

To start a port-forwarding from metropolis:

ssh metropolis
sudo su haussmae
cd /home/haussmae/temp/nc/python-port-forwardt
python2 port-forward.py

The service is now available on metropolis:5454

How to update (any) Virtuoso with custom data

Grant access rights via the ISQL tool as follows:

data/virtuoso/install/bin/isql localhost:1112 dba dba
grant execute on SPARQL_INSERT_DICT_CONTENT to SPARQL_UPDATE;
grant execute on SPARQL_INSERT_DICT_CONTENT to "SPARQL";
grant execute on SPARQL_DELETE_DICT_CONTENT to SPARQL_UPDATE;
grant execute on SPARQL_DELETE_DICT_CONTENT to "SPARQL";

Complex example SPARQL query from "Programmieren in C++, SS 2016, Ü10 (all action or animation movie with their release date, genre, director, production company, and rating):

PREFIX fb: <http://rdf.freebase.com/ns/>
 SELECT DISTINCT ?fn, ?y, ?gn, ?dn, ?pn, ?rn where {
 ?f fb:type.object.type fb:film.film .
 ?f fb:film.film.initial_release_date ?y .
 ?f fb:film.film.genre ?g .
 ?f fb:film.film.directed_by ?d .
 ?f fb:film.film.production_companies ?p .
 ?f fb:film.film.rating ?r .
 ?f fb:type.object.name ?fn .
 ?g fb:type.object.name ?gn .
 ?d fb:type.object.name ?dn .
 ?p fb:type.object.name ?pn .
 ?r fb:type.object.name ?rn
 FILTER(lang(?fn)='en')
 FILTER(lang(?gn)='en')
 FILTER(lang(?dn)='en')
 FILTER(lang(?pn)='en')
 FILTER(lang(?rn)='en')
 FILTER(?gn='Action Film'@en OR ?gn='Animation'@en)
}

Data

All of the required data to run Aqqu is part of the materials (see above). It is located in the data subfolder. The scripts to create this data are part of the (old) keyword-translation repository: https://bitbucket.org/onekonek/keyword-translation