Results for Exercise Sheet 2 (Ranking)
These results should be based on the file movies.test-benchmark.tsv. Click "Edit (Text)" in the upper bar of this page and add your row to the table below, following the examples already there. Please write down all values with exactly two digits of precision. In the first column, write your account name, or name1+name2 if you work in a group.
Name |
MP@3 |
MP@R |
MAP |
BM25 Parameters |
Refinements |
pb1042 |
0.67 |
0.67 |
0.68 |
b=0.2, k=0.8 |
None (baseline) |
mi81-mk1357 |
0.83 |
0.77 |
0.75 |
b=0, k=0 |
queries changed |
hc63 |
0.83 |
0.74 |
0.63 |
b=0, k=0 |
Query changes |
ar479-ss1736 |
0.83 |
0.71 |
0.70 |
b=0.04, k=0.7 |
None (baseline) |
ew98 |
0.77 |
0.71 |
0.70 |
b=0.1, k=0.2 |
Query changes |
tl109 |
0.72 |
0.69 |
0.67 |
b=0.01, k=0.22 |
Random search for parameters, hardcoded some simple English grammar rules (actually hurt the performance) |
lg377 |
0.78 |
0.70 |
0.69 |
b=0, k=0 |
Query changes |
bw63 |
0.77 |
0.70 |
0.70 |
b=0, k=0 |
Trying to interpret context (.01 diffrence :D) |
ls720 |
0.83 |
0.71 |
0.69 |
b=0, k=0.71 |
None (baseline) |
sr463 |
0.61 |
0.51 |
0.5 |
b=0, k=0 |
Remove most common words from query - reduced performance |
oa49 |
0.77 |
0.70 |
0.69 |
b=0, k=0 |
None (baseline) |
tb305 |
0.56 |
0.37 |
0.36 |
b=0, k=1.75 |
results with words together boosted |
mb925 |
0.77 |
0.63 |
0.67 |
b=0.048, k=0.011 |
minor query change + nevergrad for b, k |
kk486 |
0.79 |
0.72 |
0.7 |
b=0, k=1 |
Query change |
jm408-ms946 |
0.77 |
0.70 |
0.69 |
b = k = 0 |
None (baseline) |
ak482 |
0.77 |
0.7 |
0.69 |
b=1 k=0 |
None (baseline) |
pm187 |
0.77 |
0.70 |
0.69 |
b=0 k=0 |
Only optimized b,k on training set. Adding synonyms with nltk to query and/or removing common english words have decreased evaluation performance. |
sw540 |
0.78 |
0.71 |
0.69 |
b=0.56, k=0 |
pso for optimizing b and k, remove stop words in keywords using a dictionary |
ie15 |
0.83 |
0.74 |
0.72 |
b=0.11, k=0.23 |
query changes and giving more popular movies a bonus factor |
rk268-ss1731 |
0.5 |
0.57 |
0.64 |
b=0.02, k=0.7 |
Slight query changes |
us58 |
0.89 |
0.73 |
0.72 |
b=0.05, k=0.65 |
Removed the keywords "films", "movies", "with" and "for" from the queries. |
jo120-tb350 |
0.83 |
0.63 |
0.61 |
b=k=0 |
b and k were optimized on training set |
mn228 |
0.83 |
0.73 |
0.72 |
b=0 k=1 |
tested using multiple values of b and k. Also removed some common stop words |
ek247 |
0.83 |
0.72 |
0.71 |
b=0.1 k=0.85 |
Removed some unspecific words from queries and added Zipf's law refinement |
rt53 |
0.666 |
0.662 |
0.680 |
b=0.18 k = 0.70 |
None. Actually turned out better than expected (I expected < 0.5) |
ll190 |
0.78 |
0.70 |
0.68 |
b=0.05, k=0.2 |
None (baseline) |
ds565 |
0.88 |
0.753 |
0.728 |
b=0 k=0 |
modified queries |
hd103-sa285 |
0.89 |
0.75 |
0.72 |
b=0.04, k=0.7 |
Made changes in the queries. |
rr186 |
0.78 |
0.70 |
0.68 |
b=0, k=0.01 |
None (baseline) |
am802-jb978 |
0.78 |
0.66 |
0.68 |
b=0.1, k=0.1 |
Used BM25+ and played around with b, k on the training set |
sb655 |
0.89 |
0.72 |
0.71 |
b=0.1, k=0.8 |
Removed some keywords from queries. |
js1662-kp152 |
0.94 |
0.75 |
0.72 |
b=0.0, k=1 |
Changed the queries and also used optimal b and k values as derived from training benchmark. |
al453 |
0.88 |
0.52 |
0.49 |
b=0.41, k=0.31 |
None (baseline) |
pr218 |
0.56 |
0.45 |
0.48 |
b=0.3, k=0.5 |
Made changes in the Queries. |
fm213 |
0.78 |
0.70 |
0.68 |
b=0, k=0.4 |
None (baseline) |
mr427 |
0.78 |
0.71 |
0.77 |
b=0.0, k=0.0 |
None (baseline) |
ha86-kb368 |
0.83 |
0.71 |
0.71 |
b=0.0, k=0.0 |
modified queries |
mr716 |
0.77 |
0.70 |
0.61 |
b=0, k=0 |
Removes some stop words |
mk1243 |
0.72 |
0.71 |
0.70 |
b=0.0, k=0.0 |
None (baseline) |
ma441 |
0.78 |
0.60 |
0.65 |
b=0.0, k=0.0 |
Removed stopwords in query and gave more importance to movies occurrence order in movies2.txt |
hg132 |
0.77 |
0.71 |
0.63 |
b=0.0, k=0.0 |
Using Stem words to optimize the search results |
ba102 |
0.83 |
0.68 |
0.69 |
b=0.1, k=0.1 |
Changed all occurrences of 'films' to 'film' |