MS-MARCO passage replication log

[yuxuan-ji]

Colab Environment

OS: Ubuntu 18.04.3 LTS
Java: openjdk 11.0.8 2020-07-14
Python: Python 3.6.9
Tested with Tesla T4

Replication results

BERT reranking: Identical

2020-09-09 01:24:16 [INFO] evaluate_passage_ranker: Reranking:
100% 105/105 [51:33<00:00, 29.46s/it]
2020-09-09 02:15:50 [INFO] evaluate_passage_ranker: precision@1 0.27619
2020-09-09 02:15:50 [INFO] evaluate_passage_ranker: recall@3    0.42698
2020-09-09 02:15:50 [INFO] evaluate_passage_ranker: recall@50   0.81746
2020-09-09 02:15:50 [INFO] evaluate_passage_ranker: recall@1000 0.84762
2020-09-09 02:15:50 [INFO] evaluate_passage_ranker: mrr         0.4109
2020-09-09 02:15:50 [INFO] evaluate_passage_ranker: mrr@10      0.40268

T5 reranking: Identical

2020-09-09 02:20:09 [INFO] evaluate_passage_ranker: Reranking:
100% 105/105 [1:12:39<00:00, 41.52s/it]
2020-09-09 03:32:49 [INFO] evaluate_passage_ranker: precision@1 0.26667
2020-09-09 03:32:49 [INFO] evaluate_passage_ranker: recall@3    0.46032
2020-09-09 03:32:49 [INFO] evaluate_passage_ranker: recall@50   0.80635
2020-09-09 03:32:49 [INFO] evaluate_passage_ranker: recall@1000 0.84762
2020-09-09 03:32:49 [INFO] evaluate_passage_ranker: mrr         0.39734
2020-09-09 03:32:49 [INFO] evaluate_passage_ranker: mrr@10      0.39044

[lintool]

hi @yuxuan-ji can you resolve the conflicts and indicate use of Colab? Thanks!