Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Replication Log and Refactor Instructions for MS Marco Passage Retreival - Entire Dev Set #116

Merged
merged 1 commit into from
Nov 17, 2020

Conversation

rayyang29
Copy link
Contributor

Compute Canada environment:
OS: linux CentOS-7
Java 13.0.1
Python 3.6.3
Re-Ranking with monoBERT
2020-11-13 14:36:43 [INFO] evaluate_passage_ranker: Reranking:
100%|██████████| 6980/6980 [63:20:00<00:00, 32.66s/it]
2020-11-16 05:56:44 [INFO] evaluate_passage_ranker: precision@1 0.2533
2020-11-16 05:56:44 [INFO] evaluate_passage_ranker: recall@3 0.45093
2020-11-16 05:56:44 [INFO] evaluate_passage_ranker: recall@50 0.80609
2020-11-16 05:56:44 [INFO] evaluate_passage_ranker: recall@1000 0.86289
2020-11-16 05:56:44 [INFO] evaluate_passage_ranker: mrr 0.38789
2020-11-16 05:56:44 [INFO] evaluate_passage_ranker: mrr@10 0.37922

Re-Ranking with monoT5
2020-11-13 14:46:35 [INFO] evaluate_passage_ranker: Reranking:
100%|██████████| 6980/6980 [26:50:07<00:00, 13.84s/it]
2020-11-14 17:36:42 [INFO] evaluate_passage_ranker: precision@1 0.25129
2020-11-14 17:36:42 [INFO] evaluate_passage_ranker: recall@3 0.45362
2020-11-14 17:36:42 [INFO] evaluate_passage_ranker: recall@50 0.80709
2020-11-14 17:36:42 [INFO] evaluate_passage_ranker: recall@1000 0.86289
2020-11-14 17:36:42 [INFO] evaluate_passage_ranker: mrr 0.38839
2020-11-14 17:36:42 [INFO] evaluate_passage_ranker: mrr@10 0.37986

I have also slightly refactored the instructions for MS Marco Entire. I've fixed some typos and added a note clarifying the installing of tensorflow on CC with the instruction: pip install tensorflow_gpu

Copy link
Member

@ronakice ronakice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for clarifying!

@ronakice ronakice merged commit bcbb07e into castorini:master Nov 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants