-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix critical issues in FastText
#2313
Merged
Merged
Commits on Dec 16, 2018
-
Configuration menu - View commit details
-
Copy full SHA for 94a20e9 - Browse repository at this point
Copy the full SHA 94a20e9View commit details -
Configuration menu - View commit details
-
Copy full SHA for fb2b5b0 - Browse repository at this point
Copy the full SHA fb2b5b0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1a41182 - Browse repository at this point
Copy the full SHA 1a41182View commit details -
Configuration menu - View commit details
-
Copy full SHA for cd0b318 - Browse repository at this point
Copy the full SHA cd0b318View commit details
Commits on Dec 23, 2018
-
Configuration menu - View commit details
-
Copy full SHA for 3b31288 - Browse repository at this point
Copy the full SHA 3b31288View commit details -
Configuration menu - View commit details
-
Copy full SHA for 42626a2 - Browse repository at this point
Copy the full SHA 42626a2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 12cc3e2 - Browse repository at this point
Copy the full SHA 12cc3e2View commit details
Commits on Dec 28, 2018
-
Configuration menu - View commit details
-
Copy full SHA for 64f7f39 - Browse repository at this point
Copy the full SHA 64f7f39View commit details -
Configuration menu - View commit details
-
Copy full SHA for 00b472b - Browse repository at this point
Copy the full SHA 00b472bView commit details
Commits on Dec 29, 2018
-
Configuration menu - View commit details
-
Copy full SHA for abfd573 - Browse repository at this point
Copy the full SHA abfd573View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4e46062 - Browse repository at this point
Copy the full SHA 4e46062View commit details -
$ ~/src/fastText-0.1.0/fasttext cbow -input toy-data.txt -output toy-model -bucket 100 Read 0M words Number of words: 22 Number of labels: 0 Progress: 100.0% words/sec/thread: 209 lr: 0.000000 loss: 4.100698 eta: 0h0m -14m
Configuration menu - View commit details
-
Copy full SHA for d3544c7 - Browse repository at this point
Copy the full SHA d3544c7View commit details -
Configuration menu - View commit details
-
Copy full SHA for b98bc0b - Browse repository at this point
Copy the full SHA b98bc0bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 30be5bd - Browse repository at this point
Copy the full SHA 30be5bdView commit details -
Configuration menu - View commit details
-
Copy full SHA for ab1eaf6 - Browse repository at this point
Copy the full SHA ab1eaf6View commit details
Commits on Dec 30, 2018
-
retrain model with a smaller dimensionality
this will make it easier to debug manually $ ~/src/fastText-0.1.0/fasttext cbow -input toy-data.txt -output toy-model -bucket 100 -dim 5 Read 0M words Number of words: 22 Number of labels: 0 Progress: 100.0% words/sec/thread: 199 lr: 0.000000 loss: 0.000000 eta: 0h0m
Configuration menu - View commit details
-
Copy full SHA for e59d1db - Browse repository at this point
Copy the full SHA e59d1dbView commit details
Commits on Dec 31, 2018
-
Configuration menu - View commit details
-
Copy full SHA for 392201b - Browse repository at this point
Copy the full SHA 392201bView commit details -
Configuration menu - View commit details
-
Copy full SHA for f25607f - Browse repository at this point
Copy the full SHA f25607fView commit details -
Configuration menu - View commit details
-
Copy full SHA for ef394ed - Browse repository at this point
Copy the full SHA ef394edView commit details -
Configuration menu - View commit details
-
Copy full SHA for fe10ca7 - Browse repository at this point
Copy the full SHA fe10ca7View commit details
Commits on Jan 2, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 4c2223c - Browse repository at this point
Copy the full SHA 4c2223cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 28bf757 - Browse repository at this point
Copy the full SHA 28bf757View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8e0d04f - Browse repository at this point
Copy the full SHA 8e0d04fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 795fed0 - Browse repository at this point
Copy the full SHA 795fed0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 671b3c0 - Browse repository at this point
Copy the full SHA 671b3c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9adb532 - Browse repository at this point
Copy the full SHA 9adb532View commit details -
it cannot pass by design: training is non-deterministic, so conditions must be tightly controlled to guarantee reproducibility, and that is too much effort for a unit test
Configuration menu - View commit details
-
Copy full SHA for 6de08de - Browse repository at this point
Copy the full SHA 6de08deView commit details -
Configuration menu - View commit details
-
Copy full SHA for 81dd478 - Browse repository at this point
Copy the full SHA 81dd478View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5c500f0 - Browse repository at this point
Copy the full SHA 5c500f0View commit details -
Configuration menu - View commit details
-
Copy full SHA for cb045de - Browse repository at this point
Copy the full SHA cb045deView commit details -
Configuration menu - View commit details
-
Copy full SHA for a916266 - Browse repository at this point
Copy the full SHA a916266View commit details -
Configuration menu - View commit details
-
Copy full SHA for cee6311 - Browse repository at this point
Copy the full SHA cee6311View commit details -
Configuration menu - View commit details
-
Copy full SHA for 752cf9b - Browse repository at this point
Copy the full SHA 752cf9bView commit details -
initialize wv.buckets_word prior to continuing training
This avoid a null dereference that could previously be reproduced with: python -c "from gensim.test.test_fasttext;import NativeTrainingContinuationTest as A;A().test_continuation_gensim()"
Configuration menu - View commit details
-
Copy full SHA for ad3342a - Browse repository at this point
Copy the full SHA ad3342aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 64caa3c - Browse repository at this point
Copy the full SHA 64caa3cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2c9f2b4 - Browse repository at this point
Copy the full SHA 2c9f2b4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 80c8092 - Browse repository at this point
Copy the full SHA 80c8092View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d30cae - Browse repository at this point
Copy the full SHA 0d30caeView commit details
Commits on Jan 3, 2019
-
Configuration menu - View commit details
-
Copy full SHA for bf1c8b8 - Browse repository at this point
Copy the full SHA bf1c8b8View commit details -
Configuration menu - View commit details
-
Copy full SHA for ec92983 - Browse repository at this point
Copy the full SHA ec92983View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1c58119 - Browse repository at this point
Copy the full SHA 1c58119View commit details -
Configuration menu - View commit details
-
Copy full SHA for 91b3599 - Browse repository at this point
Copy the full SHA 91b3599View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5100335 - Browse repository at this point
Copy the full SHA 5100335View commit details -
Configuration menu - View commit details
-
Copy full SHA for aae713d - Browse repository at this point
Copy the full SHA aae713dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e5ec723 - Browse repository at this point
Copy the full SHA e5ec723View commit details -
Configuration menu - View commit details
-
Copy full SHA for 87f655a - Browse repository at this point
Copy the full SHA 87f655aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 76aca9a - Browse repository at this point
Copy the full SHA 76aca9aView commit details
Commits on Jan 4, 2019
-
refactor FB model loading code
Move the lower-level FB model loading code to a new module. Implement alternative, simpler _load_fast_text_format function. Add unit tests to compare alternative and existing implementation.
Configuration menu - View commit details
-
Copy full SHA for 8027459 - Browse repository at this point
Copy the full SHA 8027459View commit details -
Configuration menu - View commit details
-
Copy full SHA for 07f34e2 - Browse repository at this point
Copy the full SHA 07f34e2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 118cd7f - Browse repository at this point
Copy the full SHA 118cd7fView commit details
Commits on Jan 5, 2019
-
Configuration menu - View commit details
-
Copy full SHA for ef58c7c - Browse repository at this point
Copy the full SHA ef58c7cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 799596d - Browse repository at this point
Copy the full SHA 799596dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6cf3d1f - Browse repository at this point
Copy the full SHA 6cf3d1fView commit details -
Configuration menu - View commit details
-
Copy full SHA for b58a50b - Browse repository at this point
Copy the full SHA b58a50bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 97baf3c - Browse repository at this point
Copy the full SHA 97baf3cView commit details -
Configuration menu - View commit details
-
Copy full SHA for ef90436 - Browse repository at this point
Copy the full SHA ef90436View commit details
Commits on Jan 6, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 8956530 - Browse repository at this point
Copy the full SHA 8956530View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2e10ece - Browse repository at this point
Copy the full SHA 2e10eceView commit details
Commits on Jan 7, 2019
-
Configuration menu - View commit details
-
Copy full SHA for cb25448 - Browse repository at this point
Copy the full SHA cb25448View commit details -
Configuration menu - View commit details
-
Copy full SHA for 901eaeb - Browse repository at this point
Copy the full SHA 901eaebView commit details -
Configuration menu - View commit details
-
Copy full SHA for f0bd22d - Browse repository at this point
Copy the full SHA f0bd22dView commit details -
Configuration menu - View commit details
-
Copy full SHA for fa34d84 - Browse repository at this point
Copy the full SHA fa34d84View commit details -
Configuration menu - View commit details
-
Copy full SHA for f9c1547 - Browse repository at this point
Copy the full SHA f9c1547View commit details -
Configuration menu - View commit details
-
Copy full SHA for de7d9ef - Browse repository at this point
Copy the full SHA de7d9efView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2946896 - Browse repository at this point
Copy the full SHA 2946896View commit details -
Configuration menu - View commit details
-
Copy full SHA for a7c14d0 - Browse repository at this point
Copy the full SHA a7c14d0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5598e19 - Browse repository at this point
Copy the full SHA 5598e19View commit details -
Configuration menu - View commit details
-
Copy full SHA for 07c84f5 - Browse repository at this point
Copy the full SHA 07c84f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for f15094d - Browse repository at this point
Copy the full SHA f15094dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e25a4f - Browse repository at this point
Copy the full SHA 5e25a4fView commit details -
Configuration menu - View commit details
-
Copy full SHA for b789971 - Browse repository at this point
Copy the full SHA b789971View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1ed35ea - Browse repository at this point
Copy the full SHA 1ed35eaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0f62660 - Browse repository at this point
Copy the full SHA 0f62660View commit details -
Configuration menu - View commit details
-
Copy full SHA for c461193 - Browse repository at this point
Copy the full SHA c461193View commit details -
Configuration menu - View commit details
-
Copy full SHA for eeafdec - Browse repository at this point
Copy the full SHA eeafdecView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3e0e656 - Browse repository at this point
Copy the full SHA 3e0e656View commit details -
Revert "refactoring: remove unused vectors_vocab_norm attribute"
This reverts commit 07c84f5. We have to worry about backwards compatibility if we remove this attribute, and it's not worth doing that as part of this PR.
Configuration menu - View commit details
-
Copy full SHA for 262599d - Browse repository at this point
Copy the full SHA 262599dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d4e60e - Browse repository at this point
Copy the full SHA 7d4e60eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6cc80de - Browse repository at this point
Copy the full SHA 6cc80deView commit details -
review response: fix docstring in fasttext_bin.py
Also ran python -m doctest gensim/models/fasttext_bin.py to check the docstring is correctly executable.
Configuration menu - View commit details
-
Copy full SHA for 069912f - Browse repository at this point
Copy the full SHA 069912fView commit details -
Configuration menu - View commit details
-
Copy full SHA for cc19393 - Browse repository at this point
Copy the full SHA cc19393View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1661c16 - Browse repository at this point
Copy the full SHA 1661c16View commit details
Commits on Jan 8, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 72b1d81 - Browse repository at this point
Copy the full SHA 72b1d81View commit details -
Configuration menu - View commit details
-
Copy full SHA for e467060 - Browse repository at this point
Copy the full SHA e467060View commit details -
Configuration menu - View commit details
-
Copy full SHA for c2740cd - Browse repository at this point
Copy the full SHA c2740cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for daa425a - Browse repository at this point
Copy the full SHA daa425aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 64844f3 - Browse repository at this point
Copy the full SHA 64844f3View commit details -
vectors_lockf is only for word2vec. FastText implementation uses vectors_ngrams_lockf and vectors_vocab_lockf only.
Configuration menu - View commit details
-
Copy full SHA for 39e85f1 - Browse repository at this point
Copy the full SHA 39e85f1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 60d0477 - Browse repository at this point
Copy the full SHA 60d0477View commit details -
Configuration menu - View commit details
-
Copy full SHA for 52e2fbe - Browse repository at this point
Copy the full SHA 52e2fbeView commit details -
Configuration menu - View commit details
-
Copy full SHA for d08500b - Browse repository at this point
Copy the full SHA d08500bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3a2f93e - Browse repository at this point
Copy the full SHA 3a2f93eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3159a18 - Browse repository at this point
Copy the full SHA 3159a18View commit details
Commits on Jan 9, 2019
-
Configuration menu - View commit details
-
Copy full SHA for f262815 - Browse repository at this point
Copy the full SHA f262815View commit details -
Configuration menu - View commit details
-
Copy full SHA for b80c329 - Browse repository at this point
Copy the full SHA b80c329View commit details -
Configuration menu - View commit details
-
Copy full SHA for 127a13e - Browse repository at this point
Copy the full SHA 127a13eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2b96550 - Browse repository at this point
Copy the full SHA 2b96550View commit details -
Configuration menu - View commit details
-
Copy full SHA for 25ad1ae - Browse repository at this point
Copy the full SHA 25ad1aeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 09388ec - Browse repository at this point
Copy the full SHA 09388ecView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6054aa8 - Browse repository at this point
Copy the full SHA 6054aa8View commit details -
Configuration menu - View commit details
-
Copy full SHA for b92f435 - Browse repository at this point
Copy the full SHA b92f435View commit details -
Configuration menu - View commit details
-
Copy full SHA for 422e3b1 - Browse repository at this point
Copy the full SHA 422e3b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 553c8e0 - Browse repository at this point
Copy the full SHA 553c8e0View commit details -
Configuration menu - View commit details
-
Copy full SHA for d42e506 - Browse repository at this point
Copy the full SHA d42e506View commit details -
fixup: introduce OrderedDict to _fasttext_bin.py
The order of the words matters. In the previous implementation, this was maintained explicitly via the index2word list, but using an OrderedDict achieves the same thing. The main idea is that we iterate over the vocab terms in the right order in the prepare_vocab function.
Configuration menu - View commit details
-
Copy full SHA for 425e942 - Browse repository at this point
Copy the full SHA 425e942View commit details -
Configuration menu - View commit details
-
Copy full SHA for 802587a - Browse repository at this point
Copy the full SHA 802587aView commit details -
Configuration menu - View commit details
-
Copy full SHA for ff82b71 - Browse repository at this point
Copy the full SHA ff82b71View commit details -
1
Configuration menu - View commit details
-
Copy full SHA for dab47f3 - Browse repository at this point
Copy the full SHA dab47f3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 914aa95 - Browse repository at this point
Copy the full SHA 914aa95View commit details -
Configuration menu - View commit details
-
Copy full SHA for 65abda9 - Browse repository at this point
Copy the full SHA 65abda9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 01d84d1 - Browse repository at this point
Copy the full SHA 01d84d1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 611cdb2 - Browse repository at this point
Copy the full SHA 611cdb2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0c959a9 - Browse repository at this point
Copy the full SHA 0c959a9View commit details -
Configuration menu - View commit details
-
Copy full SHA for c196ace - Browse repository at this point
Copy the full SHA c196aceView commit details -
Configuration menu - View commit details
-
Copy full SHA for fb51a6a - Browse repository at this point
Copy the full SHA fb51a6aView commit details
Commits on Jan 10, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 768a941 - Browse repository at this point
Copy the full SHA 768a941View commit details -
Configuration menu - View commit details
-
Copy full SHA for f4643bb - Browse repository at this point
Copy the full SHA f4643bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for d802e91 - Browse repository at this point
Copy the full SHA d802e91View commit details -
Configuration menu - View commit details
-
Copy full SHA for 92da774 - Browse repository at this point
Copy the full SHA 92da774View commit details -
Configuration menu - View commit details
-
Copy full SHA for e638628 - Browse repository at this point
Copy the full SHA e638628View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6e47a88 - Browse repository at this point
Copy the full SHA 6e47a88View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cdad39 - Browse repository at this point
Copy the full SHA 2cdad39View commit details -
Configuration menu - View commit details
-
Copy full SHA for fbaf086 - Browse repository at this point
Copy the full SHA fbaf086View commit details -
Configuration menu - View commit details
-
Copy full SHA for dc32126 - Browse repository at this point
Copy the full SHA dc32126View commit details -
Configuration menu - View commit details
-
Copy full SHA for 39e8844 - Browse repository at this point
Copy the full SHA 39e8844View commit details -
Configuration menu - View commit details
-
Copy full SHA for 08ee7d8 - Browse repository at this point
Copy the full SHA 08ee7d8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6d8a648 - Browse repository at this point
Copy the full SHA 6d8a648View commit details -
Configuration menu - View commit details
-
Copy full SHA for 734a0ac - Browse repository at this point
Copy the full SHA 734a0acView commit details -
Configuration menu - View commit details
-
Copy full SHA for 143445e - Browse repository at this point
Copy the full SHA 143445eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 250d388 - Browse repository at this point
Copy the full SHA 250d388View commit details -
review response: get rid of struct_unpack
This is an internal method masquerading as a public one. There is no reason for anyone to call it. Removing it will have no effect on pickling/unpickling, as methods do not get serialized. Therefore, removing it is safe.
Configuration menu - View commit details
-
Copy full SHA for 9fcf35e - Browse repository at this point
Copy the full SHA 9fcf35eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c1aeb85 - Browse repository at this point
Copy the full SHA c1aeb85View commit details -
Configuration menu - View commit details
-
Copy full SHA for 58c1166 - Browse repository at this point
Copy the full SHA 58c1166View commit details -
Configuration menu - View commit details
-
Copy full SHA for e5960ed - Browse repository at this point
Copy the full SHA e5960edView commit details
Commits on Jan 11, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 52230aa - Browse repository at this point
Copy the full SHA 52230aaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 14c497d - Browse repository at this point
Copy the full SHA 14c497dView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.