Different ways of loading data model

[pokey]

I get different word vectors depending how I load the data model. For example:

import spacy.en
import spacy
import numpy

nlp1 = spacy.en.English()
nlp2 = spacy.load('en')

word = 'shop'

# The following assert fails:
assert(numpy.allclose(nlp1.vocab[word].vector, nlp2.vocab[word].vector))

My Environment

• OS X 10.11.6
• Python 3.5.2
• spacy 1.2.0

[honnibal]

Thanks, this was a bad bug! When I switched default support to the GloVe vectors in 1.0, I added a hack to the spacy.load() function, to provide temporary backwards compatibility to existing data installations. This hack should have been inserted into spacy.en.English().

The version of the vectors loaded by spacy.load() is the "correct" one, which will continue to be supported. If you want to see whether this bug affected some model you've been using, the easiest way to check whether you're using the correct GloVe vector data is to count the number of lexemes with a word vector:

>>> import spacy
>>> nlp = spacy.load('en')
>>> sum(w.has_vector for w in nlp.vocab)
645315

If you see a ~300,000, your model has the older vectors trained on Wikipedia loaded.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.