Lemmatization of won't

[chrisjbryant]

can't and won't get tokenized as ca n't and wo n't respectively.
In previous versions of spacy, the lemma of ca and wo was ca and wo.
Sicne the recent update, ca now is correctly lemmatized as can, but wo is still wo when it should be will.

Edit:
Also just noticed the lemma of sha in sha n't is not converted to shall.

Your Environment

• spaCy version: 1.7.3
• Platform: Linux-4.8.0-34-generic-x86_64-with-Ubuntu-16.04-xenial
• Installed models: en, en_core_web_md
• Python version: 3.5.2

[honnibal]

Thanks! Will fix.

[ines]

Just fixed this on master and it should work now – the tokenizer exceptions for the contractions were missing a TAG, so the lemma was overwritten by the lemmatizer.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.