[Bug] GoldParse instantiation with Misaligned tokens.

[tamuhey]

How to reproduce the behaviour

1. IndexError

text = "A'B C"
words = ["A", "'", "B", "C"]
doc = tokenizer(text)
GoldParse(doc, words=words)

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-7-45b5f2ec7502> in <module>
      2 words = ["A", "'", "B", "C"]
      3 doc = tokenizer(text)
----> 4 GoldParse(doc, words=words)

gold.pyx in spacy.gold.GoldParse.__init__()

IndexError: list index out of range

2. TypeError

text = "A-B"
words = ["A-B"]
doc = tokenizer(text)
GoldParse(doc, words=words)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-9ae6959d65fe> in <module>
      2 words = ["A-B"]
      3 doc = tokenizer(text)
----> 4 GoldParse(doc, words=words)

gold.pyx in spacy.gold.GoldParse.__init__()

TypeError: list indices must be integers or slices, not NoneType

Your Environment

• Operating System: mac OSX
• Python Version Used: 3.7
• spaCy Version Used: master

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.