Doc.to_json() method fails when called on doc generated via Span.as_doc()

[JohnStuartRutledge]

How to reproduce the behaviour

It would appear that calling to_json() on a Doc that was created via the as_doc() method is prone to failure. This may very well be connected to other issues with with as_doc like the that mentioned in issue #3669

import spacy

nlp = spacy.load('en')
doc = nlp('He jests at scars, that never felt a wound.')
span = doc[0:3]      # "He jests at"
doc2 = span.as_doc()
doc2.to_json()
# IndexError: [E040] Attempt to access token at 7, max length 3.

The above error seems to trigger when second token of the span (jests), calls spacy.tokens.Token.head.__get__() and hits the self.c.head

Environment

• spaCy version: 2.1.6
• Platform: Darwin-18.6.0-x86_64-i386-64bit
• Python version: 3.6.4
• Models: en

[svlandeg]

Yep, I can replicate this bug with en_core_web_md (not with en_core_web_lg) - looks like it's definitely connected to the trouble caused by the dependency parsing, as discussed in that other issue...

[svlandeg]

@JohnStuartRutledge : I think we can fix this, cf. #3969. Basically what happens in your example, is that the second word jests refers to felt (index: 7) as its head word in the dependency graph, but after the as_doc operation that index is out of bounds. So we need to prevent copying that kind of out-of-span information ...

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.