Trailing whitespace is removed after parsing.

[latkins]

There appears to be a bug when parsing a string:

import spacy

nlp = spacy.load('en')

text = "This is a string "
doc = nlp(text)
print(doc.text_with_ws)
print([token.text_with_ws for token in doc])
assert(doc.text_with_ws == text)

Gives

'This is a string'
['This ', 'is ', 'a ', 'string']

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-51-da27fad27bad> in <module>()
      5 print([t.text_with_ws for t in doc])
      6 
----> 7 assert(doc.text_with_ws == text)

AssertionError: 

Your Environment

• Operating System: OSX 10.12.2

• Python Version Used: 3.5

• spaCy Version Used: 1.6.0

• Environment Information:

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.