Pickling of PhraseMatcher not yet resolved in 2.1.3

[kushalc]

#1971 and #3248 show that pickling works, but I'm still able to repro issues in 2.1.3.

Minimal Reproducible Example

Building off #3252 unit test:

from spacy.matcher import PhraseMatcher
from spacy.lang.en import English
from spacy.compat import pickle
nlp = English()

matcher = PhraseMatcher(nlp.vocab)
matcher.add("TEST1", None, nlp("a"), nlp("b"), nlp("c"))
matcher.add("TEST2", None, nlp("d"))
data = pickle.dump(matcher, open("matcher.pickle", "wb"))
new_matcher = pickle.load(open("matcher.pickle", "rb"))
print(new_matcher(nlp("a b c")))  # prints [(9789093191506324669, 0, 1), (9789093191506324669, 1, 2), (9789093191506324669, 2, 3)]
print(matcher(nlp("a b c")))  # prints [(9789093191506324669, 0, 1), (9789093191506324669, 1, 2), (9789093191506324669, 2, 3)]

(re-start Python)

from spacy.lang.en import English
from spacy.compat import pickle
nlp = English()
new_matcher = pickle.load(open("matcher.pickle", "rb"))
new_matcher(nlp("a b c"))  # prints []

Info about spaCy

• spaCy version: 2.1.3
• Platform: Darwin-18.2.0-x86_64-i386-64bit
• Python version: 3.7.2
• Models: en

[svlandeg]

I can reproduce your error with the code above. However, I don't think it's a Matcher pickling issue. If you would keep the same nlp object rather than constructing a new one, you wouldn't run into this, right?

[kushalc]

Is there a recommended best practice for pickling nlp/Matcher between different processes? The fundamental use case we have to solve is serialization across different machines so we can't simply re-use the same nlp instance. Kushal

…

On Thu, Jul 11, 2019 at 8:34 AM Sofie Van Landeghem < ***@***.***> wrote: I can reproduce your error with the code above. However, I don't think it's a Matcher pickling issue. If you would keep the same nlp object rather than constructing a new one, you wouldn't run into this, right? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3494?email_source=notifications&email_token=AAAMMOUHRGYG5WXGP5W4HFTP65HH5A5CNFSM4HBV3CS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZXCYNA#issuecomment-510536756>, or mute the thread <https:/notifications/unsubscribe-auth/AAAMMOWGIE266T6BTKJRNBTP65HH5ANCNFSM4HBV3CSQ> .

[svlandeg]

Is it an option to serialize the nlp object (https://spacy.io/usage/saving-loading) ?

nlp1.to_disk("mynlp")
nlp2 = spacy.load("mynlp")

[no-response]

This issue has been automatically closed because there has been no response to a request for more information from the original author. With only the information that is currently in the issue, there's not enough information to take action. If you're the original author, feel free to reopen the issue if you have or find the answers needed to investigate further.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.