Bug in example

[jack-rory-staunton]

There is a slight error in the logic that causes incorrect deletion of entities which should be kept.

Which page or section is this issue related to?

https://spacy.io/usage/rule-based-matching
in the section Models and Rules,
in the example code snippet below,

from spacy.tokens import Span


def expand_person_entities(doc):
    new_ents = []
    for ent in doc.ents:
        # Only check for title if it's a person and not the first token
        if ent.label_ == "PERSON" and ent.start != 0:
            prev_token = doc[ent.start - 1]
            if prev_token.text in ("Dr", "Dr.", "Mr", "Mr.", "Ms", "Ms."):
                new_ent = Span(doc, ent.start - 1, ent.end, label=ent.label)
                new_ents.append(new_ent)
        else:
            new_ents.append(ent)
    doc.ents = new_ents
    return doc

One quick way to fix this would be:

from spacy.tokens import Span

def expand_person_entities(doc):
    new_ents = []
    for ent in doc.ents:
        # Only check for title if it's a person and not the first token
        if ent.label_ == "PERSON" and ent.start != 0:
            prev_token = doc[ent.start - 1]
            if prev_token.text in ("Dr", "Dr.", "Mr", "Mr.", "Ms", "Ms."):
                new_ent = Span(doc, ent.start - 1, ent.end, label=ent.label)
                new_ents.append(new_ent)
           else:
              new_ents.append(ent)
        else:
            new_ents.append(ent)
    doc.ents = new_ents
    return doc

[adrianeboyd]

That does look like a bug. Would you like to submit a PR to fix it?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.