Improvements on setting attributes in merge_noun_chunks function

[alaponin]

Feature description

It would be nice if during the merging of noun chunks the lemma would also be properly set. Currently, the lemma of the merged token is the lemma of the first word in the noun chunk sequence, which I don't think is the desired behavior in most cases.
Additionally, it would be nice to have more flexibility in setting the attributes of the merged token. For example, I would like to keep the entity type of the root token as the entity type of the whole merged token.

[ines]

I think those both sound like reasonable defaults, so we should probably consider just adding them to the built-in function 🙂 Feel free to submit a PR btw.

We probably want to avoid introducing settings to the built-in component functions – it adds too much complexity for what they are (small wrappers around doc.retokenize), and once we add settings, we then also need to add serialization methods to preserve them.

Btw, also in case others come across this issue later: the merge_noun_chunks function itself is tiny, so if you ever need fully custom settings that might not be good defaults, I'd recommend just copying it and writing your own:

spaCy/spacy/pipeline/functions.py

Lines 7 to 21 in 250a544

	def merge_noun_chunks(doc):
	"""Merge noun chunks into a single token.
	doc (Doc): The Doc object.
	RETURNS (Doc): The Doc object with merged noun chunks.
	DOCS: https://spacy.io/api/pipeline-functions#merge_noun_chunks
	"""
	if not doc.is_parsed:
	return doc
	with doc.retokenize() as retokenizer:
	for np in doc.noun_chunks:
	attrs = {"tag": np.root.tag, "dep": np.root.dep}
	retokenizer.merge(np, attrs=attrs)
	return doc

[ines]

Resolved by #4219!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.