💫 Merge conllu converters into one function

[ines]

I was just updating the spacy convert command and docs and noticed that the converters are currently a bit messy. We might be able to eliminate some code duplication and make them a bit nicer overall. It'd be nice if we could merge the conll converters (or at least conll/conllu and conllubio) into a single script.

(That said, we do have to accept that they'll always be kinda hacky, simply because the formats we're dealing with aren't always 100% consistent. There are several variations of the .conll format alone that we need to handle).

Resources

• spacy.cli.converters: https:/explosion/spaCy/tree/develop/spacy/cli/converters
• Conll(u)-bio converter #2525: original PR and motivation for the conllubio converter

FYI: Future plans

1. Change converters to create and return Doc objects instead of JSON objects. This means that the new Doc.to_json object will be the single source of truth for the JSON format and we don't end up with arbitrary data transformation logic all over the place.
2. Update the training data format to a more straightforward JSONL format. See 💫 Proposal: New JSON(L) format for training and improved training commands #2928.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.