Train EntityLinker on GPU

[Joozty]

How to reproduce the behaviour

When I try to run train nel-emerson from tutorials on GPU it fails with this error. I noticed that you added support for GPU training some time ago. So I guess it's just a bug?

Running command: /home/joozty/Documents/projects/tutorials/nel_emerson/venv/bin/python -m spacy train configs/nel.cfg --output training --paths.train corpus/train.spacy --paths.dev corpus/dev.spacy --paths.kb temp/my_kb --paths.base_nlp temp/my_nlp -c scripts/custom_functions.py -g 0
ℹ Using GPU: 0

=========================== Initializing pipeline ===========================
Set up nlp object from config
Pipeline: ['sentencizer', 'entity_ruler', 'entity_linker']
Created vocabulary
Finished initializing nlp object
Initialized pipeline components: ['entity_linker']
✔ Initialized pipeline

============================= Training pipeline =============================
ℹ Pipeline: ['sentencizer', 'entity_ruler', 'entity_linker']
ℹ Frozen components: ['sentencizer', 'entity_ruler']
ℹ Initial learn rate: 0.001
E    #       LOSS ENTIT...  SENTS_F  SENTS_P  SENTS_R  ENTS_F  ENTS_P  ENTS_R  NEL_MICRO_F  NEL_MICRO_R  NEL_MICRO_P  SCORE 
---  ------  -------------  -------  -------  -------  ------  ------  ------  -----------  -----------  -----------  ------
  0       0           2.85   100.00   100.00   100.00   16.67   16.67   16.67        33.33        33.33        33.33    0.49
⚠ Aborting and saving the final best model. Encountered exception:
TypeError('array(2.8528614, dtype=float32) is not JSON serializable')
Traceback (most recent call last):
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 114, in train
    raise e
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 104, in train
    save_checkpoint(is_best_checkpoint)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 67, in save_checkpoint
    before_to_disk(nlp).to_disk(output_path / DIR_MODEL_LAST)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/language.py", line 1662, in to_disk
    util.to_disk(path, serializers, exclude)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/util.py", line 1127, in to_disk
    writer(path / key)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/language.py", line 1653, in <lambda>
    serializers["meta.json"] = lambda p: srsly.write_json(p, self.meta)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/srsly/_json_api.py", line 72, in write_json
    json_data = json_dumps(data, indent=indent)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/srsly/_json_api.py", line 26, in json_dumps
    result = ujson.dumps(data, indent=indent, escape_forward_slashes=False)
TypeError: array(2.8528614, dtype=float32) is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/cli/_util.py", line 65, in setup_cli
    command(prog_name=COMMAND)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/cli/train.py", line 59, in train_cli
    train(nlp, output_path, use_gpu=use_gpu, stdout=sys.stdout, stderr=sys.stderr)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 118, in train
    save_checkpoint(False)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 67, in save_checkpoint
    before_to_disk(nlp).to_disk(output_path / DIR_MODEL_LAST)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/language.py", line 1662, in to_disk
    util.to_disk(path, serializers, exclude)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/util.py", line 1127, in to_disk
    writer(path / key)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/language.py", line 1653, in <lambda>
    serializers["meta.json"] = lambda p: srsly.write_json(p, self.meta)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/srsly/_json_api.py", line 72, in write_json
    json_data = json_dumps(data, indent=indent)
  File "/home/joozty/Documents/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/srsly/_json_api.py", line 26, in json_dumps
    result = ujson.dumps(data, indent=indent, escape_forward_slashes=False)
TypeError: array(2.8528614, dtype=float32) is not JSON serializable

Your Environment

• Operating System: Ubuntu 20.04
• Python Version Used: 3.8
• spaCy Version Used: spacy-nightly[cuda101]==3.0.0rc5 with pytorch

Many thanks :)

[svlandeg]

Thanks for the report! It turned out there was a small bug that should be fixed by PR #6928. It'll be included in the next small release, I'm guessing 3.0.2.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.