💫 Make TextCategorizer default to a simpler, GPU-friendly model

[honnibal]

Currently the TextCategorizer defaults to a fairly complicated model, designed partly around the active learning requirements of Prodigy. The model's a bit slow, and not very GPU-friendly.

This patch implements a straightforward CNN model that still performs pretty well. The replacement model also makes it easy to use the LMAO pretraining, since most of the parameters are in the CNN.

The replacement model has a flag to specify whether labels are mutually exclusive, which defaults to True. This has been a common problem with the text classifier. We'll also now be able to support adding labels to pretrained models again.

Resolves #2934, #2756, #1798, #1748.

[luoy2]

may I ask how should enable gpu computation based on this merge now? I changed the _ml.py, pipline.py as the commit, and added spacy.prefer_gpu() on top of the textcat example code. still getting

Traceback (most recent call last):
  File "D:/SkyDrive/Documents/UIUC CS/CS 410 Text Information Systems/UIUC-DS410-TwitterStockPrediction/tweetsClassifier/spacyTrainer.py", line 173, in <module>
    plac.call(main)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\plac_core.py", line 328, in call
    cmd, result = parser.consume(arglist)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "D:/SkyDrive/Documents/UIUC CS/CS 410 Text Information Systems/UIUC-DS410-TwitterStockPrediction/tweetsClassifier/spacyTrainer.py", line 84, in main
    losses=losses)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\spacy\language.py", line 421, in update
    proc.update(docs, golds, drop=drop, sgd=get_grads, losses=losses)
  File "pipeline.pyx", line 876, in spacy.pipeline.TextCategorizer.update
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\thinc\api.py", line 61, in begin_update
    X, inc_layer_grad = layer.begin_update(X, drop=drop)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\thinc\api.py", line 176, in begin_update
    values = [fwd(X, *a, **k) for fwd in forward]
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\thinc\api.py", line 176, in <listcomp>
    values = [fwd(X, *a, **k) for fwd in forward]
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\thinc\api.py", line 258, in wrap
    output = func(*args, **kwargs)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\thinc\api.py", line 61, in begin_update
    X, inc_layer_grad = layer.begin_update(X, drop=drop)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\spacy\_ml.py", line 128, in _preprocess_doc
    keys = ops.xp.concatenate(keys)
  File "D:\SkyDrive\Documents\UIUC CS\CS 410 Text Information Systems\UIUC-DS410-TwitterStockPrediction\env\lib\site-packages\cupy\manipulation\join.py", line 49, in concatenate
    return core.concatenate_method(tup, axis)
  File "cupy\core\core.pyx", line 2797, in cupy.core.core.concatenate_method
  File "cupy\core\core.pyx", line 2810, in cupy.core.core.concatenate_method
TypeError: Only cupy arrays can be concatenated

Should I wait for the merge to master?

Thanks!

• spaCy version: 2.0.18
• Platform: Windows10
• Python version: 3.6.5
• Models: en
• thinc 6.12.1
• cupy-cuda92 5.1.0
• Nvidia build version 417.35