ERROR:blocks.main_loop:Error occured during training. #1184
Comments
|
Please provide a minimal example to reproduce this issue. |
|
After loading data and when it starts to train, it gets wrong. ERROR:blocks.main_loop:Error occured during training. Blocks will attempt to run on_error extensions, potentially saving data, before exiting and reraising the error. Note that the usual after_training extensions will not be run. The original error will be re-raised and also stored in the training log. Press CTRL + C to halt Blocks immediately. |
|
It seems wrong about _run_extensions('before_training') |
Excuse me, I want to know how to solve the problem? If you can process it? Recently,I also have this problem, but I do not know how to do. |
I have tried the older blocks and it is still not running.Somebody please help!
Using gpu device 1: GeForce GTX TITAN X (CNMeM is enabled with initial size: 30.0% of memory, cuDNN 5105)
execute
Output will be stored in result
../data/CBTest/data
Computing new vocabulary for file ../data/CBTest/data/cbtest_CN_train.txt.
Processed line 100000
Processed line 200000
Processed line 300000
Processed line 400000
Processed line 500000
Processed line 600000
Processed line 700000
Processed line 800000
Processed line 900000
Processed line 1000000
Processed line 1100000
Processed line 1200000
Processed line 1300000
Processed line 1400000
Processed line 1500000
Processed line 1600000
Processed line 1700000
Processed line 1800000
Processed line 1900000
Processed line 2000000
Processed line 2100000
Processed line 2200000
Processed line 2300000
Processed line 2400000
Processed line 2500000
Processed line 2600000
STATISTICS
Total words: 64517417
Total distinct words: 60566
cbt
../data/CBTest/data
STATISTICS
Total words: 1019990
Total distinct words: 10552
Added 877 new words from file ../data/CBTest/data/cbtest_CN_valid_2000ex.txt to previous vocabulary.
cbt
../data/CBTest/data
STATISTICS
Total words: 1313107
Total distinct words: 12891
Added 1245 new words from file ../data/CBTest/data/cbtest_CN_test_2500ex.txt to previous vocabulary.
cbt
text_comprehension_base model
False
Parameters:
backward.state_to_state (128, 128).size=16384
backward.state_to_gates (128, 256).size=32768
backward.initial_state (128,).size=128
fork_gate_inputs.b (256,).size=256
fork_gate_inputs.W (1, 256).size=256
lookuptable.W (62692, 1).size=62692
fork_inputs.b (128,).size=128
fork_inputs.W (1, 128).size=128
forward.state_to_state (128, 128).size=16384
forward.state_to_gates (128, 256).size=32768
forward.initial_state (128,).size=128
fork_gate_inputs.b (256,).size=256
fork_gate_inputs.W (1, 256).size=256
fork_inputs.b (128,).size=128
fork_inputs.W (1, 128).size=128
backward.state_to_state (128, 128).size=16384
backward.state_to_gates (128, 256).size=32768
backward.initial_state (128,).size=128
fork_gate_inputs.b (256,).size=256
fork_gate_inputs.W (1, 256).size=256
fork_inputs.b (128,).size=128
fork_inputs.W (1, 128).size=128
forward.state_to_state (128, 128).size=16384
forward.state_to_gates (128, 256).size=32768
forward.initial_state (128,).size=128
fork_gate_inputs.b (256,).size=256
fork_gate_inputs.W (1, 256).size=256
fork_inputs.b (128,).size=128
fork_inputs.W (1, 128).size=128
Trained parameters count: 262884
BEFORE FIRST EPOCH
Training status:
batch_interrupt_received: False
epoch_interrupt_received: False
epoch_started: True
epochs_done: 0
iterations_done: 0
received_first_batch: False
resumed_from: None
training_started: True
Log records from the iteration 0:
time_initialization: 12.1652331352
ERROR:blocks.main_loop:Error occured during training.
Blocks will attempt to run
on_errorextensions, potentially saving data, before exiting and reraising the error. Note that the usualafter_trainingextensions will not be run. The original error will be re-raised and also stored in the training log. Press CTRL + C to halt Blocks immediately.Traceback (most recent call last):
File "/home/wh/eclipseworkspace/Theano/asreader-master/asreader/text_comprehension/as_reader.py", line 170, in
exp.execute()
File "/home/wh/eclipseworkspace/Theano/asreader-master/asreader/text_comprehension/text_comprehension_base.py", line 423, in execute
use_own_validation=True)
File "../learning_experiment_base.py", line 372, in train
main_loop.run()
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/main_loop.py", line 197, in run
reraise_as(e)
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/utils/init.py", line 258, in reraise_as
six.reraise(type(new_exc), new_exc, orig_exc_traceback)
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/main_loop.py", line 183, in run
while self._run_epoch():
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/main_loop.py", line 232, in _run_epoch
while self._run_iteration():
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/main_loop.py", line 251, in _run_iteration
self._run_extensions('before_batch', batch)
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/main_loop.py", line 263, in _run_extensions
extension.dispatch(CallbackName(method_name), *args)
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/extensions/init.py", line 67, in dispatch
getattr(self, str(callback_name))(*args)
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/extensions/init.py", line 501, in before_batch
self.bar = self.create_bar()
File "/usr/local/lib/python2.7/dist-packages/blocks-0.2.0-py2.7.egg/blocks/extensions/init.py", line 487, in create_bar
max_value=iter_per_epoch)
TypeError: init() got an unexpected keyword argument 'max_value'
Original exception:
TypeError: init() got an unexpected keyword argument 'max_value'
The text was updated successfully, but these errors were encountered: