get_minirocket_features Chunksize question

[YojoNick]

What does the chunksize argument do in get_minirocket_features?

Say I have a 650,000 samples, that are each 30 time steps with 4 dimensions per time step. If I want to calculate the MiniRocket Features for these samples, shouldn't I be able to pass them all at once to the get_minirocket_features function if I appropriately set the chunksize argument such that I'll have enough memory to process that chunk on the GPU?

What I'm noticing is even with a chunk size of 32, and 11 gigs of GPU memory, I'm getting out of memory errors when I run the below code.

Thanks in advance for any help you can offer with this :)

import os.path
import sys
import time
from tsai.all import *

trainFeat = np.random.rand(650000,4,30).astype('float32')

# Offline feature calculation
chunksize = 32
startFeatCalc = time.time()
mrf = MiniRocketFeatures(trainFeat.shape[1], trainFeat.shape[2]).to(default_device())
mrf.fit(trainFeat)
trainFeat = get_minirocket_features(trainFeat, mrf, chunksize=chunksize, to_np=True)

/usr/local/lib/python3.6/dist-packages/tsai/models/MINIROCKET_Pytorch.py in _get_PPVs(self, C, bias)
103 C = C.unsqueeze(-1)
104 bias = bias.view(1, bias.shape[0], 1, bias.shape[1])
--> 105 return (C > bias).float().mean(2).flatten(1)
106
107 def _set_dilations(self, input_length):

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 11.17 GiB total capacity; 10.67 GiB already allocated; 6.81 MiB free; 10.68 GiB reserved in total by PyTorch)

[YojoNick]

I'm not sure why I get a CUDA error, but even if I didn't get a CUDA error, I realize now I don't think I have enough CPU memory to hold all processed 650,000 samples. With 650,000 samples, each of them being a float32 at 4 bytes each, and each sample having 10,000 features that results in 650,000 samples x 10,000 features per sample x 4 bytes / 1e9 bytes in a gigabyte = 26 gigabytes. So it takes 26 gigs of CPU memory just to hold a single copy of the data, never mind any derivative memory consumption for processing it.

I guess the moral of the story here is be mindful of the number of samples you have when asking get_minirocket_features to calculate the features...

[oguiza]

Hi @YojoNick,
Sorry for my late reply.
I think your own reply is correct. Actually, for datasets with > 10_000 samples, the recommended approach is to use the online Pytorch version. This will avoid the OOM issue as features are only calculated for a single batch at a time.