models

This repository provides a modular and flexible implementation of general deep sequence models.

baselines/   Ported baseline models
functional/  Mathematical utilities
hippo/       Utilities for defining HiPPO operators
nn/          Standalone neural network components (nn.Module)
s4/          Standalone S4 modules
sequence/    Modular sequence model interface

S4

In v3, a standalone implementation of S4 could be found inside s4/. It has been moved to /models/s4/. The fully tested S4 implementation is inside sequence/.

Modular Sequence Model Interface

A general deep sequence model framework can be found in sequence/. All models and experiments that this repository official supports used this framework. See sequence/README.md for more information.

Baselines

Other sequence models are easily incorporated into this repository, and several other baselines have been ported. These include CNNs such as CKConv and continuous-time/RNN models such as UnICORNN and LipschitzRNN.

Models and datasets can be flexibly interchanged. Examples:

python -m train pipeline=cifar model=ckconv
python -m train pipeline=mnist model=lipschitzrnn

The distinction between baselines in baselines/ and models in sequence/ is that the baselines do not necessarily subscribe to the modular SequenceModule interface, and are usually monolithic end-to-end models adapted from other codebases.