Installation¶

Install the project¶

This project has provided an environment setting file of conda, users can easily reproduce the environment by the following commands:

git clone https://github.com/chengtan9907/OpenSTL
cd OpenSTL
conda env create -f environment.yml
conda activate OpenSTL
python setup.py develop  # or `pip install -e .`

Requirements

Linux (Windows is not officially supported)
Python 3.7+
PyTorch 1.8 or higher
CUDA 10.1 or higher
NCCL 2
GCC 4.9 or higher

Dependencies

argparse
fvcore
numpy
hickle
scikit-image
scikit-learn
torch
timm
tqdm
xarray==0.19.0

Note:

Installation errors.
- If you are installing cv2 for the first time, ImportError: libGL.so.1 will occur, which can be solved by apt install libgl1-mesa-glx.
- Errors might occur with hickle and this dependency when using KittiCaltech dataset. You can solve the issues by installing additional packages according to the output message.
- As for WeatherBench, you encounter some import or runtime errors in the version of xarray. You can install the latest version or xarray==0.19.0 to solve the errors, i.e., pip install xarray==0.19.0, and then install required packages according to error messages.
Following the above instructions, OpenSTL is installed on dev mode, any local modifications made to the code will take effect. You can install it by pip install . to use it as a PyPi package, and you should reinstall it to make the local modifications effect.

Prepare datasets¶

It is recommended to symlink your dataset root (assuming $YOUR_DATA_ROOT) to $OPENSTL/data. If your folder structure is different, you need to change the corresponding paths in config files.

We support following datasets: Human3.6M [download], KTH Action [download], KittiCaltech Pedestrian [download], Moving MNIST [download], TaxiBJ [download], WeatherBench [download]. Please prepare datasets with tools and scripts under tools/prepare_data. You can also download the version we used in experiments from Baidu Cloud (kjfk). Please do not distribute the datasets and only use them for research. After all, the related datasets under $OPENSTL/data will look like this:

OpenSTL
├── configs
└── data
    ├── caltech
    │   ├── set06
    │   ├── ...
    │   ├── set10
    │   ├── data_cache.npy
    │   ├── indices_cache.npy
    ├── human
    |   ├── images
    |   ├── test.txt
    |   ├── train.txt
    |── kitti_hkl
    |   ├── sources_test_mini.hkl
    |   ├── ...
    |   ├── X_train.hkl
    │   ├── X_val.hkl
    |── kth
    |   ├── boxing
    |   ├── ...
    |   ├── walking
    |── moving_fmnist
    |   ├── fmnist_test_seq.npy
    |   ├── train-images-idx3-ubyte.gz
    |── moving_mnist
    |   ├── mnist_test_seq.npy
    |   ├── train-images-idx3-ubyte.gz
    |── taxibj
    |   ├── dataset.npz
    |── weather
    |   ├── 2m_temperature
    |   ├── ...
    |── weather_1_40625deg
    |   ├── 2m_temperature
    |   ├── ...

Moving MNIST / FMNIST¶

Moving MNIST and Moving FMNIST are toy datasets, which generate gray-scale videos (64x64 resolutions) with two objects. We provide download_mmnist.sh and download_mfmnist.sh, which download datasets from MMNIST download and MFMNIST download. Note that the train set is generated online while the test set is fixed to ensure the consistency of evaluation results.

KittiCaltech Pedestrian¶

The KittiCaltech Pedestrian dataset uses Kitti Pedestrian as the train set (2042 videos) and uses Caltech Pedestrian as the test set (1983 videos). We provide download_kitticaltech.sh to prepare the datasets. The data preprocessing of RGB videos (128x160 resolutions) and experiment settings are adopted from PredNet.

KTH Action¶

The KTH Action dataset contains grey-scale videos (128x128 resolutions) of six types of human actions performed several times by 25 subjects in four different scenarios. It has 5200 and 3167 videos for the train and test sets and can be downloaded from KTH download. We provide download_kth.sh to prepare the dataset. The data preprocessing and experiment settings are adopted from KTH and PredRNN.

Human 3.6M¶

The Human3.6M dataset contains high-resolution videos (1024x1024 resolutions) of seventeen scenarios of human actions performed by eleven professional actors, which can be downloaded from Human3.6M download. We provide download_human3.6m.sh to prepare the dataset. We borrow the train and test splitting files from STRPM but use 256x256 resolutions in our experiments.

WeatherBench¶

WeatherBench is the publicly available dataset for global weather prediction, which can be downloaded and processed from WeatherBench download. We choose some important weather variants with certain vertical levels and resolutions, e.g., 2m_temperature, relative_humidity, and total_cloud_cover. You can download the specific dataset of WeatherBench with download_weatherbench.sh. Note that 5.625deg and 1.40625deg indicate 32x64 and 128x256 resolutions, and the data can have multiple channels.

TaxiBJ¶

TaxiBJ is a popular traffic trajectory prediction dataset, which contains the trajectory data (32x32) in Beijing collected from taxicab GPS with two channels, which can be downloaded from OneDrive or Baidu Cloud. We borrow the data preprocessing scripts from DeepST and provide the processed data in our Baidu Cloud.