BasicLFSR

BasicLFSR is an open-source and easy-to-use Light Field (LF) image Super-Ressolution (SR) toolbox based on PyTorch, including a collection of papers on LF image SR and a benchmark to comprehensively evaluate the performance of existing methods. We also provided simple pipelines to train/valid/test state-of-the-art methods to get started quickly, and you can transform your methods into the benchmark.

Note: This repository will be updated on a regular basis, and the pretrained models of existing methods will be open-sourced one after another. So stay tuned!

Methods

Datasets

We used the EPFL, HCInew, HCIold, INRIA and STFgantry datasets for both training and test. Please first download our datasets via Baidu Drive (key:7nzy) or OneDrive, and place the 5 datasets to the folder ./datasets/.

• After downloading, you should find following structure:

  ├──./datasets/
  │    ├── EPFL
  │    │    ├── training
  │    │    │    ├── Bench_in_Paris.mat
  │    │    │    ├── Billboards.mat
  │    │    │    ├── ...
  │    │    ├── test
  │    │    │    ├── Bikes.mat
  │    │    │    ├── Books__Decoded.mat
  │    │    │    ├── ...
  │    ├── HCI_new
  │    ├── ...
  

• Run Generate_Data_for_Training.m to generate training data. The generated data will be saved in ./data_for_train/ (SR_5x5_2x, SR_5x5_4x).

• Run Generate_Data_for_Test.m to generate test data. The generated data will be saved in ./data_for_test/ (SR_5x5_2x, SR_5x5_4x).

Benchmark

We benchmark several methods on above datasets, and PSNR and SSIM metrics are used for quantitative evaluation.

PSNR and SSIM values achieved by different methods for 2xSR:

PSNR and SSIM values achieved by different methods for 4xSR:

Train

• Run train.py to perform network training. Example for training [model_name] on 5x5 angular resolution for 2x/4x SR:

  $ python train.py --model_name [model_name] --angRes 5 --scale_factor 2 --batch_size 8
  $ python train.py --model_name [model_name] --angRes 5 --scale_factor 4 --batch_size 4
  

• Checkpoints and Logs will be saved to ./log/, and the ./log/ has following structure:

  ├──./log/
  │    ├── SR_5x5_2x
  │    │    ├── [dataset_name]
  │    │         ├── [model_name]
  │    │         │    ├── [model_name]_log.txt
  │    │         │    ├── checkpoints
  │    │         │    │    ├── [model_name]_5x5_2x_epoch_01_model.pth
  │    │         │    │    ├── [model_name]_5x5_2x_epoch_02_model.pth
  │    │         │    │    ├── ...
  │    │         │    ├── results
  │    │         │    │    ├── VAL_epoch_01
  │    │         │    │    ├── VAL_epoch_02
  │    │         │    │    ├── ...
  │    │         ├── [other_model_name]
  │    │         ├── ...
  │    ├── SR_5x5_4x
  

Test

• Run test.py to perform network inference. Example for test [model_name] on 5x5 angular resolution for 2x/4xSR:

  $ python test.py --model_name [model_name] --angRes 5 --scale_factor 2  
  $ python test.py --model_name [model_name] --angRes 5 --scale_factor 4 
  

• The PSNR and SSIM values of each dataset will be saved to ./log/, and the ./log/ is following structure:

  ├──./log/
  │    ├── SR_5x5_2x
  │    │    ├── [dataset_name]
  │    │        ├── [model_name]
  │    │        │    ├── [model_name]_log.txt
  │    │        │    ├── checkpoints
  │    │        │    │   ├── ...
  │    │        │    ├── results
  │    │        │    │    ├── Test
  │    │        │    │    │    ├── evaluation.xls
  │    │        │    │    │    ├── [dataset_1_name]
  │    │        │    │    │    │    ├── [scene_1_name]
  │    │        │    │    │    │    │    ├── [scene_1_name]_CenterView.bmp
  │    │        │    │    │    │    │    ├── [scene_1_name]_SAI.bmp
  │    │        │    │    │    │    │    ├── views
  │    │        │    │    │    │    │    │    ├── [scene_1_name]_0_0.bmp
  │    │        │    │    │    │    │    │    ├── [scene_1_name]_0_1.bmp
  │    │        │    │    │    │    │    │    ├── ...
  │    │        │    │    │    │    │    │    ├── [scene_1_name]_4_4.bmp
  │    │        │    │    │    │    ├── [scene_2_name]
  │    │        │    │    │    │    ├── ...
  │    │        │    │    │    ├── [dataset_2_name]
  │    │        │    │    │    ├── ...
  │    │        │    │    ├── VAL_epoch_01
  │    │        │    │    ├── ...
  │    │        ├── [other_model_name]
  │    │        ├── ...
  │    ├── SR_5x5_4x
  

Recources

We provide some original super-resolved images and useful resources to facilitate researchers to reproduce the above results.

Other Recources

• YapengTian/Single-Image-Super-Resolution
• LoSealL/VideoSuperResolution
• ChaofWang/Awesome-Super-Resolution
• ptkin/Awesome-Super-Resolution
• lightfield-analysis/resources
• Joechann0831/LFSRBenchmark
• YingqianWang/LF-Image-SR

Contact

Any question regarding this work can be addressed to [email protected].