Official implementation of Densely connected normalizing flows

Last update: Dec 12, 2022

Related tags

Overview

Densely connected normalizing flows

This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster available here.

Setup

CUDA 11.1
Python 3.8

pip install -r requirements.txt
pip install -e .

Training

cd ./experiments/image

CIFAR-10:

python train.py --epochs 400 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.9975 --warmup 5000  --eval_every 1 --check_every 10 --dataset cifar10 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/cifar10_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 420

ImageNet32:

python train.py --epochs 20 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet32 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet32_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 22

ImageNet64:

python train.py --epochs 10 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet64 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet64_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 11

CelebA:

python train.py --epochs 50 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset celeba --augmentation horizontal_flip --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/celeba_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 55

Note: Download instructions for ImageNet and CelebA are defined in denseflow/data/datasets/image/{dataset}.py

Evaluation

CIFAR-10:

python eval_loglik.py --model PATH_TO_MODEL --k 1000 --kbs 50

ImageNet32:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 50

ImageNet64 and CelebA:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 25

Model weights

Model weights are stored here.

Samples generation

Generated samples are stored in PATH_TO_MODEL/samples

python eval_sample.py --model PATH_TO_MODEL

Note: PATH_TO_MODEL has to contain check directory.

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

Significant part of this code benefited from SurVAE [1] code implementation, available under MIT license.

References

[1] Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows. InAdvances in Neural Information Processing Systems 33. Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.

Official implementation of Densely connected normalizing flows

Related tags

Overview

Densely connected normalizing flows

Setup

Training

Evaluation

Model weights

Samples generation

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

References

Owner

Matej Grcić

Original Pytorch Implementation of FLAME: Facial Landmark Heatmap Activated Multimodal Gaze Estimation

Repository For Programmers Seeking a platform to show their skills

Source code and notebooks to reproduce experiments and benchmarks on Bias Faces in the Wild (BFW).

Recurrent Scale Approximation (RSA) for Object Detection

A PyTorch implementation of "DGC-Net: Dense Geometric Correspondence Network"

The Balloon Learning Environment - flying stratospheric balloons with deep reinforcement learning.

A PyTorch-based open-source framework that provides methods for improving the weakly annotated data and allows researchers to efficiently develop and compare their own methods.

SSD: A Unified Framework for Self-Supervised Outlier Detection [ICLR 2021]

Official code implementation for "Personalized Federated Learning using Hypernetworks"

DeepRec is a recommendation engine based on TensorFlow.

Gym environments used in the paper: "Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV with Thrust Vectoring Rotors"

Code repository for paper `Skeleton Merger: an Unsupervised Aligned Keypoint Detector`.

Label Studio is a multi-type data labeling and annotation tool with standardized output format

This code implements constituency parse tree aggregation

Pytorch Lightning Distributed Accelerators using Ray

Source code for paper "Deep Diffusion Models for Robust Channel Estimation", TBA.

docTR by Mindee (Document Text Recognition) - a seamless, high-performing & accessible library for OCR-related tasks powered by Deep Learning.

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Paddle implementation for "Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation" (NAACL 2021)

Gradient-free global optimization algorithm for multidimensional functions based on the low rank tensor train format