Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

Last update: Dec 27, 2022

Related tags

Overview

SNN_Calibration

Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

Feature Comparison of SNN calibration:

Features	SNN Direct Training	ANN-SNN Conversion	SNN Calibration
Accuract (T<100)	High	Low	High
Scalability to ImageNet	Tiny	Large	Large
Training Speed	Slow	Fast	Fast
# Required Data	Full-set (1.2M For ImageNet)	~1000	~1000
Inference Speed	Fast	Slow	Fast

Requirements

Pytorch 1.8

For ImageNet experiments, please be sure that you can initialize distributed environments

For CIFAR experiments, one GPU would suffice.

Pre-training ANN on CIFAR10&100

Train an ANN model with main_train.py

python CIFAR/main_train.py --dataset CIFAR10 --arch VGG16 --dpath PATH/TO/DATA --usebn

Pre-trained results:

Dataset	Model	Random Seed	Accuracy
CIFAR10	VGG16	1000	95.76
CIFAR10	ResNet-20	1000	95.68
CIFAR100	VGG16	1000	77.98
CIFAR100	ResNet-20	1000	76.52

SNN Calibration on CIFAR10&100

Calibrate an SNN with main_calibration.py.

python CIFAR/main_calibration.py --dataset CIFAR10 --arch VGG16 --T 16 --usebn --calib advanced --dpath PATH/TO/DATA

--T is the time step, --calib is the calibration method, please use none, light, advanced for experiments.

The calibration will run for 5 times, and return the mean accuracy as well as the standard deviation.

Example results:

Architecture	Datset	T	Random Seed	Calibration	Mean Acc	Std.
VGG16	CIFAR10	16	1000	None	64.52	4.12
VGG16	CIFAR10	16	1000	Light	93.30	0.08
VGG16	CIFAR10	16	1000	Advanced	93.65	0.25
ResNet-20	CIFAR10	16	1000	None	67.88	3.63
ResNet-20	CIFAR10	16	1000	Light	93.89	0.20
ResNet-20	CIFAR10	16	1000	Advanced	94.33	0.12
VGG16	CIFAR100	16	1000	None	2.69	0.76
VGG16	CIFAR100	16	1000	Light	65.26	0.99
VGG16	CIFAR100	16	1000	Advanced	70.91	0.65
ResNet-20	CIFAR100	16	1000	None	39.27	2.85
ResNet-20	CIFAR100	16	1000	Light	73.89	0.15
ResNet-20	CIFAR100	16	1000	Advanced	74.48	0.16

Pre-training ANN on ImageNet

To be updaed

Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

Related tags

Overview

SNN_Calibration

Requirements

Pre-training ANN on CIFAR10&100

SNN Calibration on CIFAR10&100

Pre-training ANN on ImageNet

Owner

Yuhang Li

Official implementation of paper "Query2Label: A Simple Transformer Way to Multi-Label Classification".

Neural Scene Flow Fields using pytorch-lightning, with potential improvements

The fastest way to visualize GradCAM with your Keras models.

[NeurIPS 2021]: Are Transformers More Robust Than CNNs? (Pytorch implementation & checkpoints)

GAN encoders in PyTorch that could match PGGAN, StyleGAN v1/v2, and BigGAN. Code also integrates the implementation of these GANs.

Code for the paper "How Attentive are Graph Attention Networks?"

Rainbow DQN implementation that outperforms the paper's results on 40% of games using 20x less data 🌈

IOT: Instance-wise Layer Reordering for Transformer Structures

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

A Python library for Deep Probabilistic Modeling

Punctuation Restoration using Transformer Models for High-and Low-Resource Languages

PyTorch code for MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning

Tool for installing and updating MiSTer cores and other files

The AWS Certified SysOps Administrator

GBIM(Gesture-Based Interaction map)

The project is associated with the recently-launched ICASSP 2022 Multi-channel Multi-party Meeting Transcription Challenge (M2MeT) to provide participants with baseline systems for speech recognition and speaker diarization in conference scenario.

LLVM-based compiler for LightGBM gradient-boosted trees. Speeds up prediction by ≥10x.

Learning hidden low dimensional dyanmics using a Generalized Onsager Principle and neural networks

Simulated garment dataset for virtual try-on

Shared Attention for Multi-label Zero-shot Learning