RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

Research into Forex price prediction from price history using Deep Sequence Modeling with Stacked LSTMs.

Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [2021]

Libraries, tools and tasks created and used at DeepMind Robotics.

Beginner-friendly repository for Hacktober Fest 2021. Start your contribution to open source through baby steps. 💜

Official repository for "On Generating Transferable Targeted Perturbations" (ICCV 2021)

Making a music video with Wav2CLIP and VQGAN-CLIP

Official pytorch implementation of Active Learning for deep object detection via probabilistic modeling (ICCV 2021)

CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation

A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

RuleBERT: Teaching Soft Rules to Pre-Trained Language Models

The implementation code for "DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction"

Code for reproducing experiments in "Improved Training of Wasserstein GANs"

PyTorch code for the paper "Complementarity is the King: Multi-modal and Multi-grained Hierarchical Semantic Enhancement Network for Cross-modal Retrieval".

Deep-Learning-Book-Chapter-Summaries - Attempting to make the Deep Learning Book easier to understand.

PyTorch code for 'Efficient Single Image Super-Resolution Using Dual Path Connections with Multiple Scale Learning'

A Python Package for Convex Regression and Frontier Estimation

Video Swin Transformer - PyTorch

[ICCV2021] Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving

[ACM MM 2021] TSA-Net: Tube Self-Attention Network for Action Quality Assessment

Robust, modular and efficient implementation of advanced Hamiltonian Monte Carlo algorithms