Full Resolution Residual Networks for Semantic Image Segmentation

Related tags

Deep LearningFRRN
Overview

Full-Resolution Residual Networks (FRRN)

This repository contains code to train and qualitatively evaluate Full-Resolution Residual Networks (FRRNs) as described in

Tobias Pohlen, Alexander Hermans, Markus Mathias, Bastian Leibe: Full Resolution Residual Networks for Semantic Segmentation in Street Scenes. CVPR 2017.

A pre-print of the paper can be found on arXiv: arXiv:1611.08323.

Please cite the work as follows:

@inproceedings{pohlen2017FRRN,
  title={Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes},
  author={Pohlen, Tobias and Hermans, Alexander and Mathias, Markus and Leibe, Bastian},
  booktitle={Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on},
  year={2017}
}

Demo Video

Click here to watch our video.

Installation

Install the following software packages:

  • Python 2.7 or 3.4
  • Numpy
  • Scipy
  • Scikit-Learn
  • OpenCV
  • Theano
    • Scipy
    • Scikit-Learn
  • Lasagne

You may optionally install the following library for better performance.

You can check if all dependencies are installed correctly by running the check_dependencies.py script:

$ python check_dependencies.py --cs_folder=[Your CS folder]
2017-07-26 22:17:34,945 INFO Found supported Python version 3.4.
2017-07-26 22:17:35,122 INFO Successfully imported numpy.
2017-07-26 22:17:35,184 INFO Successfully imported cv2.
2017-07-26 22:17:35,666 INFO Successfully imported sklearn.
2017-07-26 22:17:35,691 INFO Successfully imported sklearn.metrics.
2017-07-26 22:17:35,691 INFO Successfully imported scipy.
Using cuDNN version 6021 on context None
Mapped name None to device cuda: TITAN X (Pascal) (0000:02:00.0)
2017-07-26 22:17:38,760 INFO Successfully imported theano.
2017-07-26 22:17:38,797 INFO Successfully imported lasagne.
2017-07-26 22:17:38,797 INFO Theano float is float32.
2017-07-26 22:17:38,803 INFO cuDNN spatial softmax found.
2017-07-26 22:17:38,807 INFO Use Chianti C++ library.
2017-07-26 22:17:38,826 INFO Found CityScapes training set.
2017-07-26 22:17:38,826 INFO Found CityScapes validation set.

If you don't see any ERROR messages, the software should run on your machine.

Qualitatively evaluation a pre-trained model

Run the script predict.py.

$ python predict.py --help
usage: predict.py [-h] --architecture {frrn_a,frrn_b} --model_file MODEL_FILE
                  --cs_folder CS_FOLDER [--sample_factor SAMPLE_FACTOR]

Shows the predictions of a Full-Resolution Residual Network on the Cityscapes
validation set.

optional arguments:
  -h, --help            show this help message and exit
  --architecture {frrn_a,frrn_b}
                        The network architecture type.
  --model_file MODEL_FILE
                        The model filename. Weights are initialized to the
                        given values if the file exists. Snapshots are stored
                        using a _snapshot_[iteration] post-fix.
  --cs_folder CS_FOLDER
                        The folder that contains the Cityscapes Dataset.
  --sample_factor SAMPLE_FACTOR
                        The sampling factor.

Train a new model

Run the train.py script.

$ python train.py --help
usage: train.py [-h] --architecture {frrn_a,frrn_b,frrn_c} --model_file
                MODEL_FILE --log_file LOG_FILE --cs_folder CS_FOLDER
                [--batch_size BATCH_SIZE]
                [--validation_interval VALIDATION_INTERVAL]
                [--iterator {uniform,weighted}] [--crop_size CROP_SIZE]
                [--learning_rate LEARNING_RATE]
                [--sample_factor SAMPLE_FACTOR]

Trains a Full-Resolution Residual Network on the Cityscapes Dataset.

optional arguments:
  -h, --help            show this help message and exit
  --architecture {frrn_a,frrn_b}
                        The network architecture type.
  --model_file MODEL_FILE
                        The model filename. Weights are initialized to the
                        given values if the file exists. Snapshots are stored
                        using a _snapshot_[iteration] post-fix.
  --log_file LOG_FILE   The log filename. Use log_monitor.py in order to
                        monitor training progress in the terminal.
  --cs_folder CS_FOLDER
                        The folder that contains the Cityscapes Dataset.
  --batch_size BATCH_SIZE
                        The batch size.
  --validation_interval VALIDATION_INTERVAL
                        The validation interval.
  --iterator {uniform,weighted}
                        The dataset iterator type.
  --crop_size CROP_SIZE
                        The size of crops to extract from the full-resolution
                        images. If 0, then now crops will be extracted.
  --learning_rate LEARNING_RATE
                        The learning rate to use.
  --sample_factor SAMPLE_FACTOR
                        The sampling factor.

Monitor training

Start a new notebook server and open training_monitor.ipynb.

License

See LICENSE (MIT).

Copyright

Copyright (c) 2017 Google Inc.

Copyright (c) 2017 Toby Pohlen

Owner
Toby Pohlen
Toby Pohlen
GitHub Repository
SigOpt wrappers for scikit-learn methods

SigOpt + scikit-learn Interfacing This package implements useful interfaces and wrappers for using SigOpt and scikit-learn together Getting Started In

SigOpt 73 Sep 30, 2022
Object detection, 3D detection, and pose estimation using center point detection:

Objects as Points Object detection, 3D detection, and pose estimation using center point detection: Objects as Points, Xingyi Zhou, Dequan Wang, Phili

Xingyi Zhou 6.7k Jan 03, 2023
Serving PyTorch 1.0 Models as a Web Server in C++

Serving PyTorch Models in C++ This repository contains various examples to perform inference using PyTorch C++ API. Run git clone https://github.com/W

Onur Kaplan 223 Jan 04, 2023
Code artifacts for the submission "Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems"

Code Artifacts Code artifacts for the submission "Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driv

Andrea Stocco 2 Aug 24, 2022
Package for extracting emotions from social media text. Tailored for financial data.

EmTract: Extracting Emotions from Social Media Text Tailored for Financial Contexts EmTract is a tool that extracts emotions from social media text. I

13 Nov 17, 2022
PyTorch Personal Trainer: My framework for deep learning experiments

Alex's PyTorch Personal Trainer (ptpt) (name subject to change) This repository contains my personal lightweight framework for deep learning projects

Alex McKinney 8 Jul 14, 2022
📖 Deep Attentional Guided Image Filtering

📖 Deep Attentional Guided Image Filtering [Paper] Zhiwei Zhong, Xianming Liu, Junjun Jiang, Debin Zhao ,Xiangyang Ji Harbin Institute of Technology,

9 Dec 23, 2022
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation This repository contains the source code of our paper, ESPNet (acc

Sachin Mehta 515 Dec 13, 2022
EEGEyeNet is benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty

Introduction EEGEyeNet EEGEyeNet is a benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty. Overview T

Ard Kastrati 23 Dec 22, 2022
Deep Convolutional Generative Adversarial Networks

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Alec Radford, Luke Metz, Soumith Chintala All images in t

Alec Radford 3.4k Dec 29, 2022
A unified framework to jointly model images, text, and human attention traces.

connect-caption-and-trace This repository contains the reference code for our paper Connecting What to Say With Where to Look by Modeling Human Attent

Meta Research 73 Oct 24, 2022
Implementation of the ivis algorithm as described in the paper Structure-preserving visualisation of high dimensional single-cell datasets.

Implementation of the ivis algorithm as described in the paper Structure-preserving visualisation of high dimensional single-cell datasets.

beringresearch 285 Jan 04, 2023
Release of SPLASH: Dataset for semantic parse correction with natural language feedback in the context of text-to-SQL parsing

SPLASH: Semantic Parsing with Language Assistance from Humans SPLASH is dataset for the task of semantic parse correction with natural language feedba

Microsoft Research - Language and Information Technologies (MSR LIT) 35 Oct 31, 2022
Minimal deep learning library written from scratch in Python, using NumPy/CuPy.

SmallPebble Project status: experimental, unstable. SmallPebble is a minimal/toy automatic differentiation/deep learning library written from scratch

Sidney Radcliffe 92 Dec 30, 2022
Virtual Dance Reality Stage: a feature that offers you to share a stage with another user virtually

Portrait Segmentation using Tensorflow This script removes the background from an input image. You can read more about segmentation here Setup The scr

291 Dec 24, 2022
official code for dynamic convolution decomposition

Revisiting Dynamic Convolution via Matrix Decomposition (ICLR 2021) A pytorch implementation of DCD. If you use this code in your research please cons

Yunsheng Li 110 Nov 23, 2022
The Easy-to-use Dialogue Response Selection Toolkit for Researchers

Easy-to-use toolkit for retrieval-based Chatbot Recent Activity Our released RRS corpus can be found here. Our released BERT-FP post-training checkpoi

GMFTBY 32 Nov 13, 2022
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model a

Google 3.2k Dec 31, 2022
Using NumPy to solve the equations of fluid mechanics together with Finite Differences, explicit time stepping and Chorin's Projection methods

Computational Fluid Dynamics in Python Using NumPy to solve the equations of fluid mechanics 🌊 🌊 🌊 together with Finite Differences, explicit time

Felix Köhler 4 Nov 12, 2022
A transformer model to predict pathogenic mutations

MutFormer MutFormer is an application of the BERT (Bidirectional Encoder Representations from Transformers) NLP (Natural Language Processing) model wi

Wang Genomics Lab 2 Nov 29, 2022