code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

Related tags

Deep LearningMMNet
Overview

MMNet

This repo is the official implementation of ICCV 2021 paper "Multi-scale Matching Networks for Semantic Correspondence.".

Pre-requisite

conda create -n mmnet python==3.8.0
conda activate mmnet
conda install torch==1.8.1 torchvision==0.9.1
pip install matplotlib scikit-image pandas

for installation of gluoncvth (fcn-resnet101):

git clone https://github.com/StacyYang/gluoncv-torch.git
cd gluoncv-torch
python setup.py install

Reproduction

for test

Trained models are available on [google drive].

pascal with fcn-resnet101 backbone([email protected]:81.6%):

python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name path\to\ckp_pascal_fcnres101.pth --resize 224,320

spair with fcn-resnet101 backbone([email protected]:46.6%):

python test.py --alpha 0.05 --benchmark spair --backbone fcn-resnet101 --ckp_name path\to\ckp_spair_fcnres101.pth --resize 224,320

Bibtex

If you use this code for your research, please consider citing:

@article{zhao2021multi,
  title={Multi-scale Matching Networks for Semantic Correspondence},
  author={Zhao, Dongyang and Song, Ziyang and Ji, Zhenghao and Zhao, Gangming and Ge, Weifeng and Yu, Yizhou},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021}
}
You might also like...
A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

null 69 Dec 10, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 72 Dec 14, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 26 Sep 26, 2021
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
A PyTorch implementation of
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

Jiaqi Gu 2 Jan 4, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
《Dual-Resolution Correspondence Network》(NeurIPS 2020)
《Dual-Resolution Correspondence Network》(NeurIPS 2020)

Dual-Resolution Correspondence Network Dual-Resolution Correspondence Network, NeurIPS 2020 Dependency All dependencies are included in asset/dualrcne

Active Vision Laboratory 45 Nov 21, 2022
Comments
  • NaN during training

    NaN during training

    Hi, congrats on your paper! I was trying to run your training code (with resnet 101 on pf-pascal) but directly after a couple of iterations, nan appear in the input. Have you ever seen this issue? Thanks

    opened by PruneTruong 2
  • In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    Hello,this paper is very nice,i am very love it. I read your code,in Model.py, def calLayer1(self, feats): sum1 = self.conv1_1_down(self.msblock1_1(feats[1])) +
    self.conv1_2_down(self.msblock1_2(feats[2])) +
    self.conv1_3_down(self.msblock1_3(feats[3])) sum1 = self.wa_1(sum1) return sum1 I do not find where are these operation,self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1,so where are these ,in which document.Thank you,looking forward to your reply.

    opened by liang532 1
  • How to prepare the PF-Pascal dataset?

    How to prepare the PF-Pascal dataset?

    I downloaded the PF-dataset-Pascal.zip from the Proposal Flow paper's web page, extracted it, and run the next line of command, but get errors about missing data files.

    Input:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_pascal.pth --resize 224,320

    Expected output: some results about the benchmark results.

    Actual output:

    currently executing test.py file.
2021-11-19 02:01:59,172 - INFO - Options listed below:----------------
2021-11-19 02:01:59,172 - INFO - name: framework_train
2021-11-19 02:01:59,172 - INFO - benchmark: pfpascal
2021-11-19 02:01:59,172 - INFO - thresh_type: auto
2021-11-19 02:01:59,172 - INFO - backbone_name: fcn-resnet101
2021-11-19 02:01:59,172 - INFO - ms_rate: 4
2021-11-19 02:01:59,173 - INFO - feature_channel: 21
2021-11-19 02:01:59,173 - INFO - batch: 5
2021-11-19 02:01:59,173 - INFO - gpu: 0
2021-11-19 02:01:59,173 - INFO - data_path: /data/SC_Dataset
2021-11-19 02:01:59,173 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 02:01:59,173 - INFO - visualization_path: visualization
2021-11-19 02:01:59,173 - INFO - model_type: MMNet
2021-11-19 02:01:59,173 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_pascal.pth
2021-11-19 02:01:59,173 - INFO - log_path: ./logs/
2021-11-19 02:01:59,173 - INFO - resize: 224,320
2021-11-19 02:01:59,173 - INFO - max_kps_num: 50
2021-11-19 02:01:59,173 - INFO - split_type: test
2021-11-19 02:01:59,173 - INFO - alpha: 0.05
2021-11-19 02:01:59,173 - INFO - resolution: 2
2021-11-19 02:01:59,173 - INFO - Options all listed.------------------
2021-11-19 02:01:59,173 - INFO - ckp file: assets/model/mmnet_fcnresnet101_pascal.pth
Traceback (most recent call last):
  File "/home/runner/MMNet/test.py", line 127, in <module>
    test(logger, options)
  File "/home/runner/MMNet/test.py", line 65, in test
    test_dataset = Dataset.CorrespondenceDataset(
  File "/home/runner/MMNet/data/PascalDataset.py", line 32, in __init__
    self.train_data = pd.read_csv(self.spt_path)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/common.py", line 702, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/data/SC_Dataset/PF-PASCAL/test_pairs.csv'

    P.S. Output of executing ls /data/SC_Dataset/PF-PASCAL/:

    Annotations  html  index.html  JPEGImages  parsePascalVOC.mat  ShowMatchingPairs
    opened by tjyuyao 2
  • How to reproduce the reported test accuracy?

    How to reproduce the reported test accuracy?

    By running given following command with code on the main branch:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_spair.pth --resize 224,320 --benchmark spair

    I expect to get the reported accuracy in the Table.2 of paper, i.e. 50.4 "all" accuracy, or spair with fcn-resnet101 backbone([email protected]:46.6%): as noted in the README.md file. However I get the following output, finding nowhere the related results. Can you point out the steps to reproduce the test accuracy?

    2021-11-19 00:49:54,452 - INFO - Options listed below:----------------
2021-11-19 00:49:54,452 - INFO - name: framework_train
2021-11-19 00:49:54,453 - INFO - benchmark: spair
2021-11-19 00:49:54,453 - INFO - thresh_type: auto
2021-11-19 00:49:54,454 - INFO - backbone_name: fcn-resnet101
2021-11-19 00:49:54,455 - INFO - ms_rate: 4
2021-11-19 00:49:54,455 - INFO - feature_channel: 21
2021-11-19 00:49:54,456 - INFO - batch: 5
2021-11-19 00:49:54,456 - INFO - gpu: 0
2021-11-19 00:49:54,457 - INFO - data_path: /data/SC_Dataset
2021-11-19 00:49:54,457 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 00:49:54,458 - INFO - visualization_path: visualization
2021-11-19 00:49:54,458 - INFO - model_type: MMNet
2021-11-19 00:49:54,459 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:49:54,459 - INFO - log_path: ./logs/
2021-11-19 00:49:54,460 - INFO - resize: 224,320
2021-11-19 00:49:54,460 - INFO - max_kps_num: 50
2021-11-19 00:49:54,461 - INFO - split_type: test
2021-11-19 00:49:54,461 - INFO - alpha: 0.05
2021-11-19 00:49:54,462 - INFO - resolution: 2
2021-11-19 00:49:54,462 - INFO - Options all listed.------------------
2021-11-19 00:49:54,463 - INFO - ckp file: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:50:04,950 - INFO - [    0/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] aeroplane
2021-11-19 00:50:04,953 - INFO - [    1/12234]: 	 [Pair PCK: 0.100]	[Average: 0.217] aeroplane
2021-11-19 00:50:04,956 - INFO - [    2/12234]: 	 [Pair PCK: 0.308]	[Average: 0.247] aeroplane
2021-11-19 00:50:04,958 - INFO - [    3/12234]: 	 [Pair PCK: 0.364]	[Average: 0.276] aeroplane
2021-11-19 00:50:04,960 - INFO - [    4/12234]: 	 [Pair PCK: 0.000]	[Average: 0.221] aeroplane
2021-11-19 00:50:05,575 - INFO - [    5/12234]: 	 [Pair PCK: 0.200]	[Average: 0.217] aeroplane
2021-11-19 00:50:05,577 - INFO - [    6/12234]: 	 [Pair PCK: 0.250]	[Average: 0.222] aeroplane
2021-11-19 00:50:05,580 - INFO - [    7/12234]: 	 [Pair PCK: 0.308]	[Average: 0.233] aeroplane
2021-11-19 00:50:05,583 - INFO - [    8/12234]: 	 [Pair PCK: 0.182]	[Average: 0.227] aeroplane
2021-11-19 00:50:05,585 - INFO - [    9/12234]: 	 [Pair PCK: 0.636]	[Average: 0.268] aeroplane
2021-11-19 00:50:06,153 - INFO - [   10/12234]: 	 [Pair PCK: 0.667]	[Average: 0.304] aeroplane
2021-11-19 00:50:06,156 - INFO - [   11/12234]: 	 [Pair PCK: 0.385]	[Average: 0.311] aeroplane
2021-11-19 00:50:06,158 - INFO - [   12/12234]: 	 [Pair PCK: 0.455]	[Average: 0.322] aeroplane
2021-11-19 00:50:06,160 - INFO - [   13/12234]: 	 [Pair PCK: 0.250]	[Average: 0.317] aeroplane
2021-11-19 00:50:06,163 - INFO - [   14/12234]: 	 [Pair PCK: 0.615]	[Average: 0.337] aeroplane
2021-11-19 00:50:06,731 - INFO - [   15/12234]: 	 [Pair PCK: 0.000]	[Average: 0.316] aeroplane
...
2021-11-19 01:13:47,264 - INFO - [12216/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,265 - INFO - [12217/12234]: 	 [Pair PCK: 0.200]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,266 - INFO - [12218/12234]: 	 [Pair PCK: 0.250]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,268 - INFO - [12219/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,837 - INFO - [12220/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,838 - INFO - [12221/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,848 - INFO - [12222/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,850 - INFO - [12223/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,853 - INFO - [12224/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,422 - INFO - [12225/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,424 - INFO - [12226/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,425 - INFO - [12227/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,427 - INFO - [12228/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,429 - INFO - [12229/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,896 - INFO - [12230/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12231/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12232/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,901 - INFO - [12233/12234]: 	 [Pair PCK: 0.111]	[Average: 0.333] tvmonitor
    opened by tjyuyao 1
Releases(v0.1.0)
Owner
joey zhao
Master in Computer Sciences and Technology at Fudan University
joey zhao
GitHub Repository
Self-supervised learning (SSL) is a method of machine learning

Self-supervised learning (SSL) is a method of machine learning. It learns from unlabeled sample data. It can be regarded as an intermediate form between supervised and unsupervised learning.

Ashish Patel 4 May 26, 2022
The official implementation of the Hybrid Self-Attention NEAT algorithm

PUREPLES - Pure Python Library for ES-HyperNEAT About This is a library of evolutionary algorithms with a focus on neuroevolution, implemented in pure

Adrian Westh 91 Dec 12, 2022
Compact Bidirectional Transformer for Image Captioning

Compact Bidirectional Transformer for Image Captioning Requirements Python 3.8 Pytorch 1.6 lmdb h5py tensorboardX Prepare Data Please use git clone --

YE Zhou 19 Dec 12, 2022
Parametric Contrastive Learning (ICCV2021)

Parametric-Contrastive-Learning This repository contains the implementation code for ICCV2021 paper: Parametric Contrastive Learning (https://arxiv.or

DV Lab 156 Dec 21, 2022
SkipGNN: Predicting Molecular Interactions with Skip-Graph Networks (Scientific Reports)

SkipGNN: Predicting Molecular Interactions with Skip-Graph Networks Molecular interaction networks are powerful resources for the discovery. While dee

Kexin Huang 49 Oct 15, 2022
This repository contains a PyTorch implementation of the paper Learning to Assimilate in Chaotic Dynamical Systems.

Amortized Assimilation This repository contains a PyTorch implementation of the paper Learning to Assimilate in Chaotic Dynamical Systems. Abstract: T

4 Aug 16, 2022
The official repository for "Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning" paper.

Intermdiate layer matters - SSL The official repository for "Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning" paper. Downl

Aakash Kaku 35 Sep 19, 2022
Probabilistic-Monocular-3D-Human-Pose-Estimation-with-Normalizing-Flows

Probabilistic-Monocular-3D-Human-Pose-Estimation-with-Normalizing-Flows This is the official implementation of the ICCV 2021 Paper "Probabilistic Mono

62 Nov 23, 2022
Bald-to-Hairy Translation Using CycleGAN

GANiry: Bald-to-Hairy Translation Using CycleGAN Official PyTorch implementation of GANiry. GANiry: Bald-to-Hairy Translation Using CycleGAN, Fidan Sa

Fidan Samet 10 Oct 27, 2022
Cross-Task Consistency Learning Framework for Multi-Task Learning

Cross-Task Consistency Learning Framework for Multi-Task Learning Tested on numpy(v1.19.1) opencv-python(v4.4.0.42) torch(v1.7.0) torchvision(v0.8.0)

Aki Nakano 2 Jan 08, 2022
This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning"

CSP_Deep_EEG This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning" {https://www

Seyed Mahdi Roostaiyan 2 Nov 08, 2022
TorchPQ is a python library for Approximate Nearest Neighbor Search (ANNS) and Maximum Inner Product Search (MIPS) on GPU using Product Quantization (PQ) algorithm.

Efficient implementations of Product Quantization and its variants using Pytorch and CUDA

146 Dec 28, 2022
UMPNet: Universal Manipulation Policy Network for Articulated Objects

UMPNet: Universal Manipulation Policy Network for Articulated Objects Zhenjia Xu, Zhanpeng He, Shuran Song Columbia University Robotics and Automation

Columbia Artificial Intelligence and Robotics Lab 33 Dec 03, 2022
Bootstrapped Unsupervised Sentence Representation Learning (ACL 2021)

Install first pip3 install -e . Training python3 training/unsupervised_tuning.py python3 training/supervised_tuning.py python3 training/multilingual_

yanzhang_nlp 26 Jul 22, 2022
3D Human Pose Machines with Self-supervised Learning

3D Human Pose Machines with Self-supervised Learning Keze Wang, Liang Lin, Chenhan Jiang, Chen Qian, and Pengxu Wei, “3D Human Pose Machines with Self

Chenhan Jiang 398 Dec 20, 2022
Graph Robustness Benchmark: A scalable, unified, modular, and reproducible benchmark for evaluating the adversarial robustness of Graph Machine Learning.

Homepage | Paper | Datasets | Leaderboard | Documentation Graph Robustness Benchmark (GRB) provides scalable, unified, modular, and reproducible evalu

THUDM 66 Dec 22, 2022
Official Implementation for "StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery" (ICCV 2021 Oral)

StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (ICCV 2021 Oral) Run this model on Replicate Optimization: Global directions: Mapper: Check ou

3.3k Jan 05, 2023
Data augmentation for NLP, accepted at EMNLP 2021 Findings

AEDA: An Easier Data Augmentation Technique for Text Classification This is the code for the EMNLP 2021 paper AEDA: An Easier Data Augmentation Techni

Akbar Karimi 81 Dec 09, 2022
ExCon: Explanation-driven Supervised Contrastive Learning

ExCon: Explanation-driven Supervised Contrastive Learning Contributors of this repo: Zhibo Zhang ( Zhibo (Darren) Zhang 18 Nov 01, 2022

The Fundamental Clustering Problems Suite (FCPS) summaries 54 state-of-the-art clustering algorithms, common cluster challenges and estimations of the number of clusters as well as the testing for cluster tendency.

FCPS Fundamental Clustering Problems Suite The package provides over sixty state-of-the-art clustering algorithms for unsupervised machine learning pu

9 Nov 27, 2022