code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

Related tags

Deep LearningMMNet
Overview

MMNet

This repo is the official implementation of ICCV 2021 paper "Multi-scale Matching Networks for Semantic Correspondence.".

Pre-requisite

conda create -n mmnet python==3.8.0
conda activate mmnet
conda install torch==1.8.1 torchvision==0.9.1
pip install matplotlib scikit-image pandas

for installation of gluoncvth (fcn-resnet101):

git clone https://github.com/StacyYang/gluoncv-torch.git
cd gluoncv-torch
python setup.py install

Reproduction

for test

Trained models are available on [google drive].

pascal with fcn-resnet101 backbone([email protected]:81.6%):

python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name path\to\ckp_pascal_fcnres101.pth --resize 224,320

spair with fcn-resnet101 backbone([email protected]:46.6%):

python test.py --alpha 0.05 --benchmark spair --backbone fcn-resnet101 --ckp_name path\to\ckp_spair_fcnres101.pth --resize 224,320

Bibtex

If you use this code for your research, please consider citing:

@article{zhao2021multi,
  title={Multi-scale Matching Networks for Semantic Correspondence},
  author={Zhao, Dongyang and Song, Ziyang and Ji, Zhenghao and Zhao, Gangming and Ge, Weifeng and Yu, Yizhou},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021}
}
You might also like...
A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

null 69 Dec 10, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 72 Dec 14, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 26 Sep 26, 2021
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
A PyTorch implementation of
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

Jiaqi Gu 2 Jan 4, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
《Dual-Resolution Correspondence Network》(NeurIPS 2020)
《Dual-Resolution Correspondence Network》(NeurIPS 2020)

Dual-Resolution Correspondence Network Dual-Resolution Correspondence Network, NeurIPS 2020 Dependency All dependencies are included in asset/dualrcne

Active Vision Laboratory 45 Nov 21, 2022
Comments
  • NaN during training

    NaN during training

    Hi, congrats on your paper! I was trying to run your training code (with resnet 101 on pf-pascal) but directly after a couple of iterations, nan appear in the input. Have you ever seen this issue? Thanks

    opened by PruneTruong 2
  • In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    Hello,this paper is very nice,i am very love it. I read your code,in Model.py, def calLayer1(self, feats): sum1 = self.conv1_1_down(self.msblock1_1(feats[1])) +
    self.conv1_2_down(self.msblock1_2(feats[2])) +
    self.conv1_3_down(self.msblock1_3(feats[3])) sum1 = self.wa_1(sum1) return sum1 I do not find where are these operation,self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1,so where are these ,in which document.Thank you,looking forward to your reply.

    opened by liang532 1
  • How to prepare the PF-Pascal dataset?

    How to prepare the PF-Pascal dataset?

    I downloaded the PF-dataset-Pascal.zip from the Proposal Flow paper's web page, extracted it, and run the next line of command, but get errors about missing data files.

    Input:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_pascal.pth --resize 224,320

    Expected output: some results about the benchmark results.

    Actual output:

    currently executing test.py file.
2021-11-19 02:01:59,172 - INFO - Options listed below:----------------
2021-11-19 02:01:59,172 - INFO - name: framework_train
2021-11-19 02:01:59,172 - INFO - benchmark: pfpascal
2021-11-19 02:01:59,172 - INFO - thresh_type: auto
2021-11-19 02:01:59,172 - INFO - backbone_name: fcn-resnet101
2021-11-19 02:01:59,172 - INFO - ms_rate: 4
2021-11-19 02:01:59,173 - INFO - feature_channel: 21
2021-11-19 02:01:59,173 - INFO - batch: 5
2021-11-19 02:01:59,173 - INFO - gpu: 0
2021-11-19 02:01:59,173 - INFO - data_path: /data/SC_Dataset
2021-11-19 02:01:59,173 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 02:01:59,173 - INFO - visualization_path: visualization
2021-11-19 02:01:59,173 - INFO - model_type: MMNet
2021-11-19 02:01:59,173 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_pascal.pth
2021-11-19 02:01:59,173 - INFO - log_path: ./logs/
2021-11-19 02:01:59,173 - INFO - resize: 224,320
2021-11-19 02:01:59,173 - INFO - max_kps_num: 50
2021-11-19 02:01:59,173 - INFO - split_type: test
2021-11-19 02:01:59,173 - INFO - alpha: 0.05
2021-11-19 02:01:59,173 - INFO - resolution: 2
2021-11-19 02:01:59,173 - INFO - Options all listed.------------------
2021-11-19 02:01:59,173 - INFO - ckp file: assets/model/mmnet_fcnresnet101_pascal.pth
Traceback (most recent call last):
  File "/home/runner/MMNet/test.py", line 127, in <module>
    test(logger, options)
  File "/home/runner/MMNet/test.py", line 65, in test
    test_dataset = Dataset.CorrespondenceDataset(
  File "/home/runner/MMNet/data/PascalDataset.py", line 32, in __init__
    self.train_data = pd.read_csv(self.spt_path)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/common.py", line 702, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/data/SC_Dataset/PF-PASCAL/test_pairs.csv'

    P.S. Output of executing ls /data/SC_Dataset/PF-PASCAL/:

    Annotations  html  index.html  JPEGImages  parsePascalVOC.mat  ShowMatchingPairs
    opened by tjyuyao 2
  • How to reproduce the reported test accuracy?

    How to reproduce the reported test accuracy?

    By running given following command with code on the main branch:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_spair.pth --resize 224,320 --benchmark spair

    I expect to get the reported accuracy in the Table.2 of paper, i.e. 50.4 "all" accuracy, or spair with fcn-resnet101 backbone([email protected]:46.6%): as noted in the README.md file. However I get the following output, finding nowhere the related results. Can you point out the steps to reproduce the test accuracy?

    2021-11-19 00:49:54,452 - INFO - Options listed below:----------------
2021-11-19 00:49:54,452 - INFO - name: framework_train
2021-11-19 00:49:54,453 - INFO - benchmark: spair
2021-11-19 00:49:54,453 - INFO - thresh_type: auto
2021-11-19 00:49:54,454 - INFO - backbone_name: fcn-resnet101
2021-11-19 00:49:54,455 - INFO - ms_rate: 4
2021-11-19 00:49:54,455 - INFO - feature_channel: 21
2021-11-19 00:49:54,456 - INFO - batch: 5
2021-11-19 00:49:54,456 - INFO - gpu: 0
2021-11-19 00:49:54,457 - INFO - data_path: /data/SC_Dataset
2021-11-19 00:49:54,457 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 00:49:54,458 - INFO - visualization_path: visualization
2021-11-19 00:49:54,458 - INFO - model_type: MMNet
2021-11-19 00:49:54,459 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:49:54,459 - INFO - log_path: ./logs/
2021-11-19 00:49:54,460 - INFO - resize: 224,320
2021-11-19 00:49:54,460 - INFO - max_kps_num: 50
2021-11-19 00:49:54,461 - INFO - split_type: test
2021-11-19 00:49:54,461 - INFO - alpha: 0.05
2021-11-19 00:49:54,462 - INFO - resolution: 2
2021-11-19 00:49:54,462 - INFO - Options all listed.------------------
2021-11-19 00:49:54,463 - INFO - ckp file: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:50:04,950 - INFO - [    0/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] aeroplane
2021-11-19 00:50:04,953 - INFO - [    1/12234]: 	 [Pair PCK: 0.100]	[Average: 0.217] aeroplane
2021-11-19 00:50:04,956 - INFO - [    2/12234]: 	 [Pair PCK: 0.308]	[Average: 0.247] aeroplane
2021-11-19 00:50:04,958 - INFO - [    3/12234]: 	 [Pair PCK: 0.364]	[Average: 0.276] aeroplane
2021-11-19 00:50:04,960 - INFO - [    4/12234]: 	 [Pair PCK: 0.000]	[Average: 0.221] aeroplane
2021-11-19 00:50:05,575 - INFO - [    5/12234]: 	 [Pair PCK: 0.200]	[Average: 0.217] aeroplane
2021-11-19 00:50:05,577 - INFO - [    6/12234]: 	 [Pair PCK: 0.250]	[Average: 0.222] aeroplane
2021-11-19 00:50:05,580 - INFO - [    7/12234]: 	 [Pair PCK: 0.308]	[Average: 0.233] aeroplane
2021-11-19 00:50:05,583 - INFO - [    8/12234]: 	 [Pair PCK: 0.182]	[Average: 0.227] aeroplane
2021-11-19 00:50:05,585 - INFO - [    9/12234]: 	 [Pair PCK: 0.636]	[Average: 0.268] aeroplane
2021-11-19 00:50:06,153 - INFO - [   10/12234]: 	 [Pair PCK: 0.667]	[Average: 0.304] aeroplane
2021-11-19 00:50:06,156 - INFO - [   11/12234]: 	 [Pair PCK: 0.385]	[Average: 0.311] aeroplane
2021-11-19 00:50:06,158 - INFO - [   12/12234]: 	 [Pair PCK: 0.455]	[Average: 0.322] aeroplane
2021-11-19 00:50:06,160 - INFO - [   13/12234]: 	 [Pair PCK: 0.250]	[Average: 0.317] aeroplane
2021-11-19 00:50:06,163 - INFO - [   14/12234]: 	 [Pair PCK: 0.615]	[Average: 0.337] aeroplane
2021-11-19 00:50:06,731 - INFO - [   15/12234]: 	 [Pair PCK: 0.000]	[Average: 0.316] aeroplane
...
2021-11-19 01:13:47,264 - INFO - [12216/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,265 - INFO - [12217/12234]: 	 [Pair PCK: 0.200]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,266 - INFO - [12218/12234]: 	 [Pair PCK: 0.250]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,268 - INFO - [12219/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,837 - INFO - [12220/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,838 - INFO - [12221/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,848 - INFO - [12222/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,850 - INFO - [12223/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,853 - INFO - [12224/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,422 - INFO - [12225/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,424 - INFO - [12226/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,425 - INFO - [12227/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,427 - INFO - [12228/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,429 - INFO - [12229/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,896 - INFO - [12230/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12231/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12232/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,901 - INFO - [12233/12234]: 	 [Pair PCK: 0.111]	[Average: 0.333] tvmonitor
    opened by tjyuyao 1
Releases(v0.1.0)
Owner
joey zhao
Master in Computer Sciences and Technology at Fudan University
joey zhao
GitHub Repository
MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution (CVPR2021)

MASA-SR Official PyTorch implementation of our CVPR2021 paper MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Re

DV Lab 126 Dec 20, 2022
Playable Video Generation

Playable Video Generation Playable Video Generation Willi Menapace, Stéphane Lathuilière, Sergey Tulyakov, Aliaksandr Siarohin, Elisa Ricci Paper: ArX

Willi Menapace 136 Dec 31, 2022
A python package simulating the quasi-2D pseudospin-1/2 Gross-Pitaevskii equation with NVIDIA GPU acceleration.

A python package simulating the quasi-2D pseudospin-1/2 Gross-Pitaevskii equation with NVIDIA GPU acceleration. Introduction spinor-gpe is high-level,

2 Sep 20, 2022
R-Drop: Regularized Dropout for Neural Networks

R-Drop: Regularized Dropout for Neural Networks R-drop is a simple yet very effective regularization method built upon dropout, by minimizing the bidi

756 Dec 27, 2022
This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection

Bridge-damage-segmentation This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge c

Jingxiao Liu 5 Dec 07, 2022
Genpass - A Passwors Generator App With Python3

Genpass Welcom again into another python3 App this is simply an Passwors Generat

Mal4D 1 Jan 09, 2022
An official implementation of the Anchor DETR.

Anchor DETR: Query Design for Transformer-Based Detector Introduction This repository is an official implementation of the Anchor DETR. We encode the

MEGVII Research 276 Dec 28, 2022
Python library containing BART query generation and BERT-based Siamese models for neural retrieval.

Neural Retrieval Embedding-based Zero-shot Retrieval through Query Generation leverages query synthesis over large corpuses of unlabeled text (such as

Amazon Web Services - Labs 35 Apr 14, 2022
State of the Art Neural Networks for Deep Learning

pyradox This python library helps you with implementing various state of the art neural networks in a totally customizable fashion using Tensorflow 2

Ritvik Rastogi 60 May 29, 2022
Load What You Need: Smaller Multilingual Transformers for Pytorch and TensorFlow 2.0.

Smaller Multilingual Transformers This repository shares smaller versions of multilingual transformers that keep the same representations offered by t

Geotrend 79 Dec 28, 2022
The description of FMFCC-A (audio track of FMFCC) dataset and Challenge resluts.

FMFCC-A This project is the description of FMFCC-A (audio track of FMFCC) dataset and Challenge resluts. The FMFCC-A dataset is shared through BaiduCl

18 Dec 24, 2022
Robust Partial Matching for Person Search in the Wild

APNet for Person Search Introduction This is the code of Robust Partial Matching for Person Search in the Wild accepted in CVPR2020. The Align-to-Part

Yingji Zhong 36 Dec 18, 2022
[UNMAINTAINED] Automated machine learning for analytics & production

auto_ml Automated machine learning for production and analytics Installation pip install auto_ml Getting started from auto_ml import Predictor from au

Preston Parry 1.6k Jan 02, 2023
Pytorch implementations of popular off-policy multi-agent reinforcement learning algorithms, including QMix, VDN, MADDPG, and MATD3.

Off-Policy Multi-Agent Reinforcement Learning (MARL) Algorithms This repository contains implementations of various off-policy multi-agent reinforceme

183 Dec 28, 2022
Exe-to-xlsm - Simple script to create VBscript of exe and inject to xlsm

🎁 Exe To Office Executable file injection to Office documents: .xlsm, .docm, .p

3 Jan 25, 2022
Object detection evaluation metrics using Python.

Object detection evaluation metrics using Python.

Louis Facun 2 Sep 06, 2022
Models, datasets and tools for Facial keypoints detection

Template for Data Science Project This repo aims to give a robust starting point to any Data Science related project. It contains readymade tools setu

girafe.ai 1 Feb 11, 2022
PyTorch implementation of Self-supervised Contrastive Regularization for DG (SelfReg)

SelfReg PyTorch official implementation of Self-supervised Contrastive Regularization for Domain Generalization (SelfReg, https://arxiv.org/abs/2104.0

64 Dec 16, 2022
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

ELECTRA Introduction ELECTRA is a method for self-supervised language representation learning. It can be used to pre-train transformer networks using

Google Research 2.1k Dec 28, 2022
The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks This folder contains the code to reproduce the data in "The Implicit Bias o

Samuel Lippl 0 Feb 05, 2022