SeqNet: Learning Descriptors for Sequence-Based Hierarchical Place Recognition
[ArXiv+Supplementary] [IEEE Xplore RA-L 2021] [ICRA 2021 YouTube Video]
and
SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition
[ArXiv] [CVPR 2021 Workshop 3DVR]
Sequence-Based Hierarchical Visual Place Recognition.
News:
Jun 23: CVPR 2021 Workshop 3DVR paper, "SeqNetVLAD vs PointNetVLAD", now available on arXiv. Oxford dataset to be released soon.
Jun 02: SeqNet code release with the Nordland dataset.
Setup (One time)
Conda
conda create -n seqnet python=3.8 mamba -c conda-forge -y
conda activate seqnet
mamba install numpy pytorch=1.8.0 torchvision tqdm scikit-learn faiss tensorboardx h5py -c conda-forge -y
Download
Run bash download.sh to download single image NetVLAD descriptors (3.4 GB) for the Nordland-clean dataset [a] and corresponding model files (1.5 GB) [b].
Run
Train
To train sequential descriptors through SeqNet:
python main.py --mode train --pooling seqnet --dataset nordland-sw --seqL 10 --w 5 --outDims 4096 --expName "w5"
To (re-)train single descriptors through SeqNet:
python main.py --mode train --pooling seqnet --dataset nordland-sw --seqL 1 --w 1 --outDims 4096 --expName "w1"
Test
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-22-44_l10_w5/
The above will reproduce results for SeqNet (S5) as per Supp. Table III on Page 10.
To obtain other results from the same table, expand this.
# Raw Single (NetVLAD) Descriptor
python main.py --mode test --pooling single --dataset nordland-sf --seqL 1 --split test
# SeqNet (S1)
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 1 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/
# Raw + Smoothing
python main.py --mode test --pooling smooth --dataset nordland-sf --seqL 5 --split test
# Raw + Delta
python main.py --mode test --pooling delta --dataset nordland-sf --seqL 5 --split test
# Raw + SeqMatch
python main.py --mode test --pooling single+seqmatch --dataset nordland-sf --seqL 5 --split test
# SeqNet (S1) + SeqMatch
python main.py --mode test --pooling s1+seqmatch --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/
# HVPR (S5 to S1)
# Run S5 first and save its predictions by specifying `resultsPath`
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-22-44_l10_w5/ --resultsPath ./data/results/
# Now run S1 + SeqMatch using results from above (the timestamp of `predictionsFile` would be different in your case)
python main.py --mode test --pooling s1+seqmatch --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/ --predictionsFile ./data/results/Jun03_16-07-36_l5_0.npz
Acknowledgement
The code in this repository is based on Nanne/pytorch-NetVlad. Thanks to Tobias Fischer for his contributions to this code during the development of our project QVPR/Patch-NetVLAD.
Citation
@article{garg2021seqnet,
title={SeqNet: Learning Descriptors for Sequence-based Hierarchical Place Recognition},
author={Garg, Sourav and Milford, Michael},
journal={IEEE Robotics and Automation Letters},
volume={6},
number={3},
pages={4305-4312},
year={2021},
publisher={IEEE},
doi={10.1109/LRA.2021.3067633}
}
@misc{garg2021seqnetvlad,
title={SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition},
author={Garg, Sourav and Milford, Michael},
howpublished={CVPR 2021 Workshop on 3D Vision and Robotics (3DVR)},
month={Jun},
year={2021},
}
Other Related Projects
Patch-NetVLAD (2021); Delta Descriptors (2020); CoarseHash (2020); seq2single (2019); LoST (2018)
[a] This is the clean version of the dataset that excludes images from the tunnels and red lights, exact image names can be obtained from here.
[b] These will automatically save to ./data/, you can modify this path in download.sh and get_datasets.py to specify your workdir.
0 Nov 17, 2022
36 Nov 26, 2022
840 Jan 03, 2023
71 Nov 25, 2022
6 Jul 24, 2022
38 Sep 16, 2022
22 Mar 16, 2022
20 Dec 30, 2022
48 Jan 02, 2023
82 Dec 27, 2022
73 Dec 27, 2022
123 Jan 04, 2023
40 Dec 16, 2022
50 Dec 03, 2022
1 Dec 15, 2021
22 Dec 09, 2022
39 Dec 27, 2022
130 Jan 06, 2023
1 Dec 28, 2021
3 May 01, 2022