Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A Wide AOI tool for discord.

drkdiscord - drk#1337 A Wide AOI tool for discord. Installation To Install you have to have python 3.x and pip installed on your system. If you have t

Darkest Surface 6 Dec 17, 2022
Cloud-native, data onboarding architecture for the Google Cloud Public Datasets program

Public Datasets Pipelines Cloud-native, data pipeline architecture for onboarding datasets to the Google Cloud Public Datasets Program. Overview Requi

Google Cloud Platform 109 Dec 30, 2022
A free and open-source SMS/Call bombing application

TBOMB V0.1 A free and open-source SMS/Call bombing application NOTE: For Termux To use the bomber type the following commands in Termux: pkg install g

ᴀɴᴋɪᴛ ᴋᴜᴍᴀʀ 2 Dec 07, 2021
Cancel all your follow requests on Instagram.

Unrequester This python code unrequests all your follow requests on Instagram, using selenium. Everything's step-by-step and understanding it is like

ChamRun 3 Apr 09, 2022
Video Stream: an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat

Video Stream is an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat 🧪 Get SESSION_NAME from below: Pyrogram

Jonathan 6 Feb 08, 2022
Self-adjusting, auto-compounding multi-pair DCA crypto trading bot using Python, AWS Lambda & 3Commas API

Self-adjusting, auto-compounding multi-pair DCA crypto trading bot using Python, AWS Lambda & 3Commas API The following code describes how we can leve

Jozef Jaroščiak 21 Dec 07, 2022
Unlimited Filter Telegram Bot 2

Mother NAther Bot Features Auto Filter Manuel Filter IMDB Admin Commands Broadcast Index IMDB search Inline Search Random pics ids and User info Stats

LɪᴏɴKᴇᴛᴛʏUᴅ 1 Oct 30, 2021
🎄 JustaGrabber - A discord token grabber written in python3

🎄 JustaGrabber - A discord token grabber written in python3 🎇 Made by kldiscord https://github.com/kldiscord 🌟 Please leave a star if you liked Jus

1 Dec 19, 2022
An Unofficial TikTok API Wrapper In Python

This is an unofficial api wrapper for TikTok.com in python. With this api you are able to call most trending and fetch specific user information as well as much more.

David Teather 2.9k Jan 08, 2023
PR Changes Matrix Builder

Pr-changes-matrix-builder - A Github Action that will output a variable to be used in a matrix strategy job based on a PR&'s changes

Kyle James Walker (he/him) 21 Oct 04, 2022
Light weight Scripts and Apps for checking availability of Covid Vaccines in India. Notifies when vaccine becomes avialable in your area.

vaccine-checker Light weight Scripts and Apps for checking availability of Covid Vaccines in India. Notifies when vaccine becomes avialable in your ar

Abishek V Ashok 8 Jun 16, 2021
SmartFile API Client (Python).

A SmartFile Open Source project. Read more about how SmartFile uses and contributes to Open Source software. Summary This library includes two API cli

SmartFile 19 Jan 11, 2022
A Python app to serve Conveyor room requests and run approvals through Slack

✨ CONVEYOR FOR SLACK ✨ This is a friendly little Python app that will allow you to integrate your instance of Conveyor with your Slack workspace. In o

Vivienne 4 Sep 27, 2021
This is to notify you via Discord whenever there is a new beacon.

BeaconNotifier-Discord This is to notify you via Discord whenever there is a new beacon. Make sure you have python3 installed Steps: Create a Discord

26 Dec 28, 2022
This script will detect changes in your session using Discords built in Gateway.

Detect Session Gateway This script will detect changes in your session using Discords built in Gateway. What does this log? Discord build version Oper

Omega 5 Dec 18, 2021
TeamDltvideo Bot you can add in your Telegram Group Thanks

TeamDltvideo TeamDltvideo Bot you can add in your Telegram Group Thanks Video Stream is a telegram bot project that's allow you to play video on teleg

TeamDlt 1 Nov 18, 2021
Tiktok 2 Instagram With Python

Tiktok2Instagram 📸 About The Project What it does: Download the source video from a user inputted Tiktok URL. 📙 Add audio to the Tiktok video from a

Carter Belisle 4 Feb 06, 2022
Discord bot for polls and votes including STV. Supports hiding results and is written with Discord.py

VoteBot Discord voting bot capable of standard polls, as found in many other bots; anonymous polls, where votes are hidden and totals are only display

6 Nov 15, 2022
Prisma Cloud utility scripts, and a Python SDK for Prisma Cloud APIs.

pcs-toolbox Prisma Cloud utility scripts, and a Python SDK for Prisma Cloud APIs. Table of Contents Support Setup Configuration Script Usage CSPM Scri

Palo Alto Networks 34 Dec 15, 2022
Pythonic wrapper for the Aladhan prayer times API.

aladhan.py is a pythonic wrapper for the Aladhan prayer times API. Installation Python 3.6 or higher is required. To Install aladhan.py with pip: pip

HETHAT 8 Aug 17, 2022