Conditional Motion In-Betweening (CMIB)
Official implementation of paper: Conditional Motion In-betweeening.
Paper(arXiv) | Project Page | YouTube
| in-betweening | pose-conditioned |
|---|---|
 |
 |
| walk | jump | dance |
|---|---|---|
 |
 |
 |
Environments
This repo is tested on following environment:
- Ubuntu 20.04
- Python >= 3.7
- PyTorch == 1.10.1
- Cuda V11.3.109
Install
-
Follow
LAFAN1dataset's installation guide. You need to install git lfs first before cloning the dataset repo. -
Run LAFAN1's
evaluate.pyto unzip and validate it. (Installnumpyfirst if you don't have it)$ pip install numpy $ python ubisoft-laforge-animation-dataset/evaluate.py
With this, you will have unpacked LAFAN dataset under
ubisoft-laforge-animation-datasetfolder. -
Install appropriate
pytorchversion depending on your device(CPU/GPU), then install packages listed inrequirements.txt. .
Trained Weights
You can download trained weights from here.
Train from Scratch
Trining script is trainer.py.
python trainer.py \
--processed_data_dir="processed_data_80/" \
--window=90 \
--batch_size=32 \
--epochs=5000 \
--device=0 \
--entity=cmib_exp \
--exp_name="cmib_80" \
--save_interval=50 \
--learning_rate=0.0001 \
--loss_cond_weight=1.5 \
--loss_pos_weight=0.05 \
--loss_rot_weight=2.0 \
--from_idx=9 \
--target_idx=88 \
--interpolation='slerp'
Inference
You can use run_cmib.py for inference. Please refer to help page of run_cmib.py for more details.
python run_cmib.py --help
Reference
- LAFAN1 Dataset
@article{harvey2020robust, author = {Félix G. Harvey and Mike Yurick and Derek Nowrouzezahrai and Christopher Pal}, title = {Robust Motion In-Betweening}, booktitle = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH)}, publisher = {ACM}, volume = {39}, number = {4}, year = {2020} }
Citation
@misc{kim2022conditional,
title={Conditional Motion In-betweening},
author={Jihoon Kim and Taehyun Byun and Seungyoun Shin and Jungdam Won and Sungjoon Choi},
year={2022},
eprint={2202.04307},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
744 Jan 04, 2023
2.1k Jan 04, 2023
4.4k Jan 08, 2023
218 Oct 27, 2022
72 Dec 18, 2022
148 Dec 29, 2022
24 Dec 12, 2022
38 Dec 28, 2022
116 Mar 07, 2022
828 Dec 28, 2022
65 Jan 07, 2023
614 Dec 08, 2022
124 Dec 28, 2022
33 Dec 01, 2022
2 Nov 07, 2022
1 Feb 09, 2022
2 Mar 16, 2022
72 Jan 01, 2023
2 May 05, 2022
196 Jan 05, 2023