Add README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,44 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
+
pipeline_tag: keypoint-detection
|
| 4 |
+
tags:
|
| 5 |
+
- hand-pose-estimation
|
| 6 |
+
- 3d-hand-reconstruction
|
| 7 |
+
- mano
|
| 8 |
+
- egocentric
|
| 9 |
+
- video
|
| 10 |
---
|
| 11 |
+
|
| 12 |
+
# ACE-Ego-Hand
|
| 13 |
+
|
| 14 |
+
Checkpoints for **ACE-Ego-Hand: Repurposing Video Diffusion Models for
|
| 15 |
+
Occlusion-Robust Egocentric 3D Hand Motion Recovery**
|
| 16 |
+
([arXiv:2608.20308](https://arxiv.org/abs/2608.20308)).
|
| 17 |
+
|
| 18 |
+
ACE-Ego-Hand recovers bimanual 3D hand motion (MANO pose, shape and
|
| 19 |
+
camera-space translation, per frame) from egocentric RGB video by repurposing a
|
| 20 |
+
video diffusion model (Wan2.2-Fun-5B-Control) as a deterministic geometry
|
| 21 |
+
encoder. Inference code:
|
| 22 |
+
**https://github.com/ggxxii/ACE-Ego-Hand** 路
|
| 23 |
+
project page: **https://ggxxii.github.io/ace-ego-hand/**
|
| 24 |
+
|
| 25 |
+
## Checkpoints
|
| 26 |
+
|
| 27 |
+
| model | checkpoint | camera intrinsics |
|
| 28 |
+
|---|---|---|
|
| 29 |
+
| K-given | `ace_ego_hand_k.pt` | required |
|
| 30 |
+
| K-free | `ace_ego_hand_kfree.pt` | not needed |
|
| 31 |
+
|
| 32 |
+
Both are the paper checkpoints. Inference code and usage:
|
| 33 |
+
**https://github.com/ggxxii/ACE-Ego-Hand**
|
| 34 |
+
|
| 35 |
+
鈿狅笍 **Pinhole only.** These checkpoints do not support native fisheye input. To
|
| 36 |
+
run on fisheye footage, undistort it to a pinhole view first and pass the
|
| 37 |
+
pinhole intrinsics.
|
| 38 |
+
|
| 39 |
+
## License
|
| 40 |
+
|
| 41 |
+
The weights are released for research purposes under CC BY-NC 4.0: they were
|
| 42 |
+
trained on research-licensed datasets and derive from the MANO body model, so
|
| 43 |
+
commercial use is not permitted. The code in the GitHub repository is
|
| 44 |
+
MIT-licensed.
|