camenduru/emage

EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

Public
109 runs

Run time and cost

This model runs on Nvidia L40S GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

🐣 Please follow me for new updates https://twitter.com/camenduru
🔥 Please join our discord server https://discord.gg/k5BwmmvJJU
🥳 Please join my patreon community https://patreon.com/camenduru

🕸 Replicate

https://replicate.com/camenduru/minigpt4-video

📋 Tutorial

  • Register at https://smpl-x.is.tue.mpg.de and download the SMPL-X for Blender add-on. The ZIP release (https://drive.google.com/drive/folders/1ukbifhHc85qWTzspEgvAxCXwn9mK4ifr) file (smplx_blender_addon_20230921.zip) will include the required SMPL-X+FLAME model which is not included in the code repository.
  • Blender>Edit>Preferences>Add-ons>Install
  • Select downloaded SMPL-X for Blender add-on ZIP file (smplx_blender_addon-YYYYMMDD.zip) and install
  • Enable SMPL-X for Blender add-on
  • Enable sidebar in 3D Viewport>View>Sidebar
  • SMPL-X tool will show up in sidebar

🧬 Code

https://github.com/PantoMatrix/PantoMatrix/tree/main/scripts/EMAGE_2024

📄 Paper

https://arxiv.org/abs/2401.00374

🌐 Page

https://pantomatrix.github.io/EMAGE/

https://replicate.com