Digestly

Dec 10, 2024

GASP: Gaussian Avatars with Synthetic Priors

Microsoft Research - GASP: Gaussian Avatars with Synthetic Priors

The video introduces GASP, a new method for generating highly realistic, real-time animatable avatars with full 360-degree rendering capabilities. Unlike previous models that require multiple cameras, GASP can be trained using data from a single camera, making it accessible to average users. The method addresses the limitations of existing Gaussian avatar techniques, which struggle with novel view synthesis when trained with single-camera data. GASP uses a generative prior trained on a large synthetic dataset to fill in missing data, such as the sides and back of the head, which are typically not captured by a single camera. This synthetic dataset provides perfectly accurate labels and correspondence to a 3D morphable model (3DMM), overcoming the challenges of real-world datasets that often lack diversity and have imperfect annotations. The training process involves using an auto-decoder where each subject is assigned a latent vector. The prior network maps these vectors, along with camera and 3DMM parameters, to obtain Gaussian avatar parameters. The model can then fit an avatar for a user using a single image or short video. The fitting process includes optimizing a latent code, fine-tuning the decoder to match unseen regions, and adjusting Gaussians to improve quality in available data areas. GASP is compared to state-of-the-art models, demonstrating superior performance in real-time animation and single-image avatar creation, while also being more computationally efficient than models like CFA, which require more resources and produce static expressions.

Key Points:

  • GASP enables realistic avatar creation using single-camera data, making it accessible for average users.
  • The method uses a generative prior trained on synthetic data to fill in gaps left by single-camera setups.
  • GASP's training process involves an auto-decoder and optimization of latent codes to fit user-specific avatars.
  • The model outperforms existing techniques in real-time animation and efficiency, running on a single GPU.
  • GASP provides a significant improvement over models like CFA, which are resource-intensive and less flexible.
View Full Content
Upgrade to Plus to unlock complete episodes, key insights, and in-depth analysis
Starting at $5/month. Cancel anytime.