# HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting

**Authors:** Yuheng Jiang, Zhehao Shen, Penghao Wang, Zhuo Su, Yu Hong, Yingliang Zhang, Jingyi Yu, Lan Xu  
**arXiv:** [2312.03461](https://arxiv.org/abs/2312.03461)  
**Date:** 2023-12-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.03461v1/

## Key points

- Significant progress in photo-real human modeling and rendering
- Challenges in efficiently rendering realistic human performance and integrating it into the rasterization pipeline
- Introduction of HiFi4G - a novel approach for high-fidelity human performance rendering from dense footage
- Core idea of combining 3D Gaussian representation with non-rigid tracking for compact and compression-friendly representation
- Dual-graph mechanism utilizing coarse deformation graphs for initialization and fine-grained Gaussian graphs for constraints enforcement
- Motion priors crucial for accurate rendering obtained through this approach
- Use of 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to enhance rendering process
- Companion compression scheme with residual compensation enabling immersive experiences on various platforms
- Impressive compression rate of approximately 25 times, requiring less than 2MB of storage per frame
- Extensive experiments demonstrating effectiveness in terms of optimization speed, rendering quality, and storage overhead
- Comprehensive solution for high-fidelity human performance rendering with applications in computer graphics and virtual reality.

## Summary

In recent years, significant progress has been made in the field of photo-real human modeling and rendering. However, efficiently rendering realistic human performance and integrating it into the rasterization pipeline still poses challenges. To address this issue, a team of researchers presents HiFi4G - a novel approach for high-fidelity human performance rendering from dense footage. The core idea behind HiFi4G is to combine 3D Gaussian representation with non-rigid tracking to achieve a compact and compression-friendly representation. The researchers propose a dual-graph mechanism that utilizes both coarse deformation graphs for effective initialization and fine-grained Gaussian graphs to enforce subsequent constraints. This approach allows them to obtain motion priors crucial for accurate rendering. To further enhance the rendering process, HiFi4G employs a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers. This optimization scheme effectively balances the non-rigid prior with Gaussian updating, resulting in improved rendering quality. Additionally, the researchers introduce a companion compression scheme with residual compensation, enabling immersive experiences on various platforms. One notable achievement of HiFi4G is its impressive compression rate of approximately 25 times, requiring less than 2MB of storage per frame. Extensive experiments demonstrate the effectiveness of this approach as it outperforms existing methods in terms of optimization speed, rendering quality, and storage overhead. Overall, offers a comprehensive solution for high-fidelity human performance rendering by leveraging  and . Its ability to efficiently integrate realistic human models into the rasterization pipeline makes it an invaluable tool for various applications in computer graphics and virtual reality.