HandOcc: NeRF-based Hand Rendering with Occupancy Networks
We propose HandOcc, a novel framework for hand rendering based upon occupancy. Popular rendering methods such as NeRF are often combined with parametric meshes to provide deformable hand models. However, in doing so, such approaches present a trade-off between the fidelity of the mesh and the complexity and dimensionality of the parametric model. The simplicity of parametric mesh structures is appealing, but the underlying issue is that it binds methods to mesh initialization, making it unable to generalize to objects where a parametric model does not exist. It also means that estimation is tied to mesh resolution and the accuracy of mesh fitting. This paper presents a pipeline for meshless 3D rendering, which we apply to the hands. By providing only a 3D skeleton, the desired appearance is extracted via a convolutional model. We do this by exploiting a NeRF renderer conditioned upon an occupancy-based representation. The approach uses the hand occupancy to resolve hand-to-hand interactions further improving results, allowing fast rendering, and excellent hand appearance transfer. On the benchmark InterHand2.6M dataset, we achieved state-of-the-art results.
Code (0)
등록된 구현이 없습니다.
Tasks
Appearance TransferNeRFSimilar Papers 제목 키워드 기반
HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network
Hands are often severely occluded by objects, which makes 3D hand mesh estimation challenging. Previous works often have disregarded information at occluded regions. However, we argue that occluded regions have strong co…
3D Hand Pose Estimationhand-object poseOccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flow
Semantic occupancy has recently gained significant traction as a prominent 3D scene representation. However, most existing methods rely on large and costly datasets with fine-grained 3D voxel labels for training, which l…
NeRFSelf-Supervised LearningPlug-and-Play Acceleration of Occupancy Grid-based NeRF Rendering using VDB Grid and Hierarchical Ray Traversal
Transmittance estimators such as Occupancy Grid (OG) can accelerate the training and rendering of Neural Radiance Field (NeRF) by predicting important samples that contributes much to the generated image. However, OG man…
NeRFGeoTransfer : Generalizable Few-Shot Multi-View Reconstruction via Transfer Learning
This paper presents a novel approach for sparse 3D reconstruction by leveraging the expressive power of Neural Radiance Fields (NeRFs) and fast transfer of their features to learn accurate occupancy fields. Existing 3D r…
3D ReconstructionNeRFSurface ReconstructionTransfer LearningCLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations
Recent advances in neural radiance fields (NeRFs) achieve state-of-the-art novel view synthesis and facilitate dense estimation of scene properties. However, NeRFs often fail for large, unbounded scenes that are captured…
Depth EstimationDepth PredictionNeRFNovel View Synthesis