Skeleton Transformer Networks: 3D Human Pose and Skinned Mesh from Single RGB Image

2018-12-29 · Yusuke Yoshiyasu, Ryusuke Sagawa, Ko Ayusawa, Akihiko Murai

In this paper, we present Skeleton Transformer Networks (SkeletonNet), an end-to-end framework that can predict not only 3D joint positions but also 3D angular pose (bone rotations) of a human skeleton from a single color image. This in turn allows us to generate skinned mesh animations. Here, we propose a two-step regression approach. The first step regresses bone rotations in order to obtain an initial solution by considering skeleton structure. The second step performs refinement based on heatmap regressor using a 3D pose representation called cross heatmap which stacks heatmaps of xy and zy coordinates. By training the network using the proposed 3D human pose dataset that is comprised of images annotated with 3D skeletal angular poses, we showed that SkeletonNet can predict a full 3D human pose (joint positions and bone rotations) from a single image in-the-wild.

📄 PDF Abstract BibTeX arXiv:1812.11328

Code (0)

등록된 구현이 없습니다.

Tasks

3D Human Pose Estimation

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Heatmap 설명 없음

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Automated Body Structure Extraction from Arbitrary 3D Mesh

2017-05-16 · Yong Khoo, Sang Chung

This paper presents an automated method for 3D character skeleton extraction that can be applied for generic 3D shapes. Our work is motivated by the skeleton-based prior work on automatic rigging focused on skeleton extr…

BLSM: A Bone-Level Skinned Model of the Human Mesh

2020-08-01 · ECCV 2020 8 · Haoyang Wang, Riza Alp Güler, Iasonas Kokkinos, George Papandreou 외

We introduce BLSM, a bone-level skinned model of the human body mesh where bone scales are set prior to template synthesis, rather than the common, inverse practice. BLSM first sets bone lengths and joint angles to speci…

Unity

DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priors

2024-09-12 · Thomas Hanwen Zhu, Ruining Li, Tomas Jakab

We present DreamHOI, a novel method for zero-shot synthesis of human-object interactions (HOIs), enabling a 3D human model to realistically interact with any given object based on a textual description. This task is comp…

Human-Object Interaction Detection NeRF

Skinned Motion Retargeting with Dense Geometric Interaction Perception

2024-10-28 · Zijie Ye, Jia-Wei Liu, Jia Jia, Shikun Sun 외

Capturing and maintaining geometric interactions among different body parts is crucial for successful motion retargeting in skinned characters. Existing approaches often overlook body geometries or add a geometry correct…

motion retargeting

HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset

2024-12-03 · CVPR 2025 1 · Zedong Chu, Feng Xiong, Meiduo Liu, Jinzhi Zhang 외

With the rapid evolution of 3D generation algorithms, the cost of producing 3D humanoid character models has plummeted, yet the field is impeded by the lack of a comprehensive dataset for automatic rigging, which is a pi…

3D Generation