Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection
AI-generated Key Points
- The paper presents a novel approach to reconstructing the 3D position and pose of a human using thermal reflections on everyday objects.
- The authors exploit the fact that the human body emits long-wave infrared light, which has a larger wavelength than visible light, causing many surfaces in typical scenes to act as infrared mirrors with strong specular reflections.
- By analyzing these thermal reflections onto objects, they can locate a person's position and reconstruct their pose, even if they are not visible to a normal camera.
- The authors propose an analysis-by-synthesis framework that jointly models the objects, people, and their thermal reflections.
- They evaluate their reconstruction by comparing the 2D keypoints and 3D skeleton estimated from synchronized images captured by a calibrated third camera.
- Their quantitative experiments and qualitative visualizations show the effectiveness of their technical approach as well as design decisions.
- Thermal cameras are powerful tools for studying human activities in daily environments extending computer vision systems' ability to function more robustly even under extreme light conditions.
- Integrating thermal cameras with modern computer vision models will bring out many downstream applications in robotics, graphics, and 3D perception.
Authors: Ruoshi Liu, Carl Vondrick
Abstract: The relatively hot temperature of the human body causes people to turn into long-wave infrared light sources. Since this emitted light has a larger wavelength than visible light, many surfaces in typical scenes act as infrared mirrors with strong specular reflections. We exploit the thermal reflections of a person onto objects in order to locate their position and reconstruct their pose, even if they are not visible to a normal camera. We propose an analysis-by-synthesis framework that jointly models the objects, people, and their thermal reflections, which allows us to combine generative models with differentiable rendering of reflections. Quantitative and qualitative experiments show our approach works in highly challenging cases, such as with curved mirrors or when the person is completely unseen by a normal camera.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through atree representation
Look for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.