Hand tracking research from IRIS-XR takes the stage at SIGGRAPH 2026
IRIS-XR was represented at SIGGRAPH 2026, one of the world's leading conferences on computer graphics and interactive techniques, held in Los Angeles on 21 July 2026.
Bringing together around 4,000 researchers, technology developers, and industry professionals, the conference provides a platform for sharing the latest advances in graphics, computer vision, artificial intelligence, and immersive technologies.
The project was represented by Dr. Alain Pagani and Christen Millerdurai from the German Research Center for Artificial Intelligence (DFKI), together with Denny Rönngren from Varjo.
During the "Humans & Hands – Reconstruction" technical session, researchers from DFKI, in collaboration with Rhineland-Palatinate Technical University of Kaiserslautern-Landau (RPTU) and the Max Planck Institute for Informatics (MPII), presented the paper "EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera".
The research introduces a monocular camera-based method for robust 3D hand pose estimation in egocentric scenarios. By improving camera-based hand understanding, the work contributes to more natural interaction in extended reality, where accurate hand tracking is essential for intuitive user experiences.
Beyond presenting the research, SIGGRAPH provided an opportunity to engage with the international computer graphics and XR community, exchange ideas on sensing technologies and immersive systems, and discuss future directions for AI-based perception and interaction. These discussions support the objectives of IRIS-XR, which focuses on advancing next-generation XR technologies through innovative sensing and intelligent interaction.
Reflecting on the event, Dr. Alain Pagani, IRIS-XR Project Coordinator and Principal Researcher and Deputy Director at DFKI, said:
"SIGGRAPH offered an excellent opportunity to present EgoForce to the international computer graphics and XR community. The discussions confirmed the importance of robust and unobtrusive hand tracking for the next generation of immersive systems and provided valuable input for the work conducted in IRIS-XR."
Join our journey!
Subscribe to our newsletter and follow us on LinkedIn, BlueSky, YouTube and X to keep up to date!
How can XR devices understand hand movements with a single camera? Find out in the EgoForce Conference paper!
Researchers from the German Research Center for Artificial Intelligence (DFKI), Rhineland-Palatinate Technical University of Kaiserslautern-Landau (RPTU), and the Max Planck Institute for Informatics (MPII) presented their latest research at SIGGRAPH 2026 on 21 July 2026 in Los Angeles, USA.
[READ THE PAPER HERE]
The paper "EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera" introduces a framework for reconstructing the absolute 3D pose and shape of the hands using only a single head-mounted RGB camera.
Advancing egocentric hand tracking
Accurate hand tracking is essential for natural interaction in extended reality. EgoForce addresses this challenge by incorporating forearm information to improve depth estimation and hand pose reconstruction while supporting fisheye, perspective, and wide field-of-view camera models within a single architecture. The framework combines a unified hand-arm architecture (HALO), a differentiable forearm representation, and a ray-space solver for accurate camera-space 3D hand pose estimation across different optical systems.
Results and relevance for IRIS-XR
The framework was evaluated on several egocentric benchmarks, achieving state-of-the-art performance. On the HOT3D dataset, EgoForce reduced camera-space joint error by up to 28% compared with previous methods while demonstrating robust tracking during hand-object interactions and severe occlusions. The system also runs interactively on smart glasses using a single RGB camera.
The research reflects several technological directions that are central to IRIS-XR, including AI-based perception, lightweight sensing, and natural interaction for the next generation of XR devices.
Acknowledgments
This work was partially funded by the Horizon Europe programme under the projects dAIEDGE, Grant Agreement No. 101120726, and IRIS-XR, Grant Agreement No. 101298672. The authors thank the anonymous reviewers for their valuable feedback.
Join our journey!
Subscribe to our newsletter and follow us on LinkedIn, BlueSky, YouTube and X to keep up to date!
Kick-off: IRIS-XR officially launches in Europe to build the future of Extended Reality
The IRIS-XR consortium officially gathered in Kaiserslautern, Germany, for our project's Kick-off Meeting. Hosted by DFKI (German Research Center for Artificial Intelligence), the project coordinator, the event marked the beginning of an ambitious three-year journey to develop the next generation of human-centred Extended Reality (XR) technologies.
The meeting brought together leading European experts across XR hardware, artificial intelligence, event-based sensing, optics, healthcare, aviation, and automotive innovation to align on our roadmap, vision, and technical objectives.
The vision: more natural, intelligent, and trustworthy XR
Current XR solutions face major hurdles, from visual lag and high energy consumption to user discomfort and privacy concerns. IRIS-XR is designed to overcome these limitations.
The project will create a new hardware-software platform designed to deliver more immersive, responsive, energy-efficient, and accessible XR experiences. By making interaction more intuitive and adaptive, IRIS-XR will help bring XR technologies closer to the needs of real users and professional environments.
From research to real-world impact
The project's innovations will be validated in three pilot areas: mental health and rehabilitation, professional flight simulation, and automotive design reviews. Alongside technological development, IRIS-XR places strong emphasis on accessibility, inclusiveness, privacy, and compliance with European regulations.
Alongside technological breakthroughs, IRIS-XR is deeply committed to accessibility, inclusivity, data privacy, and strict compliance with European standards (including the GDPR and the EU AI Act). By championing trustworthy AI and open interoperability standards, we aim to strengthen Europe's competitive edge in the global spatial computing ecosystem.
Key project highlights
- Development of a next-generation XR hardware-software platform
- Advanced XR display technologies for enhanced visual quality and lower energy consumption
- Integrated eye tracking and event-based sensing for natural interaction
- AI-powered multimodal interaction combining gaze, gesture, and contextual information
- Validation in healthcare, flight simulation, and automotive design environments
- Strong focus on accessibility, ethics, privacy, and trustworthy AI
- Contributions to European interoperability and standardisation initiatives