In this paper, we examine the potential of incorporating transparent, handheld devices into head-mounted Augmented Reality (AR). Additional mobile devices have long been successfully used in head-mounted AR, but they obscure the visual context and real world objects during interaction. Transparent tangible displays can address this problem, using either transparent OLED screens or rendering by the head-mounted display itself. However, so far, there is no systematic analysis of the use of such transparent tablets in combination with AR head-mounted displays (HMDs), with respect to their benefits and arising challenges. We address this gap by introducing a research platform based on a touch-enabled, transparent interaction panel, for which we present our custom hardware design and software stack in detail. Furthermore, we developed a series of interaction concepts for this platform and demonstrate them in the context of three use case scenarios: the exploration of 3D volumetric data, collaborative visual data analysis, and the control of smart home appliances. We validate the feasibility of our concepts with interactive prototypes that we used to elicit feedback from HCI experts. As a result, we contribute to a better understanding of how transparent tablets can be integrated into future AR environments.

Related Publications

CleAR Sight: Exploring the Potential of Interacting with Transparent Tablets in Augmented Reality
Katja Krug*, Wolfgang Büschel*, Konstantin Klamka, Raimund Dachselt
In Proceedings of the 21st IEEE International Symposium on Mixed and Augmented Reality. ISMAR '22, Singapore. IEEE, 196-205, 2022.
Publisher Video Research Website
Demonstrating CleAR Sight: Transparent Interaction Panels for Augmented Reality
Wolfgang Büschel, Katja Krug, Konstantin Klamka, Raimund Dachselt
In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. CHI '23, Hamburg, Germany. ACM 2023.
Publisher Video Research Website

CleAR Sight:
Exploring the Potential of Interacting with Transparent Tablets in Augmented Reality

Category Mixed Reality
Project date 2022-2023
Project URL imld.de/clear-sight/
Collaborators Katja Krug,
Dr. Wolfgang Büschel,
Prof. Raimund Dachselt
Video Watch on YouTube
and Methods
Unity HoloLens 2 C# Raspberry Pi C Rapid Prototyping 3D-Printing Soldering Streamed UI IR Tracking
Interactivity at ACM CHI '23
in Hamburg, Germany.
CleAR Sight demo booth at CHI 2023. From left to right: Wolfgang Bueschel, Raimund Dachselt, Katja Krug and Konstantin Klamka.
Further presentations
Interactivity at IEEE 6G Summit '24
Interactivity at German Pre-CHI '23
Interactivity at Long Night of Science '23
Interactivity at CeTI GA '23
Paper presentation at IEEE ISMAR '22