Using a Dual-Camera Smartphone to Recognize Imperceptible 2D Barcodes Embedded in Videos Yuki Kakui Kota Araki Changyo Han The University of Tokyo The University of Tokyo The University of Tokyo Tokyo, Japan Tokyo, Japan Tokyo, Japan ykakui@nae-lab.org araki@nae-lab.org hanc@nae-lab.org Shogo Fukushima Takeshi Naemura Kyushu University The University of Tokyo Fukuoka, Japan Tokyo, Japan shogo@ait.kyushu-u.ac.jp naemura@nae-lab.org (a) Capture method. The proposed method uses a dual-camera smartphone for simulta- neously capturing two images with diferent exposure times. (b) Decoding result of the video clip, "football" [7] obtained by the sim- ulation experiment. While the QR code cannot be read in the previous method, it can be read in the proposed method since the artifacts caused by the interframe diferences are reduced. Figure 1: Comparison of the previous method [1] and the proposed method ABSTRACT Invisible screen-camera communication is promising in that it does not interfere with the video viewing experience. In the impercepti- ble color vibration method, which displays two colors of the same luminance alternately at high speed for each pixel, embedded infor- mation is decoded by taking the diference between distant frames on the time axis. Therefore, the interframe diferences of the original video contents afect the decoding performance. In this study, we propose a decoding method which utilizes simultaneously captured images using a dual-camera smartphone with diferent exposure times. This allows taking the color diference between the frames that are close to each other on the time axis. The feasibility of this approach is demonstrated through several application examples. CCS CONCEPTS • Human-centered computing → Ubiquitous and mobile com- puting systems and tools. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). UIST ’22 Adjunct, October 29-November 2, 2022, Bend, OR, USA © 2022 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-9321-8/22/10. https://doi.org/10.1145/3526114.3558672 KEYWORDS screen–camera communication, ubiquitous computing ACM Reference Format: Yuki Kakui, Kota Araki, Changyo Han, Shogo Fukushima, and Takeshi Nae- mura. 2022. Using a Dual-Camera Smartphone to Recognize Imperceptible 2D Barcodes Embedded in Videos. In The Adjunct Publication of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ’22 Adjunct), October 29-November 2, 2022, Bend, OR, USA. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3526114.3558672 1 INTRODUCTION Visible 2D barcodes such as QR codes [2] are widely used for screen- camera communication. However, they are obtrusive and can in- terfere with the video viewing experience. Therefore, extensive research has been conducted on invisible screen-camera communi- cation. Some approaches involve making small changes in pixel val- ues that are imperceptible to human eyes [3, 5]. However, these ap- proaches cannot achieve high throughput. Meanwhile, approaches that utilize complementary brightness changes that exceed the crit- ical ficker frequency (CFF) require a display with a refresh rate of 120 Hz as the CFF of brightness is approximately 60 Hz [6, 8, 9]. Some approaches, such as imperceptible color vibration [1], lever- age the chromatic CFF, which is approximately 25 Hz [4]. In the imperceptible color vibration method, data is encoded as the color modulation in the XZ plane of the XYZ color space and two frames are displayed at 60 Hz. During the decoding process, images are captured every 1/24 s and the absolute diference is