Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items Laura Downs 1 , Anthony Francis 1 , Nate Koenig 3 , Brandon Kinman 1 , Ryan Hickman 1 , Krista Reymann 1 , Thomas B. McHugh 2 , and Vincent Vanhoucke 1 Abstract— Interactive 3D simulations have enabled break- throughs in robotics and computer vision, but simulating the broad diversity of environments needed for deep learning requires large corpora of photo-realistic 3D object models. To address this need, we present Google Scanned Objects, an open-source collection of over one thousand 3D-scanned household items released under a Creative Commons license; these models are preprocessed for use in Ignition Gazebo and the Bullet simulation platforms, but are easily adaptable to other simulators. We describe our object scanning and curation pipeline, then provide statistics about the contents of the dataset and its usage. We hope that the diversity, quality, and flexibility of Google Scanned Objects will lead to advances in interactive simulation, synthetic perception, and robotic learning. Index Terms— Data Sets for Robot Learning, Data Sets for Robotic Vision, Simulation and Animation I. I NTRODUCTION Deep learning has enabled many recent advances in com- puter vision and robotics, but training deep models requires diverse inputs in order to generalize to new scenarios [1]. Computer vision has used web scraping to gather datasets with millions of items, including ImageNet [2], Open Images [3], Youtube-8M [4], and COCO [5]; however, labeling these datasets is labor-intensive, labeling errors can distort the perception of progress [6], and this strategy does not readily generalize to 3D or real-world robotic data. Unlike images, the web does not contain a large population of high-quality 3D scenes, real-world data collection is challenging as robots are expensive and dangerous, and human labelers cannot extract 3D geometric properties from images. Simulation of robots and environments, using tools such as Gazebo [7], Bullet [8], MuJoCo [9], and Unity [10], can mitigate many of these limitations, as simulated envi- ronments can be varied safely, and semantic labels can be easily derived from the simulation state. However, simulation is always an approximation to reality: handcrafted models built from polygons and primitives correspond poorly to real objects. Even if a scene is built directly from a 3D scan of a real environment, the discrete objects in that scan will act like fixed background scenery and will not respond to 1 Laura Downs, Anthony Francis, Brandon Kinman, Ryan Hick- man, Krista Reymann, and Vincent Vanhoucke are with Robotics at Google, Mountain View, CA 94043, USA (email: ldowns@google.com, centaur@google.com, bkinman@google.com, rhickman@google.com, rey- mann@google.com, vanhoucke@google.com) 2 Thomas B. McHugh is with Northwestern University, Evanston, IL 60208, USA (email: mchugh@u.northwestern.edu) 3 Nate Koenig is with Open Robotics, Mountain View, CA 94041, USA (email: nate@openrobotics.com) Fig. 1. Custom 3D scanning hardware enabled fast capture of raw meshes, which our scanning pipeline aligned using a calibration process followed by QA curation of high-quality models for inclusion in the dataset. inputs the way that real-world objects would. A key problem, then, is providing a library of high-quality models of 3D objects which can be incorporated into physical and visual simulations to provide the required variety for deep learning. To address this issue, we present the Google Scanned Objects (GSO) dataset, 1 a curated collection of over 1000 3D scanned common household items for use in the Ignition Gazebo [7] and Bullet [8] simulators, as well as other tools that can read the SDF model format. In this letter, we describe our pipeline for object collection and curation, scalable, high-quality 3D scanning, scan quality assurance and publishing. In addition, we present breakdowns of the statistics of the objects in the dataset and the usage of the dataset in published research. Our contributions include (a) the Google Scanned Objects dataset, (b) the design of our 3D scanning pipeline, (c) the design of our 3D scan curation and publication process, and (d) a review of the impact of this dataset on research. II. RELATED WORK Many simulators are available for robotics applications [17], and many learning systems have used simulation to train models to deploy on robots. While early work used static environments in simulators similar to Bullet [8], more recent work has injected 3D objects into the environment to enable training interactive navigation, such as the Interactive Gibson benchmark [18] which uses GSO. 1 Dataset available at https://goo.gle/scanned-objects arXiv:2204.11918v1 [cs.RO] 25 Apr 2022