R. Shumaker (Ed.): VAMR/HCII 2013, Part II, LNCS 8022, pp. 281–290, 2013. © Springer-Verlag Berlin Heidelberg 2013 Natural Feature Tracking Augmented Reality for On-Site Assembly Assistance Systems Rafael Radkowski and James Oliver Iowa State University, Virtual Reality Applications Center, 1620 Howe Hall, Ames, IA, 50011, USA {rafael,oliver}@iastate.edu Abstract. We introduce a natural feature tracking approach that facilitates the tracking of rigid objects for an on-site assembly assistance system. The tracking system must track multiple circuit boards without added fiducial markers, and they are manipulated by the user. We use a common SIFT feature matching de- tector enhanced with a probability search. This search estimates how likely a set of query descriptors belongs to a particular object. The method was realized and tested. The results show that the probability search enhanced the identification of different circuit boards. Keywords: Augmented Reality, Natural Feature Tracking, Assembly Assistance. 1 Introduction An assembly assistance system is a computer terminal, which provides assembly work instructions such as the assembly sequence, the components needed for a prod- uct, the handling of tools, etc. They are located at assembly stations on a factory floor and are commonly used in a variety of industries. These systems are critical for no- vice assemblers who typically refer to them regularly. However, even experienced assemblers are required to use assembly assistance systems because product variants are difficult to memorize and these systems are also used to track production efficien- cy. Most assembly assistance systems are comprised of simple alphanumeric lists of instructions with perhaps links to associated 2D schematic drawings. To enhance the effectiveness of such systems, we developed an Augmented Reality (AR) assembly assistance system for a major manufacturer of electrical components that superimpos- es a live video image of a manual assembly station with 3D models, 2D texts, and annotations. It tracks the parts to assemble, shows the assembly sequence, and provides information about the assembly method. The assembly assistance system must track multiple rigid objects; in our case, planar printed circuit boards that are manipulated by the user (Figure 1). Therefore, we have developed a natural feature tracking (NFT) system. It relies on the so-called SIFT feature tracker [1]: feature maps are created that represent the objects to track. To identify and track an object, features need to be identified in a video stream and