Results Experts and novices chose to look at different types of triplets for both true (Χ 2 (15) = 107.3; p<0.0001) and false triplets (Χ 2 (19) = 1,019.7; p<0.0001). This means that some types of triplets were preferred by experts more than novices. The scatter plots to the right illustrate the differences between experts and novices in terms of the features that they fixate. In each graph, triplets to the upper left are fixated by experts more than novices, while triplets to the lower right are fixated by novices more than experts. Overall, experts made more fixations in the same amount of time. This indicates that experts move faster during game play. Novices fixated most on the four True triplets with a V configuration (7,8,15,16). Experts did as well, but ignored one of the V configured triplets (15). Experts and novices seem to divide their focus similarly between the True triplets with a few exceptions. The false triplets are less correlated Experts made more fixations on False triplets overall. Experts fixated most on configurations 5,13, and 18, which have the same basic pattern. Novices also fixated most on that basic pattern, but different configurations (18, 15, 14). Introduction The goal of this project is to develop a methodology that can be used to infer the feature set used by participants when they search for configurations of visual features. Many situations rely heavily on the accuracy and reliability of eye movements. Airport personnel scanning luggage, doctors reading X-rays, or even just someone searching for a friend in a crowd all involve targets that have similar visual features that differ in configuration. Previous studies that used eye- tracking methodologies have shown that those with experience and expertise in a field differ from novices in where and what they look at during the completion of vision-based activities. We chose a relatively simple video game as our test stimulus because the features are exactly known, and the configuration of features uniquely defines the task. Our goal is to refine our feature extraction procedures on this simple case and then generalize to situations where the feature set is unknown. Abstract Configurations of simple objects often play a role in real world visual search tasks. For example, diagnosing a dislocated joint in an x-ray or individualizing a fingerprint to a single person are both tasks in which the relative locations of features are more important than the identity of simple features. To study the nature of perceptual expertise in a search task in which configurations define the target, we collected eye-tracking data while participants played the video game Bejeweled 2. Participants excel at this task by attending to particular configurations of pieces and ignoring others. We analyzed the eye-tracking data at every 60ms interval. We defined a set of meaningful and misleading templates and determined how often experts and novices fixated on each. This feature induction reveals the nature of the strategies that underlie perceptual expertise in this domain and establishes this methodology as a means to uncover the feature set used by participants. Feature Selection Strategies and Perceptual Expertise in Configuration Search Tasks Lindsey Kitchell, Francisco Parada, Brandi Emerick, Tom Busey Indiana University Discussion and Conclusion These analyses demonstrate a methodology to determine which configurations are fixated by experts and novices. Experts and novices had similar general fixation patterns for True triplets, except that experts had higher counts of fixations overall and deviated from the novices on a few configurations. For the false triplets, experts made many more fixations. However, both groups seemed to focus most on the same general pattern. The results suggest that experts and novices have different game playing strategies, but it is not drastically different, instead experts focus on moving faster and making more fixations, allowing them to make more moves in the same time period. The next step is to develop machine learning algorithms that will discover the underlying feature representation. We are currently exploring approaches such as the Topics Model that may reveal sets of similar configurations that all constitute a “topic”. We are also looking to reanalyse the data by scrambling the frames. Eventually we would like to apply this approach to domains where the feature set is not known a priori, such as medical x-rays or fingerprints. Results cont. To address which pattern types best capture the gaze of our subjects, we grouped the true and false pattern types into the following general types and computed the fixation rate corrected for the base rate of pattern appearance. For True triplets, subjects were more likely to fixate pattern type A than pattern type B (t(21) = 2.1; p = 0.048). Pattern type B is also fixated more often than pattern type C (t(21) = 7.67; p < 0.01). (and of course A is also fixated more often than C). For False triplets, subjects were more likely to fixate pattern type A than pattern type B (t(21) = 3.51; p < 0.001) and pattern type C (t(21) = 4.95; p < 0.001). There was no difference between B and C. Methods Stimuli We used the computer game Bejeweled 2 in this study. The objective of the game is to swap one gem with a bordering gem to create a vertical or horizontal chain of 3 or more gems. Participants 22 total subjects were used. The participants were split into two categories based on their game scores and labelled either an expert or a novice. The subjects ranged from having never played the game before to playing a few hours a day a couple times a week. Eye tracker Consists of two small cameras: • One focused on the right eye with a small infrared LED positioned next to it to illuminate the eye • One mounted on the forehead and focused on the computer Procedure • Subjects were seated in front of a computer monitor wearing a head-mounted eye- tracking device • A PowerPoint calibration sequence is played. • Eye, scene, and computer screen are recorded. • Subjects play for 15 minutes on Endless mode. • Once recording is complete, the three videos are uploaded into the ExpertEyes software, an open source software created by our research group, and divided into image sequences. • The images are then used by the software for further temporal alignment, calibration, and gaze estimation. • Export the data and import it into a program we created in Matlab • Our program then identifies the color of each gem in each frame. • Based on the structure of the game play, we created sets of templates that reflected either patterns that could or could not be manipulated to remove pieces. • True triplets refer to the configurations where one gem can be switched with another to correctly make a chain of 3 • False triplets refer to misleading configurations that appear to allow for a chain of three to be made but do not • Each gem is given a number based on how many True and False triplets it is a part of. • Using the eye gaze data, our program then determines which gem the eye was looking at by which it was closest to. • This information also gives us which triplets the subjects looked at and how many of each type of triplet. Screen capture of Bejeweled 2 ExpertEyes finds the pupil and corneal reflection and projects the gaze into the scene view. We also track the corners of the game shown in green. All possible True and False Triplet configurations. Blue circles represent gems of all one color, white circles represent gems that are not the same color as the blue. Number of True and False Triplets associated with each gem Supported by NIJ grant 2009-DN-BX-K226 to Tom Busey !"# !"$ !"% !"& ' '"! '"' '"( '") *!, ) !"# !"$ !"% !"& ' '"! '"' '"( '") * !, ) -./012 34562 780952: ;.<=: >*928: 34562 780952: ;.<=: ?4:4 @.8 34562 780952:6 !"## !$## !%## !&## '### '!## ''## !"## !$## !%## !&## '### '!## ''## ()*+,- /01- /0+23-4 5)164 782-04 /01- /0+23-4 5)164 9:4: ;)0 /01- /0+23-4<