Eye of the Dragon : Exploring Discriminatively Minimalist Sketch-based Abstractions for Object Categories Ravi Kiran Sarvadevabhatla Video Analytics Lab Supercomputer Education and Research Centre Indian Institute of Science, Bangalore, India ravikiran@ssl.serc.iisc.in Venkatesh Babu R. Video Analytics Lab Supercomputer Education and Research Centre Indian Institute of Science, Bangalore, India venky@serc.iisc.in ABSTRACT As a form of visual representation, freehand line sketches are typically studied as an end product of the sketching pro- cess. However, from a recognition point of view, one can also study various orderings and properties of the primitive strokes that compose the sketch. Studying sketches in this manner has enabled us to create novel sparse yet discrimina- tive sketch-based representations for object categories which we term category-epitomes. Concurrently, the epitome con- struction provides a natural measure for quantifying the sparseness underlying the original sketch, which we term epitome-score. We analyze category-epitomes and epitome- scores for hand-drawn sketches from a sketch dataset of 160 object categories commonly encountered in daily life. Our analysis provides a novel viewpoint for examining the com- plexity of representation for visual object categories. Categories and Subject Descriptors I.5.4 [Applications]: Computer Vision General Terms Experimentation Keywords freehand sketch; object category recognition; deep learning 1. INTRODUCTION A master painter in ancient China was once painting an elaborate dragon. His son, carefully observing his father’s craft, noticed that the eye of the dragon had not been painted. When he pointed it out, the painter replied: 画龙点睛 (para- phrased: “Adding the eye will make the dragon come alive and fly out of the painting”). A fascinating spectrum – from realistic depictions to sparsely drawn sketches – exists for hand-drawn art depicting ob- jects[27, 3]. Across this spectrum, the level of detail in the Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. MM ’15 October 26-30, 2015, Brisbane, Australia Copyright 2015 ACM 978-1-4503-3459-4/15/10 DOI: http://dx.doi.org/10.1145/2733373.2806230 ...$15.00. Figure 1: A sketch belonging to category cup. In spite of minimal detail, we can recognize the sketch easily and cor- rectly. artwork usually correlates with the confidence in recognizing the object. However, an interesting phenomenon is observed for freehand sketches : even with minimal stroke detail (see Figure 1), the underlying subject can be easily recognized by us. This suggests an inherent sparseness driving the human neuro-visual representation mechanism. Therefore, study- ing such sparsely detailed sketches can aid our understand- ing of the cognitive processes involved and spur the design of efficient representations and visual classifiers for objects. Typically, freehand line sketches are studied as an end product of the sketching process. Indeed, sketches have been utilized in-toto in the context of classification and content- based retrieval problems [11, 13, 22]. However, from a recog- nition point of view, we believe it is instructive to study the primitives (strokes) that compose the sketch, starting with the first hand-drawn stroke until the last stroke which fi- nalizes the sketch. Our belief originates in a discovery we have made : for a given sketch of an object and an associ- ated sketch classifier, there exists a minimal subset of strokes which ensures consistent and correct identification of the ob- ject category. We term this new sketch, constructed using the minimal stroke subset, as a category-epitome. Figure 2 shows examples of freehand line sketches and their corre- sponding sparse category-epitomes. In this paper, we explore the multi-faceted ways in which category-epitomes convey insights into the nature of visual object category represen- tation. In the spirit of the dragon story mentioned at the beginning, we wish to examine sketches at the moment they come “alive”.