Content-Based Management Service for Medical Videos Engin Mendi, PhD, 1, * Coskun Bayrak, PhD, 1 Songul Cecen, MS, 1 and Emre Ermisoglu, MS 2 1 University of Arkansas at Little Rock, Little Rock, Arkansas. 2 University of Arkansas for Medical Sciences, Little Rock, Arkansas. *Current address: Department of Computer Engineering, KTO Karatay University, Karatay, Konya, Turkey. Abstract Development of health information technology has had a dramatic impact to improve the efficiency and quality of medical care. Devel- oping interoperable health information systems for healthcare pro- viders has the potential to improve the quality and equitability of patient-centered healthcare. In this article, we describe an automated content-based medical video analysis and management service that provides convenience and ease in accessing the relevant medical video content without sequential scanning. The system facilitates effective temporal video segmentation and content-based visual information retrieval that enable a more reliable understanding of medical video content. The system is implemented as a Web- and mobile-based service and has the potential to offer a knowledge-sharing platform for the purpose of efficient medical video content access. Key words: medical digital video libraries, health information technologies, temporal video segmentation, content-based informa- tion retrieval Introduction M edical video repositories play important roles for many health-related issues such as medical imaging, medical research and education, medical diagnostics, and training of medical professionals. Because of limita- tions in accessing medical expertise, the health maintenance system of a country may face a variety of problems that will directly affect an individual’s quality of life and also the entire well-being of a society. Connecting as many hospitals as possible to a medical information system would be very beneficial in terms of an improved standard of medical practice and educational aspects for medical students and staff who cannot reach medical resources because of resource, geo- graphical, and time constraints. 1,2 The maturation of Adobe Flash and Web 2.0 has led the launch of several video-sharing Web sites such as YouTube 3 and Vimeo, 4 al- lowing users to post video clips online and share them with others. 5 There are also topic-specific video sites like OrLive, 6 an online sur- gical and healthcare video and Web cast platform. These Web sites enable users to stream video content. Although they are efficient in distributing videos over the broadband network, they lack mecha- nisms for effective content management and organization. Video content in digital libraries is of only limited utility without appro- priate organization and management. 7 Video streams must be di- vided into smaller meaningful segments, and their semantics must be described in order to construct an index for effective retrieval. In- dexing the video makes access to certain entities of the content timely and efficient. The data must be partitioned in a hierarchical fashion into meaningfully clustered subgroups so that the foundational structures required for conducting point operation for extracting the related information are obtained. 8 In this article, we present a content-based management service for medical videos that provides convenience and ease in accessing the relevant medical video content without sequential scanning. The pro- posed service (1) automatically detects the boundaries of the shot changes and partitions a video into shorter segments, (2) provides a pictorial summarization of the video, (3) enables retrieval and access of the video content based on query image, and (4) provides a variety of ways for accessing any particular part of the video (i.e., clicking the key frame starts the video playing from that point in time). We implemented the system for both Web and mobile environments. We give a high- level description of key system components here. The details of the algorithms used in the system can be found in our previous work. 9,10 Materials and Methods BACKGROUND This section presents a brief overview of temporal video seg- mentation, including shot boundary detection and key frame ex- traction processes and video content retrieval problems. Temporally segmenting videos by detecting the shot boundaries aims to break up the video into meaningful segments so designated shots contain the same semantic information and then key frames are selected to represent each shot. Most existing methods use a similarity metric between successive frames to detect shot boundaries. Based on the similarity measure, the algorithms can be divided into three categories: pixel, block-based, and histogram comparisons. Pixel-level comparison 11–15 is the simplest way to evaluate the intensity values of corresponding difference in pixels between suc- cessive frames. A shot boundary has been found if the difference in mean absolute change in the intensity value of the pixels is greater than a prespecified threshold T. Block-based approaches 11,13,16–18 are based on the comparison of corresponding regions (blocks) in two successive frames. Frames are divided into blocks that are compared with their corresponding 36 TELEMEDICINE and e-HEALTH JANUARY 2013 DOI: 10.1089/tmj.2011.0239