International Journal of Medical Informatics (2003) 72, 17—28 Automated coded ambulatory problem lists: evaluation of a vocabulary and a data entry tool Samuel J. Wang a, * , David W. Bates a,b , Henry C. Chueh c , Andrew S. Karson a,d , Saverio M. Maviglia a,b , Julie A. Greim a , Jennifer P. Frost a , Gilad J. Kuperman a a Department of Information Systems, Partners HealthCare System, 93 Worcester Street, Wellesley, MA 02481, USA b Division of General Internal Medicine, Brigham & Women’s Hospital, Boston, MA, USA c Laboratory for Computer Science, Massachusetts General Hospital, Boston, MA, USA d Department of Medicine, Massachusetts General Hospital, Boston, MA, USA Received 19 December 2002; received in revised form 11 August 2003; accepted 12 August 2003 KEYWORDS Vocabulary, controlled; Medical record systems, computerized; Medical records, problem-orientated Summary Background: Problem lists are fundamental to electronic medical records (EMRs). However, obtaining an appropriate problem list dictionary is difficult, and getting users to code their problems at the time of data entry can be challenging. Objective: To develop a problem list dictionary and search algorithm for an EMR sys- tem and evaluate its use. Methods: We developed a problem list dictionary and lookup tool and implemented it in several EMR systems. A sample of 10,000 problem entries was reviewed from each system to assess overall coding rates. We also performed a manual review of a subset of entries to determine the appropriateness of coded en- tries, and to assess the reasons other entries were left uncoded. Results: The overall coding rate varied significantly between different EMR implementations (63—79%). Coded entries were virtually always appropriate (99%). The most frequent reasons for uncoded entries were due to user interface failures (44—45%), insufficient dictionary coverage (20—32%), and non-problem entries (10—12%). Conclusion: The problem list dictionary and search algorithm has achieved a good coding rate, but the rate is de- pendent on the specific user interface implementation. Problem coding is essential for providing clinical decision support, and improving usability should result in better coding rates. © 2003 Elsevier Ireland Ltd. All rights reserved. 1. Introduction In 1968, Weed asserted that a ‘‘complete and accurate’’ problem list was important in the un- derstanding of patients and populations and that *Corresponding author. Tel.: +1-781-453-0095; fax: +1-978-334-7742. E-mail address: sjwang@partners.org (S.J. Wang). ‘‘storage (of the problem list) in the computer should receive high priority’’ [1]. The Institute of Medicine’s report on computer-based patient record also emphasized the importance of having ‘‘an easily reviewed and updated problem list’’ in any computer-based patient record system [2]. Accordingly, many clinical information systems de- veloped in the last several years allow management of patient problem lists [3—10]. While substantial evidence regarding the importance of problem list 1386-5056/$ — see front matter © 2003 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijmedinf.2003.08.002