Developing and validating a palliative care subject search filter R M Sladek 1 , J Tieman 1 , B Fazekas 2 , A Abernethy 3 , D Currow 1 1. Flinders University, Adelaide, South Australia, Australia 2. Southern Adelaide Palliative Services, Repatriation General Hospital, South Australia, Australia 3. Duke University Medical Center, Durham, USA It’s a scene replayed daily in libraries throughout Australia and the world. A user approaches the librarian, requesting their expert help in searching for information on a given topic. With the librarian’s knowledge and skills in searching, they interrogate a bibliographic database. But how good are expertly generated searches on bibliographic databases? How well do the databases themselves perform? Without an objective standard of measurement, we can only rely on the feedback of our users regarding the success of the search. Yet this may have no relationship to the true quality of the search results in terms of the original query. Our presentation provides the results of a research study which assessed specially constructed subject searches of OVID Medline using an objective standard of measurement. It was undertaken to support evidence based clinical practice in palliative care. We sought to develop validated subject search filters for ultimate placement on the CareSearch website to assist researchers and clinicians in quickly finding palliative care journal content [1]. Subjectively, our experience was that relevant literature in our field is often published in the general medical literature and is hard to identify. Objectively, we demonstrated this in our study. Our findings are revealing because they raise questions about the effectiveness of searching bibliographic databases. The questions arising are likely relevant for various disciplines where there is reliance on expert searches and formal databases to retrieve bibliographic references (and where the discipline covers one or more distinct fields). Essentially we identified the exact ‘true’ number of relevant papers by hand searching four major medical journals over three years. We constructed search strategies for finding these articles on OVID Medline, and then compared the number of articles found with the articles we identified earlier. Using a well known approach borrowed from medicine (where the performance of a new diagnostic test is compared to the existing test considered to be the gold standard), and used by others to develop methodologically based search filters, we calculated various rates to express how well our searches performed: sensitivity, specificity, accuracy and precision [2]. We had hoped to create two types of searches, one highly sensitive, the other with reduced sensitivity, but improved specificity. However we failed to achieve any retrieval rates higher than 45%, all of which had high specificity rates of 99% -100%. We then sought to improve the sensitivity by modifying and adapting three published but unvalidated search strategies that were relevant to palliative care [3-5]. However only in one case did we increase sensitivity, and it was at the considerable expense of precision and accuracy. Our presented paper will discuss the findings and their implications, and importantly, identify future directions for research. It is important that we understand why our searches performed so poorly.