STUDY The Performance of SolarScan An Automated Dermoscopy Image Analysis Instrument for the Diagnosis of Primary Melanoma Scott W. Menzies, MB, BS, PhD; Leanne Bischof, M Biomed E; Hugues Talbot, PhD; Alex Gutenev, PhD; Michelle Avramidis, BSc; Livian Wong, BSc; Sing Kai Lo, PhD; Geoffrey Mackellar, PhD, BSc; Victor Skladnev, PhD; William McCarthy, MB, BS, MEd; John Kelly, MD, BS; Brad Cranney, MB, BS; Peter Lye, MB, BS; Harold Rabinovitz, MD; Margaret Oliviero, ARNP; Andreas Blum, MD; Alexandra Virol, B Med; Brian De’Ambrosis, MB, BS; Roderick McCleod, MB, BS; Hiroshi Koga, MD; Caron Grin, MD; Ralph Braun, MD; Robert Johr, MD Objective: To describe the diagnostic performance of SolarScan (Polartechnics Ltd, Sydney, Australia), an au- tomated instrument for the diagnosis of primary mela- noma. Design: Images from a data set of 2430 lesions (382 were melanomas; median Breslow thickness, 0.36 mm) were divided into a training set and an independent test set at a ratio of approximately 2:1. A diagnostic algorithm (ab- solute diagnosis of melanoma vs benign lesion and esti- mated probability of melanoma) was developed and its performance described on the test set. High-quality clini- cal and dermoscopy images with a detailed patient his- tory for 78 lesions (13 of which were melanomas) from the test set were given to various clinicians to compare their diagnostic accuracy with that of SolarScan. Setting: Seven specialist referral centers and 2 general practice skin cancer clinics from 3 continents. Compari- son between clinician diagnosis and SolarScan diagnosis was by 3 dermoscopy experts, 4 dermatologists, 3 trainee dermatologists, and 3 general practitioners. Patients: Images of the melanocytic lesions were ob- tained from patients who required either excision or digi- tal monitoring to exclude malignancy. Main Outcome Measures: Sensitivity, specificity, the area under the receiver operator characteristic curve, me- dian probability for the diagnosis of melanoma, a direct com- parison of SolarScan with diagnoses performed by humans, and interinstrument and intrainstrument reproducibility. Results: The melanocytic-only diagnostic model was highly reproducible in the test set and gave a sensitivity of 91% (95% confidence interval [CI], 86%-96%) and specificity of 68% (95% CI, 64%-72%) for melanoma. SolarScan had comparable or superior sensitivity and specificity (85% vs 65%) compared with those of ex- perts (90% vs 59%), dermatologists (81% vs 60%), train- ees (85% vs 36%; P =.06), and general practitioners (62% vs 63%). The intraclass correlation coefficient of intra- instrument repeatability was 0.86 (95% CI, 0.83-0.88), indicating an excellent repeatability. There was no sig- nificant interinstrument variation (P = .80). Conclusions: SolarScan is a robust diagnostic instru- ment for pigmented or partially pigmented melanocytic lesions of the skin. Preliminary data suggest that its per- formance is comparable or superior to that of a range of clinician groups. However, these findings should be con- firmed in a formal clinical trial. Arch Dermatol. 2005;141:1388-1396 A LTHOUGH EARLY DETEC- tion of melanoma is criti- cal for controlling mortal- ity from the disease, it is clear that diagnostic accu- racy in the field is suboptimal. 1,2 There- fore, a considerable effort has gone into producing automated diagnostic instru- ments (so-called machine diagnosis) for primary melanoma of the skin. Studies conducted before March 2002 3 and after March 2002 4-11 were reviewed; from these reviews, basic quality requirements for de- scribing such instruments were out- lined 3 : (1) selection of lesions should be random or consecutive; (2) inclusion and exclusion criteria should be clearly stated; (3) all lesions clinically diagnosed as me- lanocytic should be analyzed; (4) the study setting should be clearly defined; (5) to avoid verification bias, clearly benign le- sions that were not excised should be in- cluded, with the diagnostic gold stan- dard being short-term follow-up with digital monitoring; (6) instrument cali- bration should be reported; (7) repeat- For editorial comment see page 1444 Author Affiliations are listed at the end of this article. (REPRINTED) ARCH DERMATOL/ VOL 141, NOV 2005 WWW.ARCHDERMATOL.COM 1388 ©2005 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 06/12/2020