New approaches to measuring the performance of programs that generate differential diagnoses using ROC curves and other metrics

INTRODUCTION: Evaluation of computer programs which generate multiple diagnoses can be hampered by a lack of effective, well recognized performance metrics. We have developed a method to calculate mean sensitivity and specificity for multiple diagnoses and generate ROC curves

Bibliographische Detailangaben
Veröffentlicht in:Proceedings. AMIA Symposium. - 1998. - (2000) vom: 01., Seite 255-9
1. Verfasser: Fraser, H S (VerfasserIn)
Weitere Verfasser: Naimi, S, Long, W J
Format: Aufsatz
Sprache:English
Veröffentlicht: 2000
Zugriff auf das übergeordnete Werk:Proceedings. AMIA Symposium
Schlagworte:Evaluation Study Journal Article Research Support, U.S. Gov't, P.H.S.
LEADER 01000naa a22002652 4500
001 NLM109954459
003 DE-627
005 20231222152439.0
007 tu
008 231222s2000 xx ||||| 00| ||eng c
028 5 2 |a pubmed24n0367.xml 
035 |a (DE-627)NLM109954459 
035 |a (NLM)11079884 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Fraser, H S  |e verfasserin  |4 aut 
245 1 0 |a New approaches to measuring the performance of programs that generate differential diagnoses using ROC curves and other metrics 
264 1 |c 2000 
336 |a Text  |b txt  |2 rdacontent 
337 |a ohne Hilfsmittel zu benutzen  |b n  |2 rdamedia 
338 |a Band  |b nc  |2 rdacarrier 
500 |a Date Completed 08.03.2001 
500 |a Date Revised 10.12.2019 
500 |a published: Print 
500 |a Citation Status MEDLINE 
520 |a INTRODUCTION: Evaluation of computer programs which generate multiple diagnoses can be hampered by a lack of effective, well recognized performance metrics. We have developed a method to calculate mean sensitivity and specificity for multiple diagnoses and generate ROC curves 
520 |a METHODS: Data came from a clinical evaluation of the Heart Disease Program (HDP). Sensitivity, specificity, positive and negative predictive value (PPV, NPV) were calculated for each diagnosis type in the study. A weighted mean of overall sensitivity and specificity was derived and used to create an ROC curve. Alternative metrics Comprehensiveness and Relevance were calculated for each case and compared to the other measures 
520 |a RESULTS: Weighted mean sensitivity closely matched Comprehensiveness and mean PPV matched Relevance. Plotting the Physician's sensitivity and specificity on the ROC curve showed that their discrimination was similar to the HDP but sensitivity was significantly lower 
520 |a CONCLUSIONS: These metrics give a clear picture of a program's diagnostic performance and allow straightforward comparison between different programs and different studies 
650 4 |a Evaluation Study 
650 4 |a Journal Article 
650 4 |a Research Support, U.S. Gov't, P.H.S. 
700 1 |a Naimi, S  |e verfasserin  |4 aut 
700 1 |a Long, W J  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t Proceedings. AMIA Symposium  |d 1998  |g (2000) vom: 01., Seite 255-9  |w (DE-627)NLM098642928  |x 1531-605X  |7 nnns 
773 1 8 |g year:2000  |g day:01  |g pages:255-9 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |j 2000  |b 01  |h 255-9