Publications
A Statistical Evaluation Of The Diagnostic Performance Of
Medas - The Medical Emergency Decision Assistance System
11/04/1990
D. Christine Georgakis1, David A. Trace2, Frank Naeymi-Rad2, and Martha Evens3
1Northeastern Illinois University, Chicago, IL 60625
2University of Health Sciences / The Chicago Medical School, North Chicago, IL 60064
3Illinois Institute of Technology, Chicago, IL 60616
Abstract
Medical expert systems require comprehensive evaluation of their diagnostic accuracy. The usefulness of these systems is limited without established evaluation methods. We propose a new methodology for evaluating the diagnostic accuracy and the predictive capacity of a medical expert system. We have adapted to the medical domain measures that have been used in the social sciences to examine the performance of human experts in the decision making process. Thus, in addition to the standard summary measures, we use measures of agreement and disagreement, and Goodman and Kruskal's l and t measures of predictive association. This methodology is illustrated by a detailed retrospective evaluation of the diagnostic accuracy of the MEDAS system. In a study using 270 patients admitted to the North Chicago Veterans Administration Hospital, diagnoses produced by MEDAS are compared with the discharge diagnoses of the attending physicians. The results of the analysis confirm the high diagnostic accuracy and predictive capacity of the MEDAS system. Overall, the agreement of the MEDAS system with the "gold standard" diagnosis of the attending physician has reached a 90% level.
Download this publication as a PDF file