Publications
Performance Evaluation of Medical Expert Systems
01/01/1991
D. Christine Georgakis
Northeastern Illlinois University
Chicago, Illinois 60625
Martha Evens
Illinois Institute of Technology
Chicago, Illinois 60616
Frank Naeymi-Rad, David A. Trace
University of Health Sciences/The Chicago Medical School
North Chicago, Illinois 60064
Abstract
The major problem in the evaluation of expert systems is the selection of the appropriate statistical measures of performance consistent with the parameters of the system domain. The objective of this paper is to develop the statistical evaluation methodology needed to assess the performance of medical expert systems including MEDAS - the Medical Emergency Decision Assistance Sys-tem. The measures of performance are selected so as to have an operational interpretation and also reflect the predictive diagnostic capacity of a medical expert system. Certain summary measures are used that represent the sensi-tivity, specificity, and system response of a medical expert system. Measures of agreement such as the kappa statistic and the measure of conditional agreement are used to measure the agreement between the medical expert system and the physician. Goodman and Kruskal's l and t measures of predictive as-sociation are introduced to evaluate the predictive capacity of a medical expert system. This methodology has been partially implemented in the performance evaluation of MEDAS.
Download this publication as a PDF file