Each subject worked on nine cases. For each case, subjects were asked to provide a diagnosis and a measure of their confidence in the diagnosis.
"Focusing on the residents and faculty physicians, what we found was that correctness and confidence were not aligned in about 1/3 of the cases," Dr. Friedman said. "For those cases where confidence and correctness were not aligned, physicians were more likely to lack confidence in a correct diagnosis than the reverse."
Nonetheless, he added, residents and more experienced internists were "overconfident," believing they are correct when in fact they were not, in 12 to 15 percent of the cases overall.
"Overconfidence is the zone of greatest concern when one thinks about how to improve practice using information technology," Dr. Friedman said. "While this was a study performed in the laboratory and not in the clinic, the apparent prevalence of overconfidence represents a major challenge to designers of decision support systems--if the advice of these systems is to be sought and attended to when it is really needed."
Also participating in the study were Guido G. Gatti, M.S., center for Biomedical Informatics, University of Pittsburgh; Timothy M. Franz, Ph.D., department of psychology,
St. John Fisher College; Gwendolyn C. Murphy, Ph.D., Division of Community Health, Duke University; Fredric M. Wolf, Ph.D., department of medical education and informatics, University of Washington; Paul S. Heckerling, M.D., department of medicine, University of Illinois at Chicago; Paul L. Fine, M.D., department of medicine, University of Michigan; Thomas M. Miller, M.D., department of medicine, University of North Carolina; and Arthur S
'"/>
29-Mar-2005