MANUSCRIPT: How well do health professionals interpret diagnostic information? A systematic review
OBJECTIVE:
To evaluate whether clinicians differ in how they evaluate and interpret diagnostic test information.
DESIGN:
Systematic review.
DATA SOURCES:
MEDLINE, EMBASE and PsycINFO from inception to September 2013; bibliographies of retrieved studies, experts and citation search of key included studies.
ELIGIBILITY CRITERIA FOR SELECTING STUDIES:
Primary studies that provided information on the accuracy of any diagnostic test (eg, sensitivity, specificity, likelihood ratios) to health professionals and that reported outcomes relating to their understanding of information on or implications of test accuracy.
RESULTS:
We included 24 studies. 6 assessed ability to define accuracy metrics: health professionals were less likely to identify the correct definition of likelihood ratios than of sensitivity and specificity. -25 studies assessed Bayesian reasoning. Most assessed the influence of a positive test result on the probability of disease: they generally found health professionals’ estimation of post-test probability to be poor, with a tendency to overestimation. 3 studies found that approaches based on likelihood ratios resulted in more accurate estimates of post-test probability than approaches based on estimates of sensitivity and specificity alone, while 3 found less accurate estimates. 5 studies found that presenting natural frequencies rather than probabilities improved post-test probability estimation and speed of calculations.
CONCLUSIONS:
Commonly used measures of test accuracy are poorly understood by health professionals. Reporting test accuracy using natural frequencies and visual aids may facilitate improved understanding and better estimation of the post-test probability of disease.
Written by Brian S McGowan, PhD
Brian is a research scientist and educational technologist. He helped transform Pfizer’s Medical Education Group and previously served in educational leadership roles at HealthAnswers, Inc.; Acumentis, LLC.; Cephalon; and Wyeth. He taught graduate medical education programs at Arcadia University for 10 years. Dr. McGowan recently authored the book "#socialQI: Simple Solutions for Improving Your Healthcare" and has been invited to speak internationally on the subject of information flow, technology, and learning in healthcare.
