Abstract

The recent extensive implementation of electronic health record systems worldwide in health care institutions has generated rich and dynamic data, which brought the opportunity for widespread integration of computerized clinical decision support (CCDS) into hospital systems. These CCDS systems have the potential to support early diagnosis of life-threatening health conditions, such as sepsis – it affects 50 million people per annum worldwide with 11 million sepsis-related deaths. Early diagnosis and followed by prompt treatment are associated with better outcomes for sepsis patients. CCDS systems using different algorithms have been designed to alert clinicians to patients at risk of being septic. These systems have been implemented in different clinical settings, e.g. at emergency departments (EDs) or in the intensive care units (ICUs), targeting different populations, e.g. children or adults. Based on our published scoping reviews, patient outcomes are the most frequently reported outcomes of CCDS system evaluations. However, rigorous evidence demonstrating the diagnostic accuracy of these systems is relatively limited. In addition, due to generally inverse correlation of sensitivity and specificity, more sophisticated statistical methods are required for the meta-analysis of diagnostic test accuracy. The objective of this paper is to examine the methods and discuss specific challenges of conducting meta-analysis to evaluate the diagnostic accuracy of CCDS systems. Three specific meta-analyses are drawn on the evaluations of the diagnostic accuracy of CCDS: i) for adult population presented at EDs; ii) for adult populations admitted in the ICUs; and iii) for paedicatric patients in different hospital settings. Hierarchical models, including the bivariate and hierarchical summary receiver operating characteristic (HSROC) analysis, are fitted and compared in both frequentist and Bayesian statistics frameworks. In the Bayesian framework, the prior distributions of the model parameters and the data are used to estimate the posterior distributions of the model parameters. These are obtained by using Markov Chain Monte Carlo algorithms. The point estimations of the sensitivity, the specificity and Bayesian AUC (BAUC) are calculated as an average of their respective posterior distributions. The tradeoff between sensitivity and specificity are presented and discussed. A CCDS with a high sensitivity is beneficial as sepsis patients deteriorate rapidly if they are not diagnosed and promptly treated. On the other hand, a high specificity CCDS can potentially reduce unnecessary antibiotic use or fluid resuscitation treatment, both of which have been linked to negative patient outcomes.