The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision

Cresswell, Kathrin and Rigby, Michael and Magrabi, Farah and Scott, Philip and Brender, Jytte and Craven, Catherine K. and Wong, Zoie Shui-Yee and Kukhareva, Polina and Ammenwerth, Elske and Georgiou, Andrew and Medlock, Stephanie and De Keizer, Nicolette F. and Nykänen, Pirkko and Prgomet, Mirela and Williams, Robin (2023) The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision. Health Policy, 136. ISSN 01688510

Full text not available from this repository.
Official URL:


Despite the renewed interest in Artificial Intelligence-based clinical decision support systems (AI-CDS), there is still a lack of empirical evidence supporting their effectiveness. This underscores the need for rigorous and continuous evaluation and monitoring of processes and outcomes associated with the introduction of health information technology. We illustrate how the emergence of AI-CDS has helped to bring to the fore the critical importance of evaluation principles and action regarding all health information technology applications, as these hitherto have received limited attention. Key aspects include assessment of design, implementation and adoption contexts; ensuring systems support and optimise human performance (which in turn requires understanding clinical and system logics); and ensuring that design of systems prioritises ethics, equity, effectiveness, and outcomes. Going forward, information technology strategy, implementation and assessment need to actively incorporate these dimensions. International policy makers, regulators and strategic decision makers in implementing organisations therefore need to be cognisant of these aspects and incorporate them in decision-making and in prioritising investment. In particular, the emphasis needs to be on stronger and more evidence-based evaluation surrounding system limitations and risks as well as optimisation of outcomes, whilst ensuring learning and contextual review. Otherwise, there is a risk that applications will be sub-optimally embodied in health systems with unintended consequences and without yielding intended benefits.

Item Type: Article
Additional Information: ** Article version: VoR ** From Elsevier via Jisc Publications Router ** History: accepted 04-08-2023; epub 12-08-2023; issued 31-10-2023. ** Licence for VoR version of this article starting on 07-08-2023:
Uncontrolled Keywords: Artificial Intelligence (AI)Health information technology eHealth Evidence Evaluation
Subjects: Q Science > QA Mathematics > QA76 Computer software
R Medicine > RA Public aspects of medicine > RA0421 Public health. Hygiene. Preventive Medicine
Divisions: Institutes and Academies > Institute of Management and Health > Business, Finance and Management
SWORD Depositor: JISC Publications Router
Depositing User: JISC Publications Router
Date Deposited: 23 Aug 2023 08:15
Last Modified: 22 May 2024 14:15

Administrator Actions (login required)

Edit Item - Repository Staff Only Edit Item - Repository Staff Only