Background and Aim

Materials
and
Methods

Implementation of Al technology within health organisations varies in success. This variation is partly due to organisational level barriers. This means that problems, and harm can occur not because a technology isn't useful, but because an organisation wasn't ready. This study aimed to identify what constitutes readiness to implement Al for health organisations as well as specific challenges and potential solutions.

We used an exploratory sequential mixed-methods design comprising three phases: (i) qualitative interviews with 20 UK-based experts across governance, clinical, regulatory, and digital domains; concurrently with a targeted literature review to generate a working definition and characteristics of AI readiness (ii) a framework analysis to consolidate risk and controls found in the first phase and (iii) Validation of the proposed framework via stakeholder survey (n=52) assessing agreement with the 35 candidate statements of risk and mitigation; and (iii) an expert workshop (n=34) and public workshop (n=45) to validate and operationalise findings.

We iteratively refined and grouped statements into a prototype AI readiness checklist and assurance framework.

Results:

Stakeholder interviews and literature analysis highlighted the heterogeneity in organisations, systems, technology, risk level, combined with the fast-moving pace technology make a single static state of “AI readiness” an inappropriate goal. Key themes arising included risk management, knowledge silos, and infrastructure.

A framework analysis allowed organisational levels risks and  mitigations to be collated and consolidated. These statements along with examples were evaluated for inclusion in a via survey. High levels of agreement were found for most statements but allowed refinement.

The professional  workshop allowed further iteration of statements, as well as matching controls to potential harms as well as cementing the health ecosystem’s desire for a consistent, adaptable AI implementation framework, ideally operationalised as a webtool targeted at senior organisation leaders, aligned with existing standards.

Public workshop: strong theme of building trust and understanding for both the public and professionals with diverging views around the level of transparency required – consistent with a risk based and proportionate approach, as many interventions are already based on trust or at least a trusted party able to effectively evaluate the system. Summary for of the public workshop can be seen in Figure 2.

In the resultant AI readiness checklist (AIRC) we propose 20 causes of harm across five areas, and  19 suggested controls, categorised across five types. Futhermore the checklist is designed to signpost to the controls to be considered for each cause of harm, and also signposts to more information (Figure 3).     

Discussion:

We examined what it means for organisations to be ready to implement AI, and how this can be operationalised. The initial scope was for secondary healthcare organisations, but since expanded due to high level of generalisability across health and care, as well as a surfaced need.

 

We have integrated the findings into a consultation paper and proposed AI-readiness checklist and prototype webtool.

Work to pilot and validate the tool for both health and social care is underway.

 

View the Prototype Checklist here: www.aireadiness.uk