Generating evidence for AI in healthcare
Generating evidence for AI in healthcare
Update (Jan 2024): For detailed insights on the evidence needed for AI solutions in healthcare, read our published white paper. Original post continues below.
A new white paper in development by Prova Health aims to clarify the evidence required to support AI solutions in healthcare.
Prova Health is developing a new white paper examining the unique evidence requirements of artificial intelligence (AI)-based solutions in healthcare. This report is the product of a research process that has included a review of the relevant literature and one-to-one interviews with leading experts in this field.
As well as providing important insights for AI developers, healthcare systems and providers, it is our hope that this paper will act as a catalyst for a much-needed conversation on evidence generation for AI solutions, and we look forward to consolidating our thoughts as the conversation progresses.
Key insights emerging from this work
What is unique to AI solutions is that they require both model evidence (evidence for the underlying algorithm) and solution evidence (evidence for the product in which the algorithm is embedded). Models will need validation first on internal and then on external datasets (internal and external validation).
The AI model will be part of a digital solution/product. Once a model has been internally and externally validated from a data perspective, the solution as a whole needs to be evaluated.
Given their likely use to support decision making in healthcare, it is critical for AI developers to show that their solution does not reinforce or exacerbate existing biases and inequities.
Evidence is required throughout the product life cycle (product development; regulatory approval; market access/payment; post-market surveillance): different types of evidence for different aims and to inform different stakeholders.
The evidence required to enter markets (e.g., FDA/CE/UKCA marking) will differ depending on the class of the tool and may require additional evidence generation steps.
AI developers will face an evidence limbo between the early stage evidence that is sufficient for product development and the evidence needed at later stages of the product life cycle to demonstrate specific outcomes. Innovative evidence generation methodologies like clinical simulation can help bridge that gap.
Once the solution is in use, there are significant opportunities to generate real-world evidence to demonstrate its value.
Ultimately, commercial success will depend on whether a product can deliver value for money for a client. Economic modelling and analysis evaluating the benefits against the costs of a solution is essential.
For more insights, read our draft white paper, and please contribute to the discussion with your thoughts and feedback!
Prova Health supports digital health innovators with evidence generation. To discuss how we can help with evidence generation for your digital solutions, contact us today.