The rising demand for evidence in digital health
The rising demand for evidence in digital health
Most digital solutions lack evidence. To realise their potential, innovators must demonstrate value to patients, clinicians, health systems and investors.
Key messages:
Most digital health solutions on the market lack robust evidence of safety and value. In 2022, 44% of digital health startups in the US had no regulatory filings or clinical trials published.
Regulation is increasing globally. Many countries are implementing increased levels of policy and guidance on evidence requirements for digital solutions.
Venture capital funding in digital health is down 68% from its high (Q4 2022 data, US). VCs are more selective and looking for more evidence from digital health companies.
Innovators must generate stronger evidence to meet requirements and demonstrate value to key stakeholders, including investors, health systems, clinicians and patients. This includes: 1) clinical, 2) financial, 3) operational, and 4) experience value.
Digital health innovators need a strategic approach to evidence generation, tailored to each solution and intended market. Many require guidance to navigate producing the right evidence at the right stages in a rapidly changing landscape.
Evidence is critical in healthcare
Throughout most of history, the concept of evidence as we now see it had little place in the practice of medicine. Today, scientific thinking and medicine are inextricably connected. The evidence-based medicine movement that emerged in the 1990s has placed significant emphasis on statistical data and rigorous clinical trials—particularly randomised-controlled trials (RCTs)—in the determination of best clinical practices. The evidence-based medicine model is now prevalent in medical education and has been instilled into the thinking of most physicians. This emphasis on evidence, however, has not widely permeated into the digital health field.
Digital health is not evidence-based
There has been significant growth in the digital health sector over the past decade. However, producing strong evidence for digital health solutions is a significant challenge for developers, and remains a hurdle to the wider adoption of potentially impactful products.
A large proportion of new technologies are still not supported by robust evidence for their safety, stated outcomes, or their value to patients and the health system. In 2022, a study conducted by Johns Hopkins University together with Rock Health examined digital health startups in the United States and revealed that almost half of these companies (44%) had no regulatory filings or published clinical trials for their solutions (Day et al., 2022). To learn more, watch the webinar we recently organised together with our friends at Roche Information Solutions and the senior author of the study, Dr Simon Mathews.
Evidence is needed across the product life cycle
In the context of digital health solutions, “evidence generation” is a broad term for the process by which various types of evidence are produced to support product development and validation. Many types of evidence can be generated at any stage; however, certain types of evidence are more strongly associated with specific stages. For example, product development is informed from the earliest stages by techniques such as secondary research (reviewing existing research, which helps innovators to better understand a clinical problem), user research (which can help validate a solution concept), and A/B testing (which allows comparison of different versions or features). Later, when developers have a well defined product, clinical data demonstrating safety may be critical for regulatory certification, and economic analyses will be important for showing evidence of value to health systems in order to sell a solution.
Figure 1. Examples of evidence generation at different stages of the product life cycle.
Different stakeholders need different evidence
Demonstrating the safety and value of digital health solutions is crucial. Safety is the foremost consideration. Digital solutions that directly inform or impact patient care require rigorous evaluations of clinical safety and performance to assess the risk of harm. Solutions of this kind are classified as a medical device, or ”software as a medical device” (SaMD). Obtaining regulatory approval is necessary for these products before going to market. However, this alone is unlikely to be enough to support adoption of the solution in the health system.
Health system decision-makers need to see good evidence for the value of a digital solution before they reimburse its use. The most relevant areas in which value must be demonstrated depend on the solution and its intended use, but generally include clinical, financial, operational and experience value, as shown in the graphic below.
Figure 2. Areas of value delivered by digital health solutions and examples of key audiences.
Proper evaluation of digital solutions ensures that they deliver value to users (usually patients or clinicians) and to the wider health system. Different types of value are demonstrated using different methodologies. For example, clinical value may be demonstrated by evaluations showing a positive impact on clinical outcomes, while financial value is demonstrated by economic analyses to ensure value for money for those who pay for healthcare (including taxpayers).
Consider the main decision-makers in healthcare, and why evidence is important to them:
Health systems
We are beginning to see increasing levels of regulation, policy and oversight in the digital health solutions market. In the United Kingdom, for example, NICE’s Early Value Assessment (EVA) aims to generate early signals about which digital solutions can deliver the greatest value to the NHS. EVA evaluates solutions addressing significant unmet needs in the health service. Final guidance on a topic includes an overview of existing evidence gaps and recommendations on specific areas that further evidence generation should focus on, such as specific patient groups or outcome measures. In the United States, the FDA has issued increasing levels of detail on the regulatory oversight of digital technologies. All of this activity aims to ensure that digital solutions are safe and effective. As a result, developers now need to produce more evidence for the safety and value of their solutions. This is essential to ensure adoption by health systems at scale.
Clinicians
Clinicians play a pivotal role in driving the adoption of new solutions, and it is therefore essential to demonstrate strong evidence for the value of a solution to a clinical audience. Developers must invest in generating evidence for a solution’s clinical performance, usability and its impact on patient outcomes. This may be generated from clinical trials, usability testing and other forms of evaluation.
Investors
Figure 3. Quarterly venture capital funding for digital health in the United States (Q3 2019–Q4 2022).
There has been a significant slowdown of VC funding in the industry. In 2022, total VC funding for US digital health startups was approximately half that of 2021. Investors are taking a more cautious approach to backing startups, and are looking for much stronger evidence of value showing a solution's potential to succeed in a market that is both more competitive and more tightly regulated.
Patients
Patients are the key stakeholders in healthcare and their confidence is critical to the success of any innovation. Evidence is crucial when it comes to ensuring that digital solutions deliver safe and high-quality care for patients. Without rigorous evaluation, there is a risk that solutions may cause harm, directly or indirectly.
Digital health needs a new approach to evidence
For digital health solutions to be successful and achieve their potential impact, innovators must develop a strategic approach to evidence generation that is appropriate for their product and the intended markets. Many companies require guidance and support to produce the right evidence at the right stages of their journey to market in a landscape that is changing rapidly.
Prova Health supports digital health innovators with evidence generation. To discuss how we can help with evidence generation for your digital solutions, email hello@provahealth.com
Dr Des Conroy is a Digital Health Consultant at Prova Health. He is a medical doctor who has worked in clinical practice in the UK and Ireland. He has experience developing and clinically validating artificial intelligence-based Software as a Medical Device (SaMD) products and supporting their deployment on a global scale. At Prova Health, he has led research into evolving evidence standards and reimbursement models in digital health.
Dr Saira Ghafur is Co-founder and Chief Medical Officer of Prova Health. She is an honorary consultant Respiratory Physician at St Mary’s Hospital, London, and a digital health expert who has published on topics such as cybersecurity, digital health adoption and reimbursement, data privacy and commercialising health data. She is Co-founder of mental health start-up Psyma and holds a MSc in Health Policy from Imperial. She was a Harkness Fellow in Health Policy and Practice in New York (2017).
References
Day, S., Shah, V., Kaganoff, S., Powelson, S. and Mathews, S.C., 2022. Assessing the clinical robustness of digital health startups: cross-sectional observational analysis. Journal of medical Internet research, 24(6), p.e37677.
Bryant, K., Knowles, M. and Krasniansky, A., 2023. 2022 year-end digital health funding: Lessons at the end of a funding cycle. Available at: rockhealth.com/insights/2022-year-end-digital-health-funding-lessons-at-the-end-of-a-funding-cycle/ (Accessed: 2nd February 2023).