Framework of Developing Real world Evidence (RWE)

There has been a burgeoning bid about the use of Real-World data in healthcare. The National Institute for Health and Care Excellence (NICE) has released the NICE real-world evidence framework on 23rd June 2022 which comprehensively deals with the use of real-world data to remove gaps in data, reduce uncertainty, and improve guidance.
Real-world data also known as non-randomised data, is collected outside the context of controlled clinical trials. The major sources of real-world data are electronic health records, administrative data, claims data, patient generated data, patients’ registries, audit and service reviews, observational cohort with primary data collection, health surveys, interviews etc. The need for RWE arises from the difference between the health outcomes expected and those realized from conventional RCTs.

  • The NICE has been using real world data across various programmes, especially where the effects of an intervention are not to be determined. Examples include, among others:
  • Patient generated data, characterizing health conditions, interventions, care pathways, and patient outcomes and experiences including natural history used multiple sources of real-world data to characterize spinal muscular atrophy,
  • Estimating economic burden reported data from Clinical Practice Research Datalink (CPRD) GOLD linked to Hospital Episode Statistics (HES),
  • Designing, populating, and validating among others.

Real-world data can also be used to augment and contextualize the randomized trial results to patient in the National Health Service (NHS) and estimate intervention effects. For newly developed interventions, the use of real-world data can be contended since it can be used to create a comparator arm or can be added as a control to Randomized Controlled Trials (RCTs). Besides, the real-world data enables using data from early access to medicines scheme, cross country comparison for technologies that are introduced prior to the UK.

The document lays down the framework for real world data usage under various heads. Under the conduct of quantitative real-world evidence studies, the NICE prescribes approaches that need to be followed while planning, conducting, and reporting the real-world evidence studies. The principles in this regard mainly revolve around the data provenance, data relevance and data sufficiency. Another principle that has been in focus is the need for transparency and integrity throughout the various stages of generating evidence, starting from planning to final reporting.

A general prescription that NICE has given is that patients should be consulted throughout the evidence generation scheme.

Defining the Research question must include defining the key study variables that must include population eligibility, criteria, intervention, outcomes, covariates, cofounders, subgroups, and the target quantity to be estimated (like prevalence or the average effect of an intervention). The list is only representative and not exhaustive. The developers should pre-specify the objective of the study along with the protocol regarding data identification, collection, curation, study design, and analytical methods including the subgroup and sensitivity analysis (especially for the studies concerning comparative effects).

The framework remarks that the data collection should be carried out in a systematic, transparent, and reproducible manner. The data collection should follow a predefined protocol to ensure quality, integrity, and consistency of data. The framework also canvasses an outline for the target trial approach concerning methods for real-world studies of comparative effects. A crisp representation of the guidelines is represented in the figure below.


Summary representation of planning and reporting cohort studies using real world data

As per the framework, the study design should reflect the following characteristics:

  • Nature and distribution of the outcome variable
  • Sample size
  • Structure of the data including data hierarchies or clustering (for example, patients may be clustered within hospitals or data may be collected on a patient at multiple timepoints)
  • Heterogeneity in outcomes across population groups
  • Whether data is cross-sectional or longitudinal.
  • The document outlines the common considerations under sensitivity analysis must include:
  • Varying operational definitions of key study variables
  • Differing time windows to define study variables and follow up
  • Use of alternative patient eligibility criteria
  • Address to missing data and measurement error
  • Alternative model specifications
  • Address to treatment switching or loss to follow up
  • Adjustments for non-adherence

he NICE framework necessitates, a quantitative bias analysis should be adhered to in case the residual bias remains high. It is required that while reporting results, the information should be presented in form flow diagrams representing each stage and aspect of evidence generation, patient characteristics across groups or levels of exposure, and differences in patient characteristics in analytical sample and target population. Communicating the real-world evidence studies can be challenging given the highly complex and extensive use of scientific terminology. Thus, as per the framework released, the studies should be documented in as simple a manner as possible that are easy to interpret with relevant explanations of the scientific terminology.

The NICE framework provides the “Data Suitability Assessment Tool (DataSAT)” that may be used to provide consistent and structured information on data suitability.

The document has included case studies on the reporting on methods of minimizing the risk of bias and reporting information on selected analytical methods. Interestingly, the framework was developed after thorough consultation and feedback from all stakeholders including patient organizations, health charities, healthcare professionals, the pharmaceutical and medical technologies industries, data controllers and contract research organizations, academia, international health technology assessment bodies, UK health system partners and NICE committee members.
The RCTs remain the gold standard for determining the efficacy of medicines, medical devices, and therapies. However, given the high cost and ethical issues with the use of RCTs, it is the need of the hour to recognise non-randomised data as a potential and effective way of evaluating the efficacy and safety of the new interventions. The NICE framework with respect to the use of real-world data and generating evidence is a step much awaited and is welcomed by the stakeholders. However, there is a need for proactively furthering the use of real-world data to suit our requirements and improve the quality of the research and development of new interventions.

Authors – Jyoti Sharma and Kunal Hriday