Overview of our services in the field of non-randomised studies and indirect comparisons

  • Strategic consulting
  • Systematic research
  • Study design consulting
  • Statistical consulting

Non-randomised studies and non-adjusted indirect comparisons in benefit assessments

Benefit assessments, at EU level as well as full assessments in Germany, are used to compare the efficacy and safety of a health technology with other health technologies or other existing medical procedures. Therefore, randomised controlled trials (RCTs) are considered the gold standard. However, it is often difficult to conduct RCTs, particularly for orphan drugs (ODs) and advanced therapy medicinal products (ATMPs). Consequently, non-randomised studies (NRS) or single-arm studies are used for approval. If studies are non-randomised, there may be an unequal distribution of confounders between the treatment arms, which can influence the treatment outcome. The same applies to a non-adjusted indirect comparison of several single-arm studies or studies with external historical control cohorts.

To enable a comparison nevertheless, EUnetHTA21 provided method papers D4.3.1 and D4.3.2 on direct and indirect comparisons, which describe the required statistical methods. Moreover, in the German AMNOG procedure, non-adjusted indirect comparisons with an external historical control cohort can be used to derive a quantifiable added benefit if certain quality criteria (Figure 1) are met as was seen in the reassessment of Cerliponase alfa 2022.

Figure 1: Minimum requirements for a non-adjusted indirect comparison with an external control cohort

A systematic literature search must be carried out to identify potential external control cohorts. High-quality patient-specific data from precisely defined, representative populations are preferable. If no previously published natural history cohort is available, the establishment of a patient registry prior to or in parallel with the clinical trial of the health technology to be assessed may be an option.

A major challenge in non-adjusted indirect comparisons is ensuring structural equality regarding possible confounding factors within the analysed cohorts . Both known and unknown confounding factors can influence the result and must be considered. Therefore, first all known confounding factors must be identified with the help of a systematic literature search.  Thereafter, these must then be adjusted for in the scope of predefined statistical analyses. Various statistical methods are available for this purpose. As all known confounding factors must be considered in the statistical analyses, they should be considered in the study design and in the selection of the external control cohort. Therefore, early planning is necessary.

In addition to the already known confounders, there is the possibility of the influence of potentially unknown confounders. Therefore, the observed treatment effects must be of a magnitude that cannot solely be explained by the influence of these potential confounders (see Chapter 3.2.2 of IQWiG’s General Methods on the “dramatic effect”).