Indirect comparisons and significant benefit: COMP expectations at orphan maintenance

In last month’s blog on EU orphan maintenance, we discussed how sponsors can plan effectively for the review of orphan maintenance  during the MAA, with particular emphasis on early regulatory engagement and evidence strategies that will enable significant benefit over authorised ‘satisfactory methods’ to be demonstrated.

In this month’s blog, we take a closer look at the use of ‘indirect comparisons’ to demonstrate significant benefit; these are often required when direct trials versus authorised therapies are unfeasible. We explore:

• The main forms of indirect comparisons used in orphan maintenance submissions
• Why naïve cross-trial comparisons are commonly challenged
• How adjusted comparisons can strengthen credibility when data allow
• What sponsors can learn from real-life examples in published COMP orphan maintenance assessment reports.

Indirect Comparisons: A Necessary Tool and a Regulatory Challenge

Direct, head-to-head trials vs authorised methods are the most robust method for demonstrating significant benefit, but these are often unfeasible in rare diseases. The COMP may therefore accept alternative comparative approaches such as indirect or qualitative comparisons.

A recent retrospective analysis of the EMA orphan maintenance procedures between 2012 and 2022 evaluated 418 individual comparisons derived from 151 orphan maintenance assessments with a positive COMP opinion at the time of marketing authorisation (Windfuhr et al., 2024). Each maintenance procedure included one or more comparisons against authorised satisfactory methods, reflecting the requirement to demonstrate significant benefit at the time of marketing authorisation.

• 18% direct comparisons
• 44% indirect comparisons
• 39% qualitative comparisons

Indirect comparisons are quantitative comparisons made when no head-to-head trials exist. These range from simple, naïve side-by-side comparisons of outcomes from different studies to more formal inferential methods that adjust for study- or population-level differences. They may use contextual or real-world comparator data to demonstrate:

• Superior outcomes relative to an authorised therapy
• Fewer adverse events or discontinuations in clinical practice
• Reduced hospitalisation, improved adherence, or feasibility of home administration

Qualitative comparisons are narrative justifications of significant benefit when formal numerical comparisons are not feasible. These typically arise when the comparator’s authorised indication only partially overlaps with the proposed indication, meaning significant benefit can be demonstrated for patients who would not be eligible for existing therapies. In oncology, these arguments most often relate to treatment lines (for example, use as a later-line treatment or in refractory populations). In non-oncology indications, qualitative comparisons more commonly reflect partial or absent overlap of indications or adjunctive use, such as therapies authorised for specific genetic subtypes, age groups, or disease manifestations, or products intended to be used in combination rather than as replacements for existing treatments.

Types of Indirect Comparisons

In the analysis conducted by Windfuhr, indirect comparisons were the most common approach used to compare the new orphan medicinal product to a relevant comparator. The indirect comparisons fell broadly into two categories:

• 71% - Naïve side-by-side (SBS) comparisons
  Tabulating outcomes from separate studies without statistical adjustment
• 29% - Inferential indirect comparisons
  Adjusted analyses such as matching adjusted indirect comparisons

Among the indirect comparisons, naïve SBS comparisons were most often used whereas inferential approaches that adjusted for population differences or quantified the uncertainty of the comparison, either using or not using individual patient data, were less often used.

However, over the 10-year period of the analysis, inferential methods nearly doubled in use, reflecting a growing effort by sponsors to move beyond unadjusted cross-trial comparisons.

COMP Appraisal of Comparison Methods

The acceptability of a comparison by the COMP was found to depend on both the comparison method employed and the underlying data. When a comparison was accepted by the COMP, its assessment reflected acceptance of the method in the specific regulatory context, rather than a general endorsement of the methodology.

Among the comparison types assessed, qualitative comparisons demonstrated the highest relative frequency of acceptance, followed by direct comparisons. In contrast, indirect comparisons were accepted less frequently overall. Within the group of indirect comparisons, the proportion of rejected comparisons was highest for inferential approaches based on aggregate external data. However, the majority of methodological objections explicitly raised by the COMP were observed for the frequently used naïve SBS comparisons, reflecting the COMP’s concerns about the methodological limitations of unadjusted cross-trial comparisons.

Further analysis showed that orphan maintenance assessments for which the COMP raised questions significant benefit were characterised by a higher proportion of indirect comparisons and a lower proportion of direct and qualitative comparisons. In general, applicants most often addressed the COMP’s issues by applying new methods to the same data, suggesting that indirect comparison strategies may require further refinement during regulatory review.

Oncology vs non-oncology: different therapeutic areas, different expectations

Another finding of the Windfuhr analysis was the difference between maintenance reports for oncology and non-oncology indications. In the reports that included indirect comparisons, nearly half of these were SBS, with non-oncology indications making up just over 10% of these. Non-oncology indications were more likely to use inferential methods that adjust for population differences, or to rely on qualitative or direct comparisons.

This pattern indicates that sponsors of oncology products have tended to rely more heavily on pragmatic SBS approaches, perhaps reflecting the complexity of treatment lines and rapidly changing standards of care. For non-oncology indications, in which treatment pathways are often more stable, there has been greater use of adjusted methods and of qualitative positioning based on differences in indication, line of therapy or adjunctive use of available treatments.

Case studies

The following two case studies illustrate how COMP interprets indirect evidence in practice.

Takhzyro:   When indirect comparisons support significant benefit and patient-care value also supports the orphan maintenance opinion (EMA, 2018)

Takhzyro (lanadelumab) is an orphan medicine for routine prevention of recurrent hereditary angioedema (HAE) attacks in patients aged 12 years and over. In the absence of head-to-head trials versus authorised long-term prophylaxis options, the sponsor supported significant benefit using indirect comparisons versus plasma-derived C1-INH products (Cinryze and Berinert 2000/3000); the sponsor also justified major contribution to patient care (MCPC) based on dosing convenience.

The sponsor’s approach:

• Conducted cross-trial comparisons between HELP (lanadelumab), CHANGE (Cinryze), and COMPACT (Haegarda/Berinert SC) and described key comparability limitations (e.g., parallel vs crossover designs; different durations; different analytical approaches).
• Applied parametric survival models to estimate hazard ratios and long-term attack-free probabilities for each product (lanadelumab vs Cinryze and lanadelumab vs Berinert SC).
• Extrapolated outcomes up to 60 months using parametric survival models and predicted survival curves, demonstrating numerically favourable long-term attack-free probabilities for lanadelumab vs Cinryze and numerically favourable predictions vs Berinert SC in some simulations.

The COMP evaluation:

• COMP concluded that, despite cross-trial limitations and uncertainty, the indirect comparison supported a clear clinical advantage of lanadelumab vs Cinryze, and it was considered to have overall better efficacy than Cinryze.
• For lanadelumab vs Berinert 2000/3000 (SC), COMP considered the overall efficacy of lanadelumab to be comparable but a superiority claim was not accepted, as most results as only simulated long-term attack-free rates differed.
• COMP also considered the less frequent SC dosing of lanadelumab (every 2 weeks, with possible extension to every 4 weeks in stable patients) vs Cinryze (IV every 3–4 days) and Berinert 2000/3000 (SC every 3–4 days, higher injection volume/reconstitution burden) to represent a major contribution to patient care; QoL data were viewed as supportive rather than the primary basis for the demonstration of significant benefit.
• The COMP’s positive opinion on maintaining orphan status was based on the results of the indirect comparisons and the major contribution to patient care, with three members expressing a divergent view.

Evrysdi:     Demonstrating significant benefit through inferential comparisons and patient-care considerations in a heterogeneous disease (EMA, 2021)

Evrysdi (risdiplam) is indicated for the treatment of 5q spinal muscular atrophy (SMA) in patients 2 months of age and older, with a clinical diagnosis of SMA Type 1, Type 2 or Type 3 or with one to four SMN2 copies. At the time of marketing authorisation, two authorised products existed in the EU orphan condition of SMA (Sprinraza [nusinersen] and Zolgensma [onasemnogene abeparvovec]). The COMP discussed significant benefit versus both medicinal products, but the analysis primarily focused on comparison versus nusinersen.

The sponsor’s approach:

• Versus Zolgensma, the sponsor asserted that Evrysdi targets a broader population, including patients with > 3 SMN2 copies (patients with four SMN2 copies were included in SUNFISH [Evrysdi] and JEWELFISH [Evrysdi] studies)
• Versus Sprinraza, the sponsor considered that improved efficacy, improved safety, and major contribution to patient care were demonstrated. Improved efficacy was demonstrated via an indirect comparison in Type 1 SMA vs data from FIREFISH (Evrysdi) and ENDEAR (nusinersen), including a naïve (unweighted) comparison and a matching-adjusted indirect comparison (MAIC). Results were considered to favour Evrysdi across common endpoints (e.g., overall survival and ventilation-free survival hazard ratios; motor milestone and CHOP-INTEND responder odds ratios)

The COMP evaluation:

• The COMP considered that improved safety could not be considered documented for the justification of significant benefit, in the absence of a quantitative comparison across the entire safety profile of the two compared products (Evrysdi versus Spiranza).
• The COMP considered the demonstration of major contribution to patient care related to oral daily administration versus intrathecal administration, including treatment setting, feasibility in certain populations, and preference/burden. The COMP noted that preference alone was insufficient and that PRO-driven evidence would be useful, but such data were not available at the time.
• Overall, the COMP concluded that Evrysdi targets a broader population than Zolgensma and that the sponsor provided a MAIC supporting improved survival and motor outcomes in Type 1 patients compared with nusinersen, constituting a clinically relevant advantage.

Key Takeaways

• Indirect comparisons are often unavoidable in orphan maintenance reports, but they remain one of the most closely scrutinised elements of the COMP assessment.
• Methodological credibility matters more than complexity. COMP acceptance is driven by whether the comparison is clinically meaningful, appropriate to the available data, and not by the use of increasingly sophisticated statistical techniques alone.
• Context matters. Disease area, existing standards of care, trial design constraints, and the evolving competitive landscape all influence how the demonstration of significant benefit argument is assessed.

Overall, as emphasised in last month’s blog, these observations reinforce the importance of integrating orphan maintenance requirements into clinical trial plans from an early stage of development, rather than treating maintenance as a procedural step at the end of the regulatory pathway.

References

EMA. (2018, December 17). Takhzyro—Orphan maintenance assessment report (initial authorisation. https://www.ema.europa.eu/en/medicines/human/EPAR/takhzyro

EMA. (2021, May 4). Evrysdi—Orphan maintenance assessment report (initial authorisation). https://www.ema.europa.eu/en/medicines/human/EPAR/evrysdi

Windfuhr, F., Larsson, K., Framke, T., & Lasch, F. (2024). Which clinical trial designs and statistical approaches have been used in assessments of orphan maintenance by the European Medicines Agency between 2012 and 2022? A cross-sectional study. BMJ Open, 14(12), e086171. https://doi.org/10.1136/bmjopen-2024-086171