Guidance on Noninferiority Trials

Non-Inferiority Trials Checklist

Item # Checklist Item (clients can use this tool to help make decisions regarding use of non-inferiority data in advertising claims)
What Must Appear in the Advertisement or Promotional System
3.1 Consistency with Terms of Market Authorization
3.2 Description of Interventions
3.7 Statistical Analysis
3.8 Conclusions drawn
What Must Appear in the Published Study for Claim Validation
3.1 Consistency with Terms of Market Authorization
3.2 Description of Interventions
3.3 Specification of the Non-inferiority (NI) Margin
3.4 Specification of Power and Sample Size to Minimize Type II Error
3.5 Patient Populations
3.6 Trial Conduct
3.7 Statistical Analysis
3.8 Conclusions drawn

1. Key Benefits:

If a therapy is not anticipated to be superior in terms of health gains, a trial can be designed to detect whether the therapy produces an effect that is not worse than the comparator within a margin of error. These designs, called non-inferiority trials, are increasingly being used for the regulatory approval of new therapies.

2. Key Pitfalls:

Non-inferiority trials create challenges for those who need to interpret the suitability of their findings for advertising intended to influence HCP decision making. First, they require assumptions about comparator performance, which rely on methods of quantitative synthesis such as meta-analysis. They must also rely on assumptions about what it is to be worse, and what margin of error may be achieved. They also use different approaches to statistical testing and interpretation. It may be especially difficult to reconcile the meaning of a “negative” trial that shows a new therapy is not non-inferior than established therapy. Additionally, unlike superiority trials, an underpowered non-inferiority trial may be more likely to produce an untrue positive result.

3. Managing pitfalls:

A checklist is provided to guide industry and to assist the PAAB staff in determining whether findings from the non-inferiority study may appear within advertising/promotional systems (APS). The checklist relates to factors specific to the reporting of non-inferiority studies that are important for determining their credibility and relevance to decision-making. Refer to the PAAB code for general factors relating to acceptability of a study. 

3.1 Consistency with Terms of Market Authorization

Principles:

Drug advertising should be consistent with the Health Canada approved Terms of Market Authorization (TMA). 

Rationale:

Advertising content which is inconsistent with the TMA, would contravene section 9.1 of the Food and Drugs Act.

Application: 

The observation must not contradict anything in the TMA with respect to magnitude, direction, or duration.

3.2 Description of Interventions

Principle:

The intervention being compared to must be a valid therapeutic choice with known consequences associated with its use.

Rationale:

Unavailable comparators are not relevant for decision-making. Comparators with limited evidence regarding their effect versus placebo can lead to unreliable estimates of non-inferiority.

Application:

Describe the interventions intended for each group in detail and whether the reference treatment in the non-inferiority trial is identical (or very similar) to that in any trial(s) that established efficacy, as well as how and when they were actually administered. The setting(s) and location(s) where the interventions were administered should also be described.

❐ 3.3 Specification of the Non-inferiority (NI) Margin

Principle:

The choice of a non-inferiority margin must make clinical sense and is essential to interpreting the relevance and credibility of the study findings.

Rationale:

Different non-inferiority margins can lead to different findings. For example, large non-inferiority margins, chosen after the fact, are susceptible to measurement error.

Application:

The study should describe how the NI margin was chosen and establish its clinical importance.  Non-inferiority conclusions drawn from a trial designed for superiority are inappropriate. A forest plot must be provided for estimates based on meta-analysis.

The NI margin must be selected a priori.

3.4 Specification of Power and Sample Size to Minimize Type II Error

Principle:

Type II error has heightened importance in non-inferiority trials and must be managed.

Rationale:

If sample size is inadequate, then a non-inferiority trial may lead to a false claim of a drug being non-inferior to a comparator when in fact it is worse.

Application:

A replicable method for determining sample size based on a non-inferiority criterion must be provided. If applicable, a description of any interim analyses and stopping rules should be provided as well as a description of whether these rules were related to a non-inferiority hypothesis.

3.5 Patient Populations

Principle:

The interpretation of a non-inferiority trial relies on similar participants exposed to comparator treatments in a similar fashion in historical trials. 

Rationale:

Differences in trial participant characteristics from differences in eligibility criteria, or differences in outcomes or interventions in historical trials, will lead to differences in the findings from a non-inferiority assessment.

Application: 

Differences in trial participant characteristics compared with previous trial(s) should be reported and explained. If trial participants differ, the description of differences should concentrate on deviations that are likely to modify response to treatments.

3.6 Trial Conduct

Principle:

The findings from non-inferiority trials are particularly sensitive to deviations from trial protocols.

Rationale:

Poor blinding (masking) and randomization during the trial increases the chance of a false claim of non-inferiority when in fact, the therapy is worse. This increases the chance of harm from advertising.

Application:

Provide a diagram depicting the flow of participants in the study. Specifically, for each group report the number of participants: randomly assigned, receiving intended treatment, completing the study protocol, and analyzed for the primary outcome.

Describe the method used to assign participants and whether allocation was blinded.

Describe whether those administering the interventions, and those assessing the outcomes were blinded to group assignments and whether a third active comparator arm was used to test internal validity.

3.7 Statistical Analysis

Principle:

Proper interpretation of a non-inferiority analysis requires an understanding of the methods used to create comparisons.

Rationale:

Unlike superiority trials, an underpowered non-inferiority trial may be more likely to produce an untrue positive result.

Application: 

For each analysis, provide the number of participants contributing to estimates of effectiveness. If the number is smaller than the intent-to-treat number, specify how the denominator was derived. (i.e. state from a per protocol analysis and associated criteria)

Both ITT and per protocol results should be assessed (and both should support the conclusion of non-inferiority).

Non-inferiority is established when the upper bound of the 1-sided 97.5% CI (corresponding to a 2-sided 95% CI) lies within the non-inferiority zone.

Forest plots to depict comparisons are encouraged. For example, a forest plot depicting effect sizes and NI margins for one or two-sided confidence interval approaches should be used (see Appendix for example).  Differences in means based on absolute risk difference are discouraged unless accompanied by a relative scale measure of the same effect.

3.8 Conclusions drawn

Principle:

Even with appropriate analysis, it is common to draw incorrect conclusions from non-inferiority trials.

Rationale:

Claims should accurately reflect study findings as per PAAB code section 2.3.

Application: 

When frequentist statistics (e.g. CI or p-values) are used, the suggested wording is “the test drug is no worse than the comparator” ,  “the test drug was not inferior to the comparator”, or “the test product was similar to the comparator”.

For Bayesian statistics (e.g. credibility interval), it is appropriate to conclude a probability that the test drug was “no worse than the comparator”. For example, “drug X is 98% likely to have been no worse than drug Y”.

Appendix

The following diagram guides interpretation of the statistical analysis. “M1” is the pre-determined margin of non-inferiority. “CI” is the 95% confidence interval. “NI” is non-inferiority.

X- axis: Control minus Test drug (i.e.: C-T)

Interpreting superiority claims from a non-inferiority trial:

The entire ITT 95% CI for the primary or co-primary endpoint:

  • Does not cross/touch the pre-determined margin of non-inferiority (M1 in the figure above)
  • AND  is on the side of zero which indicates superiority (does not touch zero). Replace “zero” with “one” when assessing ratios rather than means/medians.

In the event that the non-inferiority study failed to demonstrate superiority in the primary endpoint, one should not use secondary endpoints to suggest superiority of the product. 

The study description/parameters should disclose that this was a non-inferiority study (can appear in a footnote).

Note, however, that it is not appropriate to conclude non-inferiority (or similarity) based on non-significant test result in a study designed only for superiority.  The appropriate interpretation of this observation is that the test product was not statistically significantly different versus the comparator.   

code.paab.ca/resources/Guidance_on_Noninferiority_Trials.pdf

TOPICS

PAAB Q&A

Do you have questions? We have answers!

Learn More

POPULAR TOPICS

Powered by Innovasium