Age-related macular degeneration (AMD) is the leading cause of severe vision loss in people over the age of 65 in the United States, with more than 1.6 million people currently affected by advanced AMD.1 In light of the recent publication of the 1-year interim results of the Comparison of Age- Related Macular Degeneration Treatments Trial (CATT),2 a multicenter, randomized clinical trial designed to assess the relative safety and efficacy of two treatments for subfoveal neovascular AMD, a closer look at what it means to conduct a noninferiority study is warranted. Recently, there has been a growing interest in comparative treatment efficacy studies for both medical and economic reasons.3 Such studies typically use a noninferiority paradigm, as was done in CATT.4,5<.sup>

THE BASICS OF A NONINFERIORITY STUDY

The US Food and Drug Administration's (FDA) regulation on adequate and well-controlled studies (21 CFR 314.126) describes four kinds of controlled trials that provide evidence of efficacy: placebo-controlled, no treatment controlled, dose-response controlled, and comparison with an active treatment. Each of the first three study designs is intended to show that the investigational therapy is superior to the control treatment. The fourth type of trial, active control, can also be used as a superiority trial. More frequently, however, the goal of these studies is to show that the investigational treatment is equivalent to the active control, which for the purposes of statistical testing translates to “not worse than” or “noninferior.” The fundamental assumption of such trials is that if the active control has previously been shown to be efficacious, and the investigational treatment is comparable to the active control, the investigational treatment must therefore be efficacious as well.

The most considerable difference between superiority and noninferiority trials is that a well-designed and properly conducted superiority trial is interpretable without requiring additional information, assuming the study can show a difference between drugs.3 On the other hand, a noninferiority study can only be interpreted if the active control has demonstrated its expected effect. This means that the trial was properly designed and conducted so that it could identify an effective drug, demonstrating that the trial had adequate assay sensitivity. If the active control did not demonstrate the expected effect, the framework used to evaluate the efficacy of the investigational treatment is flawed.

The design (and subsequent interpretation) of a noninferiority trial relies heavily on external information, typically the performance of the active control in previous superiority studies. Because of this reliance, the design of the historical trials and new noninferiority studies should be as close to identical as possible in all major aspects. Some important design features include study population, concomitant therapy, entry criteria, endpoints, and dose of active control.3 There are two important margins that must be chosen for a noninferiority trial. M1 is the common identifier for the expected effect size of the active control, which typically should not be greater than the smallest effect size that the active control is expected to have.6 The determination of M1 is generally a statistical issue, based on balancing all previous efficacy information available for the active control. The second margin, commonly known as M2, represents the allowable difference between the active control and the investigational product (this is the noninferiority margin itself). The value of M2 is a matter of clinical judgment; the maximum value it can take should be the largest loss of treatment effect that would be clinically acceptable.3 Often, it is desirable to choose a smaller margin (M2) to ensure that the drugs are truly comparable, but at the very least M2 should never be greater than M1 because it would then be impossible to demonstrate that the test drug had any effect. Setting the margins is typically an arduous task, the difficulty of which should not be underestimated—the level of work involved has been compared with the level of work that goes into a full New Drug Application.

MAKING CONCLUSIONS FROM NONINFERIORITY STUDIES

The FDA sets forth four major criteria for making valid conclusions (ie, assessing assay sensitivity) from noninferiority trials.6 The design and implementation of a noninferiority study involves: 1) historical sensitivity of evidence to drug effects; 2) a trial design similar to the design of the trials used to determine that historical evidence of sensitivity to drug effects exists; 3) high quality trial conduct similar to that which was used in the previously designed trials; 4) an acceptable noninferiority margin, taking into account the historical data and relevant clinical and statistical considerations. It is important to evaluate the historical evidence of sensitivity to drug effects before beginning a noninferiority trial. This determines that, in the particular therapeutic area under investigation, previously designed and conducted trials that used the same active treatment reliably showed an effect.6

A noninferiority trial is dependable only when the historical estimate of drug effect size can be supported by results of previous studies; the estimate of effect size establishes a reasonable starting point for estimating its effect in the noninferiority study, assuming the studies are conducted under the “constancy assumption.”3 This is the conclusion that the noninferiority study is sufficiently similar to past studies with respect to all important study design and conduct features. The conduct of the trial should also be of high quality (compliance to study protocol, high follow-up rate) and adhere closely to that of the historical trials, as poor conduct can lead to an apparent finding of noninferiority that is incorrect.3

PROS AND CONS OF NONINFERIORITY STUDIES: DESIGN AND INTERPRET CAREFULLY

Theoretically, a noninferiority study design yields two comparisons: a direct comparison of the test drug with the active drug and an indirect comparison of the test drug with placebo (based on previous studies comparing the active drug with placebo).3 In addition to comparative effectiveness interest, noninferiority studies are also used for ethical considerations. These studies reduce the ethical concerns that arise from failure to use drugs with documented important health benefits.6 Noninferiority designs are chosen when it would not be ethical to use a placebo because there is an effective treatment that provides an important benefit available to patients.3 However, difficulty specifying the noninferiority margin and ability to detect assay sensitivity are two major concerns when conducting noninferiority trial. Noninferiority trials must rely on an estimate of assay sensitivity based on information external to the trial, and there is no direct assessment of effect size. There are a number of factors that can reduce assay sensitivity, including poor patient compliance and poor diagnostic criteria. Additionally, because noninferiority trials often choose a conservative noninferiority margin, they may require considerably larger sample sizes.6

Considering these pros and cons will help determine whether or not the noninferiority design is best suited for a specific trial. Furthermore, being aware of the issues and accounting for them appropriately can help decrease the potential difficulties inherent with these studies. In anticipation of future noninferiority studies, understanding their design and intent will lead to better interpretation of results.

Aron Shapiro is Vice President of Retina; Michael Byrnes, PhD, is an Analyst and Scientific Writer with the Retina Department; and Ashley Lafond is a medical writer at Ora, Inc., in Andover, MA.

  • Friedman DS. Vision Problems in the U.S. Prevalence of Adult Vision Impairment and Age-Related Eye Disease in America. Prevent Blindness America S, IL:2002.
  • The CATT Research Group. Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med. 2011;364:1897-1908.
  • Draft FDA Guidance for Industry: Non-Inferiority Clinical Trials. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidance s/UCM202140.pdf. Accessed April 14, 2011.
  • French Evaluation Group Avastin Versus Lucentis (GEFAL). http://clinicaltrials.gov/ct2/show/NCT01170767?term=non-inferiority&cond=retina&rank=6. Accessed May 18, 2011.
  • Pan-VEGF Blockade for the Treatment of Retinopathy of Prematurity (BLOCK-ROP). http://clinicaltrials.gov/ct2/show/NCT01232777?term=non-inferiority&cond=retina&rank=5. Accessed May 18, 2011.
  • FDA Guidance for Industry: E.10 Choice of Control Group and Related Issues in Clinical Trials. http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm129460.pdf. Accessed April 14, 2011.