Retrospective assessment of non-inferiority in the rare disease, Guillain–Barre syndrome

doi:10.15406/bbij.2014.01.00009

eISSN: 2378-315X

Biometrics & Biostatistics International Journal

Research Article Volume 1 Issue 2

Retrospective assessment of non-inferiority in the rare disease, Guillain–Barre syndrome

Chunqin Deng,

Verify Captcha

Regret for the inconvenience: we are taking measures to prevent fraudulent form submissions by extractors and page crawlers. Please type the correct Captcha word to see email ID.

Junliang Chen, Kim Hanna

Grifols Therapeutics Inc., USA

Correspondence: Chunqin Deng, 79 TW Alexander Drive, Durham, North Carolina, USA, Tel 1 919-316-6440

Received: October 10, 2014 | Published: November 8, 2014

Citation: Deng C, Chen J, Hanna K. Retrospective assessment of non-inferiority in the rare disease, Guillain–Barre syndrome Biom Biostat Int J. 2014;1(2):37-42. DOI: 10.15406/bbij.2014.01.00009

Download PDF

Abstract

Background: Non-inferiority testing is used to demonstrate that a new treatment is not unacceptably worse than an existing treatment. Such analyses are useful, for example when placebo-controlled studies are unethical, or when there may be other considerations (e.g. convenience, cost) where the new treatment has an advantage. Prospective, non-inferiority trials in orphan diseases are difficult to coordinate because they often require large sample sizes to detect small margins of difference between treatments. In this report, we present a retrospective study of non-inferiority testing in the rare disease, Guillain–Barre syndrome (GBS).

Methods: Meta-analysis results of PE versus a control group (n=623) were used to derive non-inferiority margins for two endpoints:

Improvement of ≥1 grade on the GBS disability scale,
Mean change from baseline on the GBS disability scale. These were then retrospectively applied to meta-analysis results to demonstrate the non-inferiority of IVIg to PE (n=567).

Results: For endpoint 1, the non-inferiority margin of the risk ratio was 0.865. The risk ratio of IVIg versus PE was 1.08 (95% confidence interval CI: 0.94 to 1.23). Since the lower bound of the CI is above the non-inferiority margin (0.865), IVIg can be considered non-inferior to PE on this endpoint. For endpoint 2 assessing change from baseline on GBS disability scale, the non-inferiority margin was 0.315. The treatment difference (IVIg – PE) was -0.02 (95% CI:-0.25 to 0.20). Since the upper bound of 95% CI (0.20) is less than 0.315 (the non-inferiority margin), IVIg can be considered non-inferior to PE.

Conclusion: The results demonstrate non-inferiority of IVIg to PE in GBS when the non-inferiority margins are retrospectively applied. Retrospective non-inferiority analyses may also be used in evaluation of treatment effects for other rare diseases.

Keywords: non inferiority margin, orphan disease, guillain barre syndrome, plasma exchange, intravenous immunoglobulin

Background

Randomized, placebo-controlled trials are the standard method by which the efficacy of medical treatments is determined. However, in serious diseases where there is a known effective treatment, allocating one patient group to a placebo arm may be unethical. Furthermore, in some situations, a new treatment may not be expected to be more effective than an existing treatment on the primary endpoint, but may have advantages in terms of secondary endpoints, such as safety, convenience, compliance or cost.¹^,² The non-inferiority trial is a vital tool when evaluating the efficacy of a novel therapy compared with an existing therapy. It aims to demonstrate that the test product is not worse than the comparator by more than a pre-specified, small amount. This amount is known as the non-inferiority margin, or delta (Δ).¹ Guidelines developed by the European Medicines Agency and the US Food and Drug Administration³ recommend predefining the non-inferiority margin. This margin can be derived from previous studies using historical data, and the study medication is typically expected to retain at least 50% of the original treatment effect over placebo or the standard of care to be considered non-inferior.

After a non-inferiority margin is established, a prospective non-inferiority trial is usually conducted to confirm the non-inferiority of the new product when compared to the existing product. Non-inferiority trials typically require considerably larger sample sizes than placebo-controlled trials.⁴ This is due to the fact that the margin of equivalence (non-inferiority) is often much smaller than the treatment difference, which a placebo-controlled trial must be powered to detect. It is therefore important for non-inferiority trials to have large sample sizes, and for this reason, trials of orphan drugs in rare diseases face significant challenges in terms of recruiting sufficient sample sizes to formally assess prospectively defined non-inferiority and of completing the trial within a realistic timeframe.

Here, we present a practical method for demonstrating non-inferiority of drugs for rare disease. This method is based on aggregated data from smaller studies that have been analyses in previously published meta-analyses,⁵^,⁶ and is illustrated using the example of intravenous immunoglobulin (IVIg) compared with plasma exchange (PE) for the treatment of Guillain–Barre syndrome (GBS).

GBS is a rare inflammatory disease affecting the peripheral nerves and causing weakness, numbness, breathing difficulty and paralysis. The disease affects between 0.5 and 2 per 100,000 persons per year.⁶ Although still under investigation, the cause of GBS is believed to be an autoimmune response.⁵^,⁶ In some patients, the condition can have a lasting impact after the end of its acute phase.⁵ Supportive care for GBS can include the administration of heparin, and the use of pressure stockings to prevent the onset of deep vein thrombosis in bed-bound patients, along with the monitoring of pulse, blood pressure, autonomic disturbances and respiration. Rehabilitation focuses on exercise to encourage strengthening, proper limb positioning, posture and orthotics.⁷

There are two effective immune therapies for GBS: PE, which involves separation of plasma from cells and re-infusion of those cells back into the patient, and IVIg, which uses antibodies purified from plasma that has been pooled from at least 1000 donors.⁵^,⁶ Administering IVIg is simple compared with PE. PE requires access to two veins, of which one has to permit high flow volumes, and frequently necessitates the insertion of a central venous line, a PE machine and specially trained personnel. IVIg requires access to only a single peripheral vein and no special equipment or specially trained staffs are necessary. Consistent with the difference in ease of administration, a Cochrane Review found that the risk ratio (RR) of treatment being discontinued was 0.14 less in the IVIg than in the PE group (95% confidence interval CI: 0.05 to 0.36). In addition, there is some evidence that adverse events are more frequent with PE than IVIg.⁵^,⁷

The clinical benefits of PE in GBS as measured by improvement on the GBS disability scale developed by Hughes et al.,⁸ have been confirmed in a Cochrane Review,⁶ which included six randomized, controlled trials (RCTs).⁹^–14 Few trials comparing IVIg with placebo have been conducted because PE was the standard of care when IVIg was introduced for GBS. However, a number of studies.15^–21 show that IVIg speeds recovery from GBS to a similar extent as PE, as concluded by a Cochrane Review.⁵

Due to the rarity of GBS, the majority of studies comparing IVIg and PE has used small sample sizes with limited statistical power and were not formally designed as therapeutic equivalence or non-inferiority trials.⁵ This may help to explain some inconsistency in the findings, and it is possible that some studies finding no significant difference between treatments reflect a lack of power to detect a significant difference rather than indicating true non-inferiority. The Cochrane Review by Hughes et al.,⁵ thoroughly reviewed all individual studies and performed a meta-analysis, but did not formally assess therapeutic equivalence or non-inferiority. The conclusion of no treatment difference cannot be automatically translated into either equivalence or non-inferiority.

Given the strong safety profile of IVIg,22 as well as the convenience of its use in the clinic,15 the current analysis was undertaken to formally establish the non-inferiority of IVIg to PE using existing studies from comparisons of PE versus supportive care, where much more data are available. A Cochrane Review of the benefits of PE in GBS⁶ was used to establish the non-inferiority margin, and then this derived non-inferiority margin was retrospectively applied to results from a Cochrane Review of IVIg benefits in GBS⁵ to demonstrate the non-inferiority of IVIg to PE.

Methods

In non-inferiority trials, one of the critical steps is to define the non-inferiority margin. This margin can be derived from previous studies using historical data, and the study medication is typically expected to retain at least 50% of the original treatment effect over placebo or the standard of care to be considered non-inferior. In the example in GBS, the non-inferiority margin was derived using results from the meta-analysis of previous trials comparing PE versus supportive care.

Raphael et al.,⁶ conducted a meta-analysis of five studies (623subjects, a summary of the included trials is shown in (Table 1). The RR of PE versus supportive care for the proportion of subjects with improvement of at least one grade on the GBS disability scale was calculated as 1.64 (95% CI: 1.37 to 1.96) (Table 2). For the proportion of subjects with improvement of at least one grade on the GBS disability scale, the non-inferiority margin for the RR can be derived using the fixed-margin method or the two 95% CI approach.¹^,³^,23^–24 For the purposes of this study, the new treatment is IVIg and the active control is PE. The fixed-margin approach involves determining the treatment effect (M1) of the active control group over the placebo (or no treatment) group by using the lower bound (or upper bound, depending on the direction) of the 95% CI from previous placebo-controlled trials or meta-analyses of trials. i.e., M1 = 1.37 which is the lower limit of 95% CI of the RR. Typically, preserving at least 50% of M1 from active control versus placebo (or no treatment) is recommended.³ i.e., RR of IVIg versus no treatment is greater or equal to $1 + (M 1 - 1) \times 50 % = 1.185$ . The non-inferiority margin (M2) is excluded by ensuring that the lower bound (or upper bound, depending on the direction) of the 95% CI is >M2. i.e., RR of IVIg versus PE is greater than $M 2 = \frac{1.185}{1, 37} = 0.865$ .

	Trial design	Participants	Interventions	Endpoint	Notes
Greenwood¹¹	RCT, multicentre, open, parallel groups	n=29, acute GBS only All ages No mild forms	PE versus supportive care Five PE in 10 days, 55 mL/kg per PE	1, 2	Unblinded
McKhann⁹	RCT, multicentre, open, parallel groups	n=245, acute GBS only All ages No mild forms	PE versus supportive care Three to five PE in 5 days, 40 mL/kg per PE	1, 2	Unblinded SD of the mean was not available; mean difference could not be estimated
Osterman¹²	RCT, multicentre, open, parallel groups	n=38, acute GBS only Adults only No mild forms	PE versus supportive care Three to eight PE in 7 to 10 days, 3 L per PE	1	Alternate randomization Unblinded Disability scale used was different from that used by all other trials; omitted from analysis of endpoint 2
Raphael¹³	RCT, multicentre, open, parallel groups	n=220, acute GBS only Adults only All forms	PE versus supportive care Four PE in 8 days, 3 L per PE, diluted albumin or fresh frozen plasma	1, 2	Unblinded
Raphael¹⁴	RCT, multicentre, open, parallel groups	n=91, acute GBS only Adults only Mild forms	PE versus supportive care Two PE every other day, 3 L per PE, diluted	1, 2	Unblinded

Table 1 Trials of PE versus supportive care included in meta-analysis of endpoint 1 and 2⁶

GBS, guillain-barre syndrome; PE, plasma exchange; RCT, randomised controlled trial; SD, Standard deviation

PE	Control (supportive care)	Statistical test	Point estimate (95% CI)	M1 (PE/Control)	M2 (IVIg/PE)
Endpoint 1: The proportion of subjects with improvement by at least one grade after 4 weeks
176/308 (57.1%)	110/315 (34.9%)	Risk ratio	1.64 (1.37 to 1.96)	1.37	0.865
Endpoint 2: Mean disability grade improvement after 4 weeks
N=290	N=295	Mean difference	-0.89 (-1.14 to -0.63)	-0.63	0.315

Table 2 Meta-analysis results and derivation of M1 and M2

CI, Confidence interval; IVIg, Intra Venous immunoglobulin; PE, Plasma exchange

A further meta-analysis was also performed on four studies (585 subjects) (Table 2) to assess change from baseline to week 4 using the GBS disability scale (endpoint 2). The treatment difference (PE–supportive care) was calculated as-0.89 (95% CI:-1.14 to -0.63).⁶ For the mean change from baseline on the GBS disability scale, the non-inferiority margin for the treatment difference can be derived using the fixed-margin method or the two 95% CI approach.¹^,³^,23^–24 The fixed-margin approach involves determining the treatment effect (M1) of the active control group over the placebo (or no treatment) group by using the upper bound (or lower bound, depending on the direction) of the 95% CI from previous placebo-controlled trials or meta-analyses of trials. i.e., M1 =-0.63 which is the upper limit of 95% CI of the treatment difference. Preserving at least 50% of M1 from active control versus placebo (or no treatment) is recommended.³ i.e., treatment difference of IVIg-no treatment is less or equal to $M 1 \times 50 % = - 0.315$ . The non-inferiority margin (M2) is excluded by ensuring that the upper bound (or lower bound, depending on the direction) of the 95% CI is < $M 2 = - 0.315 - (- 0.63) = 0.315$ . The detailed derivation is shown below.

Results

Endpoint 1: Improvement of at least one grade on the GBS disability scale

The treatment effect (M1) for PE versus Control (supportive care) is defined as the lower limit of the 95% CI of the RR.

M 1 = \frac{P (P E)}{P (C o n t r o l)} = 1.37

(i.e. lower limit of CI ) (Table 2), where P is proportion of subjects with improvement of at least one grade on the GBS disability scale. Assuming a need to preserve 50% of the treatment effect of PE versus Control to show that IVIg is non-inferior to PE, the treatment effect of IVIg must be:

\frac{P (I V I g)}{P (C o n t r o l)} = 1 + (1.37 - 1) \times 50 % = 1.185

The non-inferiority margin (M2) for IVIg versus PE can be calculated as follows:
$\frac{P (I V I g)}{P (P E)} = \frac{[\frac{P (I V I g)}{P (C o n t r o l)}]}{[\frac{P (P E)}{P (C o n t r o l)}]} = \frac{1.185}{1.37} = 0.865$

Therefore, the non-inferiority margin of the RR is 0.865, and IVIg is non-inferior to PE if the lower bound of the 95% CI of the RR of IVIg versus PE is greater than 0.865.

Hughes et al.⁵ conducted a meta-analysis of six studies (567 subjects).¹⁵^–²⁰ An overview of the trials included is given in (Table 3). The RR of IVIg versus PE was 1.08 (95% CI: 0.94 to 1.23) for the proportion of subjects with improvement of at least one grade on the GBS disability scale. Since the lower bound of the 95% CI (0.94) is above the non-inferiority margin (0.865), IVIg can be considered non-inferior to PE on this endpoint (Table 4).

	Trial design	Participants	Interventions	Endpoint	Notes
van der Meche²⁰	Randomized, national, multicentre, parallel group	Adults and children N=150	IVIg 0.4 g/kg daily for 5 days versus PE 200 to 250 mL/kg over 7 to 14 days	1, 2	Unblinded
Bril¹⁶	Randomized, single-centre, parallel group	Adult N=50	IVIg 0.5 g/kg daily for 4 days versus PE 40 to 50 mL/kg on five occasions over 7 to 10 days	1, 2	Unblinded
PSGBS Study Group¹⁵	Randomized, international, multicentre, parallel group	Adult N=383	IVIg 0.4 g/kg daily for 5 days versus PE 250 mL/kg over 8 to 13 days versus PE followed by IVIg	1, 2
Diener¹⁷	Randomized, multicentre, parallel group	Adults (possibly children) N=74	IVIg 0.4 g/kg daily for 5 days versus PE 40 to 50 mL/kg on five occasions within 14 days versus immune absorption on five occasions (4000 mL on two occasions and then 2000 mL on three occasions) within 14 days	1, 2	Unblinded
Nomura¹⁹	Randomized, multicentre, parallel group	Adult N=47	IVIg (Teijin brand) 0.4 g/kg daily for 5 days versus PE total 200 to 250 mL/kg in up to seven sessions over 4 weeks	1, 2	Unblinded
El-Bayoumi¹⁸	Open, parallel-group, randomized, controlled trial	Children (age not specified) with GBS requiring artificial ventilation	IVIg 0.4 g/kg daily for 5 days versus one plasma volume PE daily for 5 days	1	Unblinded

Table 3 Trials of IVIg versus PE included in meta-analysis of endpoints 1 and 2⁵

GBS, guillain–barre syndrome; IVIg, intravenous immunoglobulin; PE, plasma exchange; PSGBS, plasma exchange/sandoglobulin guillain–barre syndrome

	IVIg	PE	Statistical test	Point estimate (95% CI)	Non-inferiority margin (M2)	Non-inferiority of IVIg versus PE
Endpoint 1: The proportion of subjects with improvement by at least one grade after 4 weeks
	177/293 (60.4%)	154/274 (56.2%)	Risk ratio	1.08 (0.94 to 1.23)	0.865	Yes
Endpoint 2: Mean disability grade improvement after 4 weeks
	N=273	N=263	Mean difference	-0.02 (-0.25 to 0.20)	0.315	Yes

Table 4 Meta-analysis results and determination of non-inferiority

CI, confidence interval; IVIg, intravenous immunoglobulin; PE, plasma exchange

Endpoint 2: Mean change from baseline on the GBS disability scale

The non-inferiority margin can be derived using the two 95% CI approach.¹^,.²³ Treatment effect (M1) for PE versus Control (supportive care) is defined as the upper limit of 95% CI of treatment difference.

M1 = PE - Control = - 0.63 (i.e. upper limit of CI) (Table 2)

To demonstrate non-inferiority of IVIg versus PE, the treatment effect of IVIg must preserve 50% of M1.

$I V I g - C o n t r o l = - 0.63 \times 50 % = - 0.315$

The non-inferiority margin (M2) for IVIg versus PE is calculated as follows:
$I V I g - P E = (I V I g - C o n t r o l) - (P E - C o n t r o l) = - 0.315 - (- 0.63) = 0.315$ Therefore, 0.315 is the non-inferiority margin for the mean change from baseline on the GBS disability scale. IVIg can be considered non-inferior to PE if the upper bound of the 95% CI of mean difference of IVIg versus PE is less than 0.315.

Hughes et al.,⁵ Conducted a meta-analysis of five studies (536 subjects) (Table 3).^15-¹⁷^,¹⁹^,²⁰ The treatment difference (IVIg–PE) was-0.02 (95% CI:-0.25 to 0.20). Since the upper bound of 95% CI (0.20) is less than 0.315 (the non-inferiority margin), IVIg can be considered non-inferior to PE (Table 4).

Discussion

This analysis provides an illustration of how data collated from a number of small studies may be used to enable retrospective non-inferiority comparisons of treatments for rare diseases, for which it is often impossible to have adequate sample sizes for prospectively designed non-inferiority studies. In the example analysis presented here, the treatment effect of IVIg for GBS was compared with that of an established treatment (PE) for this condition with efficacy proven in RCTs. Based on this evaluation, we can conclude that IVIg is non-inferior to PE for the treatment of GBS.

Post-hoc analyses of non-inferiority have limitations, such as differences in study design, treatment regimens and patient characteristics across trials. Ideally, a prospective clinical trial should be undertaken to assess the non-inferiority of IVIg. However, based on the derived non-inferiority margin in this study, a sample size of more than 462 subjects would be needed without drop out consideration to conduct a prospective clinical trial to assess the non-inferiority of IVIg versus PE with 80% power for endpoint 1 assuming a rate of 60% for IVIg and 56% for PE. Similarly, a sample size of more than 622 subjects would be needed without drop out consideration to conduct a prospective clinical trial to assess the non-inferiority of IVIg versus PE with 80% power for endpoint 2 assuming no treatment difference between IVIg and PE and standard deviation of 1.4 for both treatments, which would be a considerable challenge for a disease that is as rare as GBS. In addition, since the previous studies have showed the benefit of the IVIg in treating GBS, it is quite challenging for a sponsor to perform a large scale, prospective non-inferiority study. Instead, this retrospective assessment made use of previously collected data, permitting non-inferiority of IVIg compared with PE to be demonstrated.

A 1997 study conducted by the Plasma Exchange/Sandoglobulin Guillain–Barre Syndrome (PSGBS) Trial Study Group established that IVIg is therapeutically equivalent to PE. Treatments were considered equivalent if the 95% CI of the difference in mean improvement in GBS disability scale after 4 weeks between the two groups excluded a true mean difference of more than 0.5 of a grade. Although a change of 1.0 of a grade could be reliably measured and was clinically meaningful, a mean change of less than 0.5 of a grade was considered to be insignificant; however this equivalence value is subjective, and is not based on data from randomized clinical trials. In the current study, a non-inferiority margin of 0.315 of a grade was derived using retrospective data, this is therefore more stringent than the equivalence margin of 0.5 of a grade used in the PSGBS Study Group study.¹⁵

This study demonstrates that, in the case of rare diseases where formal prospective non-inferiority design is rendered unfeasible by the large sample sizes required, retrospective data analyses can be undertaken to ascertain whether a new treatment meets criteria for non-inferiority. We recommend that this strategy be considered in other orphan diseases as a practical means to establish non-inferiority of treatment efficacy when prospectively designed non-inferiority studies are not feasible.

Conclusion

Using the example from GBS, this study presents practical methodology for retrospective non-inferiority analyses which can be used in evaluation of treatments for rare diseases where formal, prospective non-inferiority studies are not possible.

Competing Interests

The authors are all employees of Grifols Inc., manufacturer of Gamunex®-C and Flebogamma® (both IVIg products).

Authors' contributions

CD and KH initiated the idea for developing this paper. CD and JC carried out the calculations and performed the statistical analysis. CD drafted the manuscript, the final version of which was reviewed and approved by all authors.