Drug development has many moving parts. Traditionally they are handled in a more sequential and segmented way. This has served us well due to relatively slow pace of drug development. As the pace of drug development becomes faster and faster, we need to think more in a holistic way. This is especially important due to the advancement of new technologies, which has upended the way how we generate and analyze data. We present a framework to identify integrated evidence opportunities in drug development and demonstrate these opportunities using some real examples.

Keywords: integrated evidence strategy, RWE, drug development efficiency, optimal decision making, probability of success.

Technologies have progressed exponentially for decades.¹ This has upended the way we collect data and analyze data. More and more experts believe that the impact of these changes is revolutionary rather than evolutionary. Hence we can't completely rely on our past experience to guide us moving forward. We are living in an age of knowledge explosion. The advancement of new technologies has given us unprecedented access to new and untraditional data such as electronic medical record (EMR), electronic health record (EHR), genome data, social media data. There is also an encouraging environment to use data collected in the health care system such as claim databases, prescription databases etc.

In the meantime, there is an explosion of computation power. Statistical science has made progresses in leaps and bounds. The implementation of many computation heavy methods, advanced analytics such as ensemble methods, Bayesian methods, machine learning etc has become more feasible. Many advanced methods which were unusually taught at schools are now commonly used in drug development application.

However the increase in new drug approval pales in comparison.² So, the new data and new technologies have not really translated to the efficiency of drug development. There are different theories to explain this. Some say that the previous successes are due to low hanging fruits. Now all the low hanging fruits have been picked. This is not very convincing considering that advanced medical research such as immunotherapies, gene therapies, and cellular therapies have mushroomed.

Due to the mounting pressure to accelerate drug development, drug development regulation has incorporated more and more value and efficiency components in its decision making. For example, the 21st Century Cure Act (2016) is designed to help accelerate medical product development and bring new innovations and advances to patients who need them faster and more efficiently. The cure act enhances the use of real world evidence (RWE) to accelerate drug development. The sixth authorization of the Prescription Drug User Fee Act (PDUFA, 2017) enhances the use of RWE for use in regulatory decision–making. These legislative changes have led to a more emboldened FDA more receptive to innovations, more receptive to "novel" methods, which are not commonly used before. FDA has sought to adopt a "modern, risk based, and efficient" stance toward its regulatory duties.³ FDA published its framework for RWE program in 2018.⁴ FDA is expected to roll out its RWE guidance by the end of 2021. In addition, there is a shift from volume based to value based reimbursement system due to the Medicare Access and CHIP Reauthorization Act⁵ which provides a financial incentive for improved quality, outcome and cost. The change in legislative and regulatory environment has created an avenue to incorporate more RWE, more value adding PROs, more untraditional data to accelerate drug development.

Drug development has many moving parts. The interplays of these moving parts create many integrated opportunities. For example, in order to obtain registration approval, we need to demonstrate convincing benefit/risk ratio. Hence, there is the interplay between benefit and risk. In addition, we need to quantify the value of our treatment in order to reasonably price our treatment. There are different phases of drug development. The early phase development sets the foundation for the late phase development. Solid early phase research will ensure smooth transition to late phase development. Late phase results will circle back to impact early phase development. We usually develop multiple compounds, multiple indications simultaneously. These compounds and indications compete for limited resource. To make things more complicated, none of these is static. They may change over time. There are many different stakeholders: patients, HCPs, payers, HAs etc. Different people look at the same problem from different lenses. We need to work with all of them. In addition, there are different therapeutic areas. Each of these moving parts often requires different expertise. We used to handle the challenges in a more sequential way, in a more segmented way. This has served us well due to relatively slow pace of drug development. As the pace of drug development becomes faster and faster, we need more and more to think holistically, think forward, think dynamically, act at risk in order to gain efficiency in drug development.

In section 2, we will talk about integrated opportunities at different level of drug development. We will demonstrate these opportunities using some real examples. In section 3 we talk about changes needed to address the challenges in integrated strategies. In section 4, we conclude the article with some discussion.

Integration strategy opportunities

Drug development has many parts. The interplay of these parts creates many integration opportunities. We will talk about them at different development levels.

At study level

Integrated evidence strategy calls for blue sky thinking. Everything is on the table. Blue sky is the limit. There are many things we can do to accelerate drug development at the study level.

We can increase the probability of success by using enrichment strategy, using targeted enrollment, adaptive design, etc. We can increase the efficiency of the study by using group sequential design, external control etc. We can support more informative decision making by using historical data to support more accurate design assumptions, by using dynamic design.⁶ We can maximize the return of the studies by using ranked variables, by using value adding patient report outcomes (PRO), by linking the study with external databases etc.

At the indication level

Drug development is an iterative process. We can not only use the study data to gain registration approval, but also use the data to better our understanding of the disease, the treatment, and the competition landscape. For example, we can develop a screening tool to diagnose the disease earlier. We can use predictive modeling to predict which patients can benefit from the treatment. We can use meta analyses to compare our treatment with our competitors. The gained knowledge can in turn be used to improve our treatment: to increase the efficacy, to mitigate the safety issue, to develop better combinations, to develop better administration methods, to develop better formulas etc.

At compound level

We can often develop multiple indications for one compound. First and foremost, we need to assign priorities to different indications. Especially we need to handle the relationship between a big indication with moderate probability of success (POS) and a medium indication with big probability of success. We also need to know how to handle small indications, tiny indications. There are many questions we need to answer in the compound level. How much halo effect is perceived by health authorities (HA) from indication to indication. Can we piggy back a small indication with a big indication for submission? Can we combine 2 similar indications for submission? If the drug is already on the market, is there any off label use? Is there any evidence of efficacy in the off label use?  Is the data strong enough to justify further development? Is the data convincing enough to support direct registration submission?

Portfolio management

Portfolio management is very complicated with many moving parts. For example, a big program may have a moderate probability of success and lots of competition. In contrast, a small program may have a big probability of success, big regulatory incentive, little competition. A portfolio attractive to a small biotech may not be very interested to a big pharma. Managing the portfolio based on instinct alone is dangerous. A quantitative approach can guide the portfolio management to make it more objective, more efficient. Portfolio management involves large amount of uncertainties. It is critical to perform sensitivity analyses to evaluate the robustness of the conclusion.

Across organization collaboration

Drug development collaboration often goes beyond the boundary of one organization. For example small biotechs often partner with big pharmas to take advantage of each other's research capability and development capability. Pharmas often sponsor investigator initiated studies (IIS) to take advantage the investigators expertise in specific therapeutic areas. Eradication of infectious disease often requires the coordination at national level, or even international level.

So far the discussion is broad generalizations. To drive the message home, we will look at some concrete examples.

First let's look at Ibrance approval for male breast cancer patients. The Ibrance approval is famous due to its all RWE submission. However RWE can't take all the credits. The Ibrance success is due to the integration of multiple factors. First it is due to FDA's incentive to fatal, rare disease. Male breast cancer is rare disease with fewer than 3000 new cases in US in 2019. It is very challenging to develop this indication separately. FDA is eager to close this gap. Second, strong efficacy has been observed in female patients (Ibrance USPI). The strong halo effect allows the extrapolation of the efficacy from female patients to male patients. FDA just need a little more male patients specific data to justify the approval. In this specific context, the response rate shown by RWE is sufficient. In addition, RWE also confirms the similarity in safety profile between male and female patients.

Another high profile approval is Blincyto in Philadelphia chromosome negative relapsed or refractory B cell precursor ALL. This approval is well known due to the use of external control. The success is also due to the integration of multiple factors. First, the indication is a rare disease rapidly fatal. Second the single arm pivotal study design is justified and strengthened by some precedence in similar setting. Finally it is due to a reasonable choice of statistical methodology. They performed a weighted analysis using patient level data to balance the prognostic factors. A reasonable choice of statistical methodology is critical. For example Selinexor's submission also performed a comparison with external control. But their analyses were rejected by FDA as evidence to support registration decision due to major methodology issues.

The third story is Farydak's approval in Multiple Myeloma. This story is especially relevant to statistical silo. The pivotal study shows statistically significant result. But it is not considered as clinically meaningful due to some serious safety issues. FDA passed the ball to oncologic drugs advisory committee (ODAC). ODAC recommended against its approval. However FDA approved the drug based on a pre–specified sub group analyses. There are multiple things we can learn from this story. First, statistical significance may not be enough to define success especially when there are safety issues. Second, it highlights the importance to pre–specify sub group analyses with regulatory implication. We also need to control the quality and quantity of pre–specified sub–group analyses.

The 4th story is Adcetris approval in previously untreated stage III or IV classical Hodgkin Lymphoma (cHL). The Adcetris story illustrates the combination of enrichment strategy and submission strategy. Adcetris is a CD30 antibody conjugated chemo. Adcetris specifically binds to CD30 on the surface of cancer cells and got internalized to the cell. The chemo was released inside the cell and kills the cancer cell. Based on this mechanism of action (MOA), an enrichment strategy was developed to include only CD30 level more than 10% patients in the pivotal study to increase the probability of success. However this creates a gap in labeling discussion for patients with CD30 level less than 10%. The efficacy in CD30 less than 10% patients is extrapolated from more than 10% patients in the same indication and from less than 10% patients in a similar indication. The Adcetris approval also highlights the importance of development sequence choice. The pivotal study shows a marginally statistically significant result with a p value of 0.035. The treatment has some serious neuropathy issues. As mentioned in the Farydak story, such as statistical significant result may not be considered as clinically meaningful. However the indication was approved without an ODAC and with breakthrough designation. The drug was approved in several other indications with big treatment effects. The halo effect from these previous approval shines through to support a favorable benefit/risk assessment in untreated cHL.

The 5th story is the Provenge story. This story demonstrates the importance of an end to end view of drug development, especially the post marketing operation in order to be commercially successful. Provenge was approved by FDA for prostate cancer with 4 month survival benefit in 2010. However the drug developer, Dendreon Pharma, went bankrupt 4 years after its approval. Provenge is a complicated immunotherapy. It involves removing patient's blood cells out and then manipulate them to target cancer cells and put them back to patients body. It costs $93K for one course of treatment. However majority of the physicians value the treatment no more than $30K. There are many moving parts to ensure a commercial success including how many cells to move out of a patient, how many cells to put back and what co–medication to provide etc. Dendreon fails to address these scientific challenges, and fails to bring the compound to the market.

Finally we will talk about Keytruda. Keytruda is the crown jewel in Merck's portfolio. Keytruda accounts for more than $7 billion sale in 2019. The Keytruda story exemplifies the dynamics of decision making, asset development strategy and portfolio management.

Keytruda is a type of immunotherapy that mobilizes the patient's own immune system to fight cancer. It can be used across the board to fight different types of cancers. It can be combined with many different traditional therapies to generate synergy. Merck acquired Keytruda through acquisition of Sherring Plough. In early 2010, Merck terminated the development of Keytruda and began preparing to out license it. In late 2010, BMS published their promising result from their own immunotherapy Ipilimumab. Keytruda finally gained traction in Merck. Merck started developing Keytruda like crazy. Part of the secret of Keytruda success is to increase the probability of success by using biomarker guided enrichment strategy instead of blindly chasing big indications. Keytruda completely changed Merck. After Keytruda, Merck morphed into a strong hold in both oncology and immunotherapy.

Changes needed for integrated thinking

We have talked about the integration opportunities. Now let's turn to the other side of the penny: the challenges and barriers to think in an integrated way. An integrated approach can be concerning to people who are comfortable with the traditional approach. For example, people may not want to take the risk that can be put off despite the potential benefit coming with it. Integrated thinking may require new set of skills. There may be a steep learning curve to think in an integrated way. It takes both organizational and individual changes to address these challenges. Organizations can change their working models and culture to facilitate integrated thinking. Individuals can change their mindsets and behaviors to become an integrated thinker, hence a better contributor.

Culture eats strategy for breakfast. Integrated evidence strategy calls for culture change. Transparency is the foundation of integrated thinking. Transparency goes a long way to cultivate a culture of collaboration and hence integration. Without transparency, integrated strategy is built on sand. Integrated strategy calls for changes in management styles. Integrated strategies call for visionary leadership rather than micro–management. Manager's role should be more about clarifying the vision and missions. managers should let people do what do best and get out of way. Management role should be more about pulling people together, addressing challenges and road blocks rather than imposing solutions top down. Not doing micro–managing will free the power of innovation and make room for integration. Teams should be more open to share knowledge, expertise and resource sharing etc. Decision making should be data driven, science driven. Otherwise it can easily slipped into the pitfall of power play.

Integrated thinking is progressive thinking. In order to keep up with the latest and greatest technologies, the organizations should renew their work force from time to time through new hire and/or training. One barrier to integrated thinking is silos within an organization. Cross functional team can break these barriers if they are empowered to make decisions. Leaders must emphasize that there is no "one size fits all" approach. Teams should tailor evidence generation activities and timelines for individual programs according to data, risk and competitive environment.^7–15

Integrated thinking calls for mindset change for everyone and strong bonding among team members. First, everybody should position themselves as drug developers rather than just experts in specific line functions. We should adopt a patient centric view of drug development. Developing drugs to help patients is our common goal. The common ground will form the foundation for cross functional collaboration. To further expand the common ground, we can expand our knowledge across the aisle. For example, statisticians can expand their knowledge about the regulatory guidance, clinical guidance, competition landscape etc. We can think laterally to find common ground horizontally. Although we may focus on a specific area, we should have an end to end view about drug development. For example, proof of concept (POC) studies can define success in terms of the confirmatory trials.¹⁶ Confirmatory trial design can incorporate value adding PRO end points for post marketing needs. By the same token, we can think big to find common ground vertically. But at the end of the day, we need people to take ownership to have successful execution to turn the opportunities to realities.

Drug development is a multi–disciplinary endeavor that relies on the cumulative effort of highly skilled teams to be successful. Yet these teams can't work in silos. Integrated thinking can pull different parts of drug development together to generate synergy, to accelerate drug development, to increase our probability of success, to support better decision making. Hence it is critical for drug development in a quick changing landscape.

Integrated thinking is blue sky thinking. It breaks barriers to think dynamically, to think progressively, to think forward, to think laterally. Integrated thinking takes calculated risk. The new drug development environment calls for all rounded talents, who can think big and takes ownership at the same time.

Integrated strategy is the future of drug development. We should all jump on the band wagon.

This manuscript was supported by Abbvie. Abbvie contributed to the review and approval of the content. Zhongwen Tang is an employee of Abbvie Inc. and may own Abbvie stock.

None.

None.

1. Roser M, Ritchie H, Technological Progress.
2. Angelika B, Bhupinder SK, Raj K. Trends in FDA drug approvals over last 2 decades: An observational study. J Family Med Prim Care. 2020;9(1):105–114.
3. Vogenberg FR, Smart M. Regulatory change versus legislation impacting health care decisions and delivery. PT. 2018;43(1):34–38.
4. Medicare access and CHIP reauthorization act of 2015. 114th congress.
5. Framework for FDA’s Real World Evidence program, 2018.
6. Tang Z. Defensive efficacy interim design: structured benefit/risk assessment using probability of success. Journal of Biopharmaceutical Statistics, 2016.
7. Tang Z. Optimal futility interim design: a predictive probability of success approach with time to event end point. Journal of Biopharmaceutical Statistics. 2015;25(6):1312–1319.
8. https://www.accessdata.fda.gov/drugsatfda_docs/label/2019/207103Orig1s012lbl.pdf
9. https://www.accessdata.fda.gov/drugsatfda_docs/label/2020/125557s017lbl.pdf
10. https://www.accessdata.fda.gov/drugsatfda_docs/label/2020/125514s084lbl.pdf
11. https://www.accessdata.fda.gov/drugsatfda_docs/label/2019/125388s100lbl.pdf
12. https://www.accessdata.fda.gov/drugsatfda_docs/label/2015/205353s000lbl.pdf
13. Antonijevic Z, Optimization of pharmaceutical R&D Programs and Portfolios, design and investment strategy, Springer, 2015
14. Olson MS. Developing an integrated strategy for evidence generation, J Comp Eff Res. 2018;7(1):5–9.
15. Gaurav Agrawal, Harriet Keane, Maha Prabhakaran, et al. The pursuit of excellence in new drug development. 2019.
16. Tang Z. Dey J. Bayesian PPOS design for clinical trials. PaSIPHIC anual meeting. 2011.