– Written by Dr Vamsidhar Chaturvedula, Novartis

We know how complex drug development can be. Clinical trials vary in size, complexity and duration, depending on their indication, phase, setting, competitive landscape, etc., so I won’t spend too much time discussing that and instead delve into the topic of this blog post.

In recent years, I’ve developed and managed clinical trial forecasts across various indications (onco, gen meds, paediatric, rare indications) and various phases of drug development. Despite the diversity of the trials I’ve been involved in, there are five golden rules I’ve always found to be relevant.

Plan for everything you need, but not a dollar more

Securing initial budget is a critical milestone in trial delivery – and getting it right first time, without overfunding a trial, is a genius act in itself. The ability to anticipate a full trial cost three to four years ahead, with minimal overheads, needs good understanding of the trial’s phase, protocol, operational drivers and even the indication. Planning costs for a trial from early stage and getting involved in relevant discussions as protocol evolves is a good way to ensure all necessary costs for a given trial are considered.

While preparing initial budget, one of the most common discussions is, “Should we add a buffer in the budget, just to be safe?” That might be to avoid the bureaucracy of going through the budget approval process once again, or from fear of having to justify why the budget wasn’t accurate first time. For forecasting teams, incorrectly creating buffers while building initial budgets is bad in the long run. Doing so won’t allow them to validate the accuracy of the initial projections – which is crucial, as they build experience and start getting a sense of how a study of a certain indication, phase and setting is going to perform. In my experience, most trials, if completed within their timelines, over their lifetimes, do not spend the entire money planned. The actual operating conditions of trials are far from ideal – and exactly the reason why a baseline assumption should be clear and conservative. It helps identify variables that affect forecast and gives you a legitimate reason to seek approval for more funds, if needed.

Lastly, given the cost of developing new drugs is ever increasing, it’s become a moral responsibility of forecasting teams to plan study budgets according to the essentials only.

In a world full of optimism, be a realist

Often, the tricky aspect of forecasting is planning for the fine balance between what we want to achieve and what we can achieve. Competitive landscapes often govern trial timelines, which are back calculated and keep launch dates in mind. Also, given the investment that goes into each product or the noble immediacy to bring a good cure to patients in need, there is always a sense of urgency when planning trials. This can culminate in highly optimistic plans for trial delivery. You can find optimistic start-up timelines in every clinical trial activity, be it bringing countries on board, how soon the first patient can be enrolled, how soon recruitment can be completed, how soon data analysis can be performed, and eventually how soon results can be submitted to health authorities.

We know conducting global clinical trials is a highly complex process and there are so many regulatory, scientific, commercial, legal, ethical, logistical, administrative aspects that have to run coherently for the trial to be a success. And so a ‘realistic’ forecast for a given year becomes necessary.

Forecasting teams need to have detailed monthly discussions with trial leads to understand how a given trial is progressing. These discussions need to be open and candid to understand the intricacies in trial planning, the bottlenecks around key deliverables, and the interdependencies of certain tasks. It is also a good idea to compare operational performance of other trials in a given programme to validate forecast assumptions. These discussions should consider external factors, too, in response to interim study readouts, competing trials from other companies, etc. Emphasis should be given to discussing foreseeable risks and opportunities for upfront planning.

In my experience, in 90% of instances optimistic planning always results in ‘over-forecasting’. However, this is not a blanket caution as there will be some trials that promise assets in the portfolio – on which a company will weigh interest and resources – which will result in overachievement beyond the best optimistic estimates. Finally, it all comes down to what has been planned to achieve in a given year and what can be achieved – getting a pulse of this balance is essential to generate a realistic forecast.

Focus on the current step, but don’t lose sight of the horizon

Once the budget is secured, the next step is to plan the spend across the duration of the trial. A regular review of projection versus spend is an essential part of good forecasting. If this can be analysed in the context of a larger lens of the current year, or even lifetime forecasts, it will give an excellent pulse check on the spend trend against the remaining amount to be spent. This exercise is invaluable for ongoing forecast refinement. When blended tactfully with operational teams, this analysis can help identify cost-saving opportunities too.

Also, the horizon of the review can be broadened for efficiency, when the monthly pulse check of the trial gets reviewed against the spend patterns of similar or suitable trials and adjusted as per evolving global situations (such as COVID-19, which halted global clinical trial activities).

Despite the many benefits of monthly analysis, a word of caution here: overdoing month-on-month review, especially as a performance measure, burns a lot of effort for little value in retrospective over-investigations.

Don’t spend too much time crunching numbers

Firstly, purely from a forecasting point of view, when you analyse how a study was forecasted and how the budget was spent, one can easily understand how that study performed. Good retrospective assumptions can be made on its performance in terms of start-up, recruitment, country, protocol amendments, design changes, etc., despite not being involved in the study. This skill set of understanding the story/behaviour of a study through looking at numbers is unique to seasoned forecasters.

Now, to turn the tables, as is routinely done, analysing the study from an operations point of view – understanding the indication, the nature of the contract set-up with the vendors, comparison with other trials in similar indications, the risks and opportunities, the factors influencing recruitment, any niche operational challenges – will help forecasting teams develop gut feeling. When making risk assumptions for the forecast for the current year or the following year, this gut feeling can’t be ignored.

Therefore, regular discussions with trial teams on trial performance often outweigh calculations in Excel.

Explore innovation, but never ignore your intuition

When you invest in share market, especially in mutual funds, one of the big disclaimers you often see is “past performance is not an indicator of future outcomes”. I believe this to be applicable for forecasting clinical trials, too, given the complex ecosystems they run in.

Driven by the huge cost of drug development, and in an effort to reduce or streamline forecasts (read as ‘avoid unnecessary blockage of funds’), there is always great emphasis on scenario modellings and algorithms for forecasting upcoming months. I have seen scenarios up close – and been an SME on some of them – where the most common bottleneck is identifying the variables and understanding their dynamicity. Factors affecting a clinical trial performance, vis-à-vis spend, can be as subjective as lack of motivation at trial site and unpredictable geosociopolitical dynamics. The modellings can accurately generate predictions, but accepting predictions comes with risk. Even rejecting needs a skill/intuition developed over time. Hence, I will re-emphasise my point about strong communication with trial teams.

Understanding the operational landscape, observing how similar trials are performing, evaluating forecasts at regular intervals, and then speaking with the trial team again – this self-feeding cycle of activities month on month is what generates intuition, and forecasters should never ignore that. I will conclude by saying good forecasting is a combination of science and art, and it only gets better the more you do it. Perhaps there can never be a perfect forecast with a perfect spot landing (forecast = spend) during a trial’s lifetime – but the pursuit of it is where the love for the game resides.

Happy forecasting :-)

About the Author

Dr Vamsidhar Chaturvedula is a dental doctor by education. He holds an MSc in Clinical Research and completed his management certification from IIMC. He has more than 15 years of nicely blended experience, including trial management, project management and clinical budgeting. Dr Chaturvedula works at Novartis and is currently leading the launch of their global health portfolio of medicines in low socio-economic settings across the globe.

Disclaimer: The views expressed in this blog post are solely based on my personal experiences and opinions, and do not necessarily reflect the views or opinions of my employer or the company I work for. Any information, advice or recommendations provided in this blog post should not be considered official or endorsed by my employer or company and should be used at the reader’s discretion.