Multi-Stakeholder Call to Action for the Future of Vaccine Post-Marketing Monitoring: Proceedings from the First Beyond COVID-19 Monitoring Excellence (BeCOME) Conference

Authors: Vincent Bauchau, Kaatje Bollaerts, Phil Bryan, Jim Buttery, Kourtney Davis, Robert T. Chen, Daniel R. Feikin, Antonella Fretta, Sarah Frise, Sonja Gandhi-Banga, Hector S. Izurieta, Corinne Jouquelet-Royer, Alena Khromava, Lin Li, Raj Long, Sarah MacDonald, Lydie Marcelon, Robert Massouh, Wilhelmine Meeraus, Flor M. Munoz, Karen Naim, Dale Nordenberg, Hanna Nohynek, Heather Rubino, Daniel A. Salmon, Sarah Sellers, Laurence Serradell, Laurence Torcel-Pagnon & Jamie Wilkins

Springer Nature (2025)

1 Introduction

The response to the coronavirus disease 2019 (COVID-19) pandemic included a global effort to monitor the benefits and risks of vaccines and therapeutics, when used in the real world (i.e., ‘post-marketing’). This rapid and large-scale activity presented a new level of challenges, met with unprecedented levels of innovation, cooperation, and lay public communication. Similar challenges had been experienced in the 2009 H1N1 pandemic and, although some lessons learned triggered initiatives and solutions [1, 2] to improve response and cooperation among key stakeholders, there was insufficient continuity and coordination of pandemic preparedness between all stakeholders following H1N1. To improve pandemic preparedness following COVID-19, it is critical to keep the current momentum of cooperation among key stakeholders and to expand the conversation to other vaccines and to therapeutics, when relevant.

BeCOME was launched in 2022 to build on the achievements and learnings attained through multi-stakeholders’ cooperative efforts in response to the unprecedented public health challenge posed by the COVID-19 pandemic (e.g., building the COronaVIrus Disease Research Initiative on Vaccine Effectiveness [COVIDRIVE] platform, subsequently expanded as id.DRIVE [3]). Having experienced the value of a cross-company collaborative approach [4] and realizing the extent to which possible solutions to key global challenges are joint efforts with health authorities, public institutions, and other organizations, the vision was to rapidly engage with multiple stakeholders (i.e., regulatory authorities, academic institutions, public health experts, research networks, and non-government organizations) to create a sustainable forum. Thus, experts in pharmacovigilance and pharmacoepidemiology could discuss, collaborate on, and develop new methods and initiatives regarding the real-world monitoring of benefits and risks of vaccines to improve preparedness and ultimately the response for future pandemics. To initiate development of a plan for achieving this vision, BeCOME hosted its first conference in June 2023 to present and discuss mutual priorities for the next 5 years. In turn, these mutual priorities were to inform cooperative action plans to improve joint preparedness across stakeholders.

2 Inaugural Cross-Stakeholder Meeting

“Beyond COVID-19: the Future of the Real-World Monitoring of Vaccines” was held June 11–13, 2023, at Les Pensières Center for Global Health in Annecy, France (see online supplementary material for conference program). Les Pensières, set aside by the Mérieux Foundation to host events and collaborations focused on public health, was a perfect setting for this gathering of 66 international experts from a broad spectrum of stakeholders (industry, academia, public health and academic research institutes, supra-national organizations, non-government organizations, regulatory agencies) and including attendees from low- and middle-income countries (LMICs). The meeting steering committee comprised representatives from industry (n = 3), the Centers for Disease Prevention and Control (n = 2), and the Bill & Melinda Gates Foundation (n = 1). Key stakeholders came together to identify and make recommendations on strategic priorities, solutions, timelines, and models for collaboration to expand on the cooperation initiated during the pandemic. The meeting began with an inspiring keynote address from Dame June Raine, CEO at the UK Medicines and Healthcare products Regulatory Agency, highlighting the importance of vaccine vigilance and the various challenges, innovations, and collaborations seen over the last 2 decades. Dr. Raine’s words served as a call to action for the BeCOME conference participants to build upon the learnings, technological advances, new data sources and capabilities, digital transformation, new communication methods to meet growing public expectations, and novel collaboration models to create a blueprint for future systems.

Opening remarks by Dr. Shanthi Pal, Team Lead for pharmacovigilance at the World Health Organization (WHO) and sitting board member at the Uppsala Monitoring Centre, also recognized the efforts that led to BeCOME. Dr. Pal urged conference participants to think broadly, avoid siloed systems, and—importantly—to consider both the needs of vulnerable settings and special populations and how any advances in safety surveillance may benefit medicines as well as vaccines.

Day 1 included sessions covering the following BeCOME areas of focus for post-COVID-19 vaccine monitoring: background incidence rates (BIRs), safety studies, pregnancy surveillance, vaccine benefits, signal detection, digital solutions, LMICs, and a transversal session on multi-stakeholder collaborations. During each session, the audience was oriented to multiple perspectives on major challenges and potential solutions under consideration, focusing on those relevant to a cooperative approach across multiple stakeholders. Each session concluded with an engaging panel discussion. Day 2 featured a series of 90-minute workshops for each area of focus, with the goal to gain alignment and reach consensus on priorities and recommendations for solutions and actions, followed by a final plenary defining key overall priorities and next steps.

3 Key Focus Areas and Identified Strategic Priorities

3.1 Background Incidence Rates

BIRs are critical data for interpreting signal monitoring and signal refinement efforts. Timely assessments of vaccine safety signals require comprehensive and valid BIRs of adverse events of special interest (AESI), especially in mass vaccination programs. Challenges faced by multiple stakeholders were reviewed:

• During the COVID-19 pandemic, various lists of AESI emerged, each evolving over time. Furthermore, BIR could be dated, unavailable, or missing for some AESI. Developing a harmonized list of AESI that is applicable for all vaccines across stakeholders is mutually beneficial but presents challenges, as BIRs are specific to the targeted disease and vaccine platform. Nonetheless, activation of systems capable of promptly generating the most up-to-date BIR—for both established and novel AESI—in a format for efficient utilization is warranted.
• The lack of harmonized methods for validation of case definitions or phenotypes in real-world data coupled with the heterogeneity across BIRs for a given AESI makes it challenging to conduct analyses and interpret results. The credibility of safety signals hinges on consistency across BIRs, which can be accomplished through standardization and validation of methodology and case definitions.
• The absence of detailed BIRs brings difficulties in evaluating safety signals in different age groups, by gender, or in specific sub-populations such as pregnant individuals and the elderly. Additionally, BIRs are frequently unavailable for all countries of interest, especially LMICs. Generating granular-level or specific BIRs can improve the ability to assess safety signals [2].

To address these challenges, the following key strategic priorities were identified.

1. 1.Establish a global multi-stakeholder working group to enable sustainable delivery of BIRs for a harmonized list of well-established and newly identified AESI.
2. 2.Achieve multi-stakeholder agreement on best practices on the proper application of BIRs, addressing data quality, management of heterogeneity, and standardized reporting of data.
3. 3.Improve the availability and granularity of data (i.e., by age and sex stratifications [both separately and dually stratified]; in specific sub-populations, including pregnant and lactating/breastfeeding people, high-risk groups, etc.; in specific geographic regions, including LMICs).

3.2 Safety Studies (Active Pharmacovigilance Surveillance)

There is a need for timely, high-quality, fit-for-use safety studies. Study design requirements must be balanced with study feasibility, data quality, and availability of results with public health urgency. Challenges specific to designing and conducting vaccine safety studies during a pandemic include the following.

• Insufficient numbers of pharmacoepidemiologists or individuals trained in pharmacoepidemiology to design and execute safety studies.
• Lack of visibility of data sources and study methods used for safety studies, including what data sources and methods were commonly used, what worked, and what did not work and under what circumstances.
• Lack of shared understanding of what tools can be used, when, and how, across the continuum from passive surveillance to full safety studies.
• Lack of a shared understanding of data source characteristics and operationalization of such sources globally, including LMICs.
• Limited understanding of the surveillance infrastructure to conduct safety studies, especially in LMICs.

To further meet these and future challenges, the following key strategic priorities were identified.

1. 1.Improve knowledge of commonly used data sources and associated data source characteristics for vaccine safety studies.
2. 2.Increase global visibility of what data sources and study methods worked and under what circumstances to inform the areas of focus for future development and innovation.
3. 3.Create shared understanding of what tools and/or methods can be used across the continuum of surveillance approaches (i.e., passive surveillance, observational studies); this includes promoting existing yet underused methods (i.e., through preparation of whitepaper and associated toolbox for safety surveillance methods). Tools may include artificial intelligence, intelligent automation, and machine learning that can be leveraged to respond to rapidly changing vaccine platforms. Such innovations are needed to identify and track vaccine platform changes and changes to product safety that may occur over time and ensure optimized use for translation of clinical case definitions into clinical phenotypes/surveillance case definitions.
4. 4.Improve cross-stakeholder visibility of surveillance infrastructure globally, especially within LMICs.

3.3 Pregnancy Surveillance

The pregnancy surveillance workshop discussion centered around two main themes: (1) maintaining and improving the quality of studies, and (2) generating safety evidence faster and more efficiently. Challenges specific to studying vaccine exposure and safety in the pregnant population include the following.

• Inconsistent guidance from regulatory authorities on the conduct of post-marketing safety studies in pregnant people that leads to inefficiency in study design, feasibility assessments, and—in turn—later delivery of actionable evidence to inform safety decision making.
• Limitations of existing data sources to conduct post-approval pregnancy safety studies, including long data lags, limited ability to capture vaccine exposure information, and mother–infant linkage. Additional tools for characterizing and assessing data quality (i.e., relevance and reliability) are needed, as are standards to ensure the representativeness of the safety study population.
• Limitations of pregnancy surveillance and pharmacovigilance systems in LMICs, including gaps in infrastructure, resources, training, data quality, and methods.

To address these challenges, the following key strategic priorities were identified.

1. 1.Harmonization of regulatory guidance for the conduct of post-approval pregnancy safety studies by developing core safety study templates.
2. 2.Development of standard and updated information on data sources that can meet regulatory requirements (including LMIC populations) and drive increased infrastructure to deliver faster safety evidence generation.
3. 3.Expansion and improvement of pregnancy surveillance and pharmacovigilance systems in LMICs.

3.4 Vaccine Benefits

In BeCOME, the ‘vaccine benefit’ expert group broadly encompasses activities related to the benefit of vaccines, including measuring vaccine effectiveness, characterizing disease burden, and evaluating impact. Challenges specific to vaccine benefit evaluation include the following.

• High levels of resources and expertise are needed for infectious disease surveillance and primary data collection activities that evaluate vaccine benefit.
• Limited infrastructure is available to evaluate benefit, particularly in sub-populations and for rare diseases, where study power/precision of effectiveness estimates are problematic.
• Speed of data access and delays in preparing valid outcome algorithms in many databases often delay rapid assessment and communication of benefit to decision makers.
• Linkage between disease surveillance and vaccine registries is not universal, and key variables required for evaluation of vaccine benefit are often missing or unavailable in existing data sources.
• Information required to evaluate vaccine benefit is often siloed by country, pathogen, and/or population.
• Bias, particularly due to unmeasured confounding, remains a threat to the validity of estimates of vaccine effectiveness.

The following key strategic priorities for multi-stakeholder collaboration were identified as potential solutions to the specific challenges facing vaccine benefit stakeholders.

1. 1.Leveraging and expanding existing data collection systems, platforms, and networks to (a) more efficiently and quickly conduct primary data collection studies with improved power, (b) enrich secondary data sources with multiplex polymerase chain reaction testing and vaccination/brand data as alternative ways to estimate effectiveness, (c) encourage open data for multi-stakeholders’ research purposes, and (d) use partnerships to allow wider access to secondary data.
2. 2.Increasing the speed of vaccine benefit analyses by considering the use of common data models, considering integration of data from multiple sources, obtaining more timely access to data and standardized analyses (thereby reducing need for primary data to leave sites, which in turn reduces bureaucracy), and developing detailed template protocols for evaluating vaccine effectiveness using different data sources and study designs.
3. 3.Improving communication between stakeholders to identify priorities and agree on who can best deliver what information and by when. Specific to post-marketing studies of effectiveness, improved clarity is needed around responsibilities and ownership of activities with the aim of preventing duplicated work and effort.
4. 4.Developing guidance on the use of real-world evidence specifically for decision making around vaccine benefit, with input sought from multiple stakeholders on the acceptability and value of different study designs, including novel applications of self-controlled risk interval for effectiveness; the use of carefully chosen negative exposures and outcomes to determine the existence of potential bias; and other advanced methods for bias understanding and adjustment.
5. 5.Harmonize the implementation of benefit–risk studies by developing standardized methods and encouraging transparent reporting of detailed methodological and operational definitions.

3.5 Signal Detection

Participants in the signal detection workshop discussed the collective experience and potential value of traditional signal detection methods in conjunction with the observed-to-expected (O/E) analysis, mandated in many jurisdictions during the pandemic. Although the scope of signal detection activities has expanded over the past 2 decades to include new data sources, algorithms, and expertise across passive and active surveillance platforms, no studies have been conducted to determine what methods have the best predictive value, in a time- or exposure-dependent manner, across the pandemic timeline. During the pandemic, O/E analyses were required for signal generation/validation and characterization of safety data/topics in the requested safety monthly aggregate reports. This method was historically reserved for refinement of signals previously detected by traditional pharmacovigilance methods (index case on medical review, statistical, or other disproportionality analyses). To meet narrow reporting timelines, non-adjudicated cases were included as “observed” counts, which raised significant challenges for the interpretation of results. This method posed difficulties interpreting adverse events (AEs) that were also associated with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection circulating widely at the time. The heavily weighted O/E-related activities required multiple layers of stratification and impacted both AE and exposure data. Focusing on these time-consuming O/E activities diverted attention away from more traditional methods that require time for individual case scrutiny, exchange of follow-up information, and data maturation prior to the evaluation of evidence supporting or refuting a causal association with vaccine. Also, insufficient effort was devoted to analyzing the limitations of O/E studies and to promoting the use of alternative methodologies, including but not limited to self-controlled techniques with lower risk of bias. With this understanding, the following key strategic priorities were identified.

• Identify signal detection data sources and methods applied by different stakeholders across the pandemic.
• Assess performance of signal detection methods (including non-quantitative approaches such as clinical case review).
• Discuss and propose consensual best practices for safety evaluation, including but not limited to the use of innovative methods to reduce bias, improving the use and validity of O/E methods, adjust for under-reporting of AEs, and the use of safety summary reports as a tool to communicate O/E and signal detection findings to health authorities.

3.6 Digital Solutions

During the COVID-19 pandemic, the large volumes of AE reports received by industry and regulators surpassed the projected numbers. Much was done (pre- and intra-pandemic) to augment the reporting platforms (from mobile applications to digital AE reporting forms and digital self-reporting surveillance systems) [5,6,7]. These digital tools were critical in allowing timely reporting and further supplemented active surveillance of post-vaccination symptoms [5]. Beyond these digital tools, there is an equal if not more important need to enhance the global data management capabilities and implement digital innovations that can further integrate reporting tools into healthcare systems, thereby allowing active surveillance through primary and secondary data analysis. […]