MAPGuideⓇ
Equitable Access Toolkit
Measuring Impact
Developing a new medical product is a multi-year process, often involving multiple R&D and commercialization partners. Product development funders should be able to monitor the impact of their funding throughout this process, particularly as it relates to the achievement of equitable access. Given the potential for an extended timeline between the grant of product development funding and the availability of a final product for use by patients, impact measurement could be approached in two stages:
1. Impact Milestones that measure the progress of a product developer towards meeting its equitable access obligations throughout the product development process.
2. Impact Metrics that measure the extent to which a final, approved product meets the needs, and has a positive effect on, the lives of target populations.
1. Impact Milestones
Impact milestones can be structured in line with the different activities that need to be undertaken in order to meet the equitable access obligations in a funding agreement. Milestones should include realistic timeline expectations and have clear actions or deliverables that indicate achievement. Examples of potential impact milestones are outlined below, although milestones for a specific agreement need to be relevant to the scope of the funded product development activities.
Availability and Affordability
- Identify an alternative manufacturer that is willing and able to meet affordable price obligations and supply requirements for the territory.
- Deliverable: signed agreement between developer and manufacturer.
- Complete technology transfer to the alternative manufacturer.
- Deliverable: completion of technology transfer plan confirmed by both parties.
- Sign a purchase agreement with a procurement body for supply of the product to the territory (either by the developer or an alternative manufacturer).
- Deliverable: signed agreement.
Pricing Terms
- Achieve a cost of goods for the product that will enable a price that is affordable for the territory and sustainable for the developer (or alternative manufacturer).
- Deliverable: cost of goods calculation signed by Chief Financial Officer.
Data Sharing
- Publication of clinical data in a peer-reviewed, open access journal.
- Deliverable: copy of published paper.
Regulatory Strategy
- Request a WHO joint advice consultation
- Identify the reference agency to be used when working with WHO (e.g. MHRA, EMA).
- Deliverable: regulatory plan updated with selected reference agency.
- Data submission to WHO.
- Deliverable: written confirmation of submission and copy of data package.
- Submit a WHO Prequalification dossier at the time of a WHO expression of interest.
- Deliverable: copy of submission.
- Make submissions to relevant national regulatory authorities outside of major markets.
- Deliverable: copy of submission.
- Regulatory approval from national regulatory authorities.
- Deliverable: approval listing.
The achievement of milestones can be linked to the payment of tranches of grant funding. This approach can incentivize timely achievement by developers and protect a funder’s investment by ensuring that the funds paid are proportionate to the level of progress towards equitable access goals.
Examples from the MAPGuide
[Funder] may delay or condition a payment if: a. [Developer] has not achieved a milestone by the agreed time, unless such delay has been approved by the [Joint Monitoring and Advisory Group] […]
Source: taken from a development funding agreement between CEPI (Funder) and Valneva (Developer) for manufacturing and late-stage clinical development of a Chikungunya vaccine. Read in context.
Reporting & Payment Schedule: Payments are subject to [Developer’s] compliance with this Agreement, including [Developer’s] achievement, and the [Funder’s] approval, of any applicable targets, milestones, and reporting deliverables required under this Agreement. The [Funder] may, in its reasonable discretion, modify payment dates or amounts and will notify [Developer] of any such changes in writing.
Source: taken from a grant agreement between the Bill & Melinda Gates Foundation (Funder) and Icosavax (Developer) for the preparation of an Investigational New Drug (IND) Application and conduct of a phase I first-in-human clinical trial of a virus-like particle (VLP) vaccine against SARS-CoV2. Read in context.
When [Developer] considers that any Milestone has been achieved by the relevant Milestone Date:
(a) [Developer] shall as soon as reasonably practicable provide the [Funder] with a detailed report (the “Milestone Report”) setting out how the Milestone was achieved and requesting payment of the next Tranche of the Award Amount; and
(b) The [Funder] shall confirm to [Developer] in writing, within [**] Business Days of receipt by the Trust of the Milestone Report either that:
(i) the Milestone has been achieved by the Milestone Date to the [Funder’s] reasonable satisfaction, in which case the Trust shall make payment of the next Tranche of the Award Amount within [**] Business Days of the date of such written confirmation in the amount determined by the Trust from time to time; or
(ii) the Milestone has not been achieved to the [Funder’s] reasonable satisfaction by the relevant Milestone Date and that the payment shall not take place, in which case the [Funder] shall provide [Developer] with reasonable details of the grounds on which it has reached this decision.
The [Funder] may, at its sole discretion, grant [Developer] a reasonable period of time (“Milestone Extension”), in order to address the reasons why the [Funder] has judged that a particular Milestone has not been met. Upon the expiry of a Milestone Extension, the [Funder] shall, at its sole discretion, decide whether or not to permit full or partial payment of the relevant Tranche of funding to [Developer].
Source: taken from a research funding agreement between the Wellcome Trust (Funder) and PTC Therapeutics (Developer) under the Trust’s Seeding Drug Discovery Strategic Award Programme for the discovery and development of a novel glioblastoma treatment. Read in context.
2. Impact Metrics
Impact metrics for a product development funding agreement need to be set based on a funder’s goals for the disease(s) that the product(s) are intended to diagnose, prevent or treat. One potential approach is to present metrics by uptake, economic benefit and health benefit. This approach has been used by the Medicines Patent Pool, which has also published a modeling study on measuring economic and public health impact.
Uptake
- Number of doses supplied to a procurement body
- Number of doses administered
- Relative distribution vs. high income markets (geographic distribution)
Economic Benefit
- Cost savings compared to an alternative product
- Economic losses averted
- Increase in local manufacturing capacity
- Revenue to local manufacturers
Health benefit
- Deaths averted
- Volume and location of product stockpiles to be deployed in case of outbreak
Not all of the above metrics can be calculated based solely on information provided by the product developer. In order to fully assess the impact of a product developed as a result of a partnering agreement, funders, and PDPs need to consider building a network of local organizations in relevant territories. Organizations such as the International Initiative for Impact Evaluation can help to harness relevant data and provide other evidence gathering support, and the Access to Medicine Foundation highlights some best practice examples of establishing partnerships to measure and evaluate the success of access strategies. Funders may also need to consider dedicated funding for organizations to conduct detailed impact studies (see Reinvigorating Impact Evaluation for Global Development).
Examples from the MAPGuide
In addition to any and all reports required to be delivered to the [Funder] under the Investment Documents, the [Developer] shall furnish, or cause to be furnished, to the [Funder] […] the following reports and certifications (the “PRI Reports”): […]
Within 75 days after the end of each fiscal quarter of the [Developer], a quarterly written report […] on the impact performance relative to the [products], together with information on the impact key performance indicators (“Impact KPIs”) achieved that are aligned with the [Funder’s] objectives to improve global health. For the purposes of the foregoing, the priority Impact KPIs are:
Target Outcome
Performance Indicators
Program Product
Regulatory approvals in target countries
Program Products
Number of Company Products catalyzed/distributed
Lives Improved
Number of Lives improved by the provision of the Program Products in LMICs
Reach
Number of Low – Middle Income Countries benefiting from the Program Products
Source: taken from a global health agreement between the AXA Prime Impact Master Fund (Funder/Investor) and Revelation Biosciences (Developer) for the development of a therapeutic for allergic rhinitis and a diagnostic for respiratory virus infections. Read in context.
Related Commentaries
A failure to meet impact milestones in a timely manner could be a signal that a partnership will not achieve its equitable access objectives. In this case, funders may need to consider their termination rights or other rights aimed at ensuring the sustainability of the development and manufacturing of a product.
Impact milestones may form part of access plans that detail the steps that the agreement partners will undertake in order to achieve equitable access to a product.
Partnering agreements need to include reporting requirements that enable funders to monitor progress. In many cases, it will be necessary to include ongoing reporting obligations that survive the term of the agreement in order to ensure that funders can obtain information about the ultimate impact of their funding. This is particularly important for early stage research and development funding that may be later carried on by other organizations.
This toolkit has been built based on the data in the MAPGuide and the GHIAA team’s experience of negotiating and implementing agreements. We intend that the toolkit will evolve and expand over time based on input from MAPGuide users and availability of new agreements showing examples of alternative approaches. We welcome ongoing constructive dialogue around these materials and encourage you to contact us or fill in our feedback survey to share your thoughts, questions and suggestions.
Authors: Bridie Telford
First publication date: November 21, 2022
Last updated: March 20, 2023