YODA Project

Yale University Open Data Access (YODA) Project

SYSTEMATIC REVIEW AND META-ANALYSIS REQUEST FOR PROPOSALS FROM THE YALE UNIVERSITY OPEN DATA ACCESS PROJECT, 2011

RELEASE DATE: Monday, September 12, 2011
NOTICE OF INTENT DUE DATE: Monday, September 19, 2011
APPLICATION SUBMISSION DATE: Tuesday, September 27, 2011
AWARD DATE: October 2011

1. Introduction

This request for proposal (RFP) is made by the newly established Yale University Open Data Access Project, an innovative collaboration between industry and academia designed to foster independent and objective evaluation of all clinical trial programs and surveillance data relating to a product currently approved for use in practice. It is intended to create a new standard of data disclosure by industry, linked to high quality independent analysis and dissemination.

Yale University seeks proposals for the rigorous systematic review and meta-analysis of all available data from published and unpublished clinical trial program data, as well as post-market surveillance data collected through registries, cohort studies, or spontaneous case reports relating to available products containing recombinant human bone morphogenetic protein 2 (rhBMP-2). The methodology will include, but not be limited to, individual patient-level meta-analysis conducted using clinical trial program data for 17 clinical trials provided by Medtronic, Inc., the manufacturer of rhBPM-2, as well as a systematic review of data available in the peer reviewed medical literature. Two centers will be selected to carry out these analyses independently. They will be selected on the basis of their experience and expertise conducting and disseminating similar syntheses, the methodological rigor of their proposed approach, the available institutional infrastructure and resources to support the work, and adequacy of mechanisms for protecting data security and confidentiality and addressing intellectual and financial conflicts of interest. Content expertise on the team in orthopedic traumatology and/or spinal fusion surgery is desirable although not essential.

Project duration will not exceed nine months, but the ability to complete the project in 6 months will be considered favorably in the application review.

2. Background

Recombinant human bone morphogenetic protein 2 (rhBMP-2) is an orthobiologic used to promote bone growth and sold by Medtronic in combination with an implant to promote bone growth. Many individuals have raised concerns about the integrity of certain previously published peer-reviewed articles describing research of rhBMP-2. The Spine Journal devoted its June issue to the topic and one of the articles stated:  “Level I and Level II evidence from original FDA summaries, original published data, and subsequent studies suggest possible study design bias in the original trials, as well as a clear increased risk of complications and adverse events to patients receiving rhBMP-2 in spinal fusion.”

To fully evaluate these issues, Yale University has been funded by Medtronic Inc., the manufacturer of rhBMP-2, to contract for two independent evaluations of the entire clinical trial program and all other data, published and unpublished, on the safety, efficacy and effectiveness of rhBMP-2 in order to provide additional information to the public about the product. Medtronic has no influence over the conduct of these analyses, the release of the results, or the publication of the findings.

Medtronic has committed in its contract with Yale University to allow the release of all of these data to external investigators. This public release is planned to occur after the reviews are conducted, when a protocol is developed through a process that is underway and will include opportunities for public input.

3. Research Objectives

The primary objective of this RFP is to solicit the review, evaluation and summarization of data regarding the safety and effectiveness of rhBMP-2.  Successful applicant teams will receive patient level data obtained from Medtronic and also use replicable search strategies to perform their own systematic review of the English-language peer reviewed literature in order to include studies that were not funded or disseminated by Medtronic. These should include any studies of reports relating to off-label use of rhBMP-containing products.

The review should be conducted in accordance with international standards for the conduct and reporting of systematic reviews (for example, using the PRISMA guidelines), and should include but need not be limited to the following objectives:

  • Perform a systematic review of the peer reviewed literature to identify relevant publications reporting on original data among patients using rhBMP-2
  • Critically appraise the published biomedical literature as well as the trial data obtained from Medtronic with regard to design bias that affects the conclusion of the effectiveness and adverse events of rhBMP-2, and the appropriateness of the methodology and the analyses performed.
  • As appropriate, perform quantitative or qualitative synthesis of evidence derived from the patient-level Medtronic data as well as published evidence, with regard to safety, effectiveness, quality of the evidence, and areas of clinically-relevant uncertainty.
  • Explicitly evaluate the specific safety concerns raised in previous analyses relating to possible harms associated with rhBMP-2, including local effects such as inflammation, heterotopic bone formation, pain, osteolysis and instability, and downstream or systemic effects such as leg pain and weakness, retrograde ejaculation, and possible increased risk of cancer.

There is also particular interest in the evaluation of the quality of the data, especially with respect to adverse effects such as retrograde ejaculation and infertility. The review groups may consider issues in the collection of these data, including ascertainment biases, and identify opportunities for improvements in future studies. The quality of the data should be an important focus of this effort.

4. Data

Medtronic will supply full individual patient data for 17 trials relating to rhBMP-containing products, consisting of 8 pilot studies, 8 pivotal RCTs, and one study aborted for commercial reasons (n=3). The total number of subjects is 2091, consisting of 1077 rhBMP recipients and 1014 control subjects.  The data will be de-identified, and Protected Health Information (PHI) relating to trial subjects together with information about the identity of individual clinicians will be removed by Medtronic prior to release of data on CD using standard software (SAS). Copies will be sent to the participating review sites, and the full dataset will later be made publicly available on a website to be maintained by Yale University. The details of the later public release are being developed.

5. Proposal Submission/Research Plan

The research plan should be a brief summary of the topic of the study, a summary of inclusion and exclusion criteria, a list of the appropriate publications, sources and methods the researchers will use for identifying materials and a “brainstorm” of associations or concepts related to the research topic and the conduct of the review.

The following information, where applicable, should be included within the submission:

  1. Contact information for your organization including the full name of the organization, address, Tax identification number (TIN), and if different, the address to which payment should be mailed.
  2. General information about the planned research team, including roles and responsibilities on this project, short biographies and relevant bibliographies of the core members of the research team include relevant expertise and experience and anticipated involvement of any partner organizations if applicable.
  3. Project description with goals and objectives.
  4. A detailed proposal (research design and methods) for conducting the systematic review, including methods proposed for analysis and pooling of data from published and unpublished studies, methods proposed for handling individual patient data derived from Medtronic provided clinical trials, as well as methods proposed for handling spontaneous event reports submitted to the FDA.
  5. Project budget and justification.
  6. Projected timeline, including major milestones, anticipated completion dates and projected completion date for the project.
  7. Anticipated challenges for this particular review regarding time, organization, and management and how the applicant proposes to meet those challenges. 
  8. Plan for Human subjects protection, including protection of patient-level data.
  9. Declaration relating to conflicts of interest for proposed key personnel– (see Section 8, Eligibility for more detail on this requirement). Please note any potential conflicts of interest (real or perceived) and how the potential conflict would be removed or managed.
  10. Signed cover letter from an institutional official supporting the submission.
Use additional pages as necessary to include curriculum vitae of key project personnel, as well as an institutional letter of support that attests to the availability of resources to ensure that this work will be completed in a timely fashion.

6. Communication with Yale University Open Data Access Project

Teams are required to engage in regular communication with the Yale University Open Data Access project executive committee. This communication will include electronic submission of monthly progress reports and may include teleconferences; progress reports will at a minimum provide updates on deliverable status and status on major milestones. A representative of the research team will be expected to attend a winter conference and present information about their work. Travel expenses will be paid for at least one researcher from each team.

7. Deliverables

A Final Report will be due upon an agreed upon date, ideally within 6 months after the project commencement date. The final product will be a systematic review including the following elements:

  1. Executive summary, including objectives and main findings
  2. Background, including information on the indications and use of the product (rhBMP-2)
  3. Research question(s) and study objectives
  4. Search strategy for sources of evidence, including information on the databases and resources used
  5. Selection criteria for quality of evidence (inclusion and exclusion criteria)
  6. Data collection and analysis (critical assessment), including statistical pooling or meta-analysis of data, if applicable
  7. Main results and/or main conclusions of the review in answering the initial research question(s)
  8. Implications for further research
  9. Authors’ conclusions
  10. Citations
The analysis must include interpretative commentary with respect to sources of error, bias and confounders in the reviewed studies. The analysis should also address, where possible, the question of whether it is plausible that the findings could be due to unrecognized confounders. The Report may also identify key vulnerabilities in the data and note where the quality of the data compromises the ability to make inferences about efficacy, effectiveness or safety. The Report may also make suggestions for future work and methodologies to address these shortcomings, including standards for the collection of adverse events.

A comment period will commence following the submission of the Final Report.  Within a reasonable amount of time following its submission, The Yale University Open Data Access project executive committee and consultants will review each team’s Final Report and submit questions and comments. Teams are expected to provide a written response to Yale University within 2 weeks. Public release of the Final Report will occur after that time.

While a draft manuscript is not a requirement or deliverable of this project, research teams are encouraged to publish the content of the Final Report in an appropriate refereed national scientific journal. However, because the primary objective of the project is the timely and rigorous systematic review and meta-analysis of rh-BMP2 safety and efficacy, Yale University will retain the right to publicly release the Final Report prior to scientific publication. Reasonable efforts will be made to co-publish the Final Report along with a scientific publication.

8. Eligibility

The project team should include a principal investigator with extensive experience in the conduct of systematic reviews. Those investigative teams who are selected to perform the review will be expected to work independently from one another.  Individuals and teams affiliated with Yale University and Yale -New Haven Hospital (YNHH) are ineligible.  Principal investigators and co-investigators must be currently free of any funding or affiliation, and had not received any funding during the previous three years, from the following organizations: a) Medtronic, Inc.; b)Pfizer, Inc.; c) Wyeth Pharmaceuticals, Inc.; d) any of their respective affiliates. Applicants are expected to disclose any possible conflict of interest capable of influencing their judgments, including personal, political, proprietary, family, academic and financial. This would include any personal or financial relationship with manufacturers of products which are intended to stimulate bone healing and/or enhance the success of spinal fusion surgery. These conflicts will not be an exclusion criterion, but if present, the applicants must make clear how they will manage them. An ICMJE uniform disclosure form for competing interests must be completed by all named persons on the research team and submitted with the RFP application. The form can be accessed at www.icmje.org/coi_disclosure.pdf

9. How to Apply

Notice of Intent

Prospective applicants must first electronically submit a notice of their intention to apply for funding. The notice of intent is due by 11 pm EST Monday, September 19, 2011, and should include:

  • Research issue/question under review
  • Name, address and telephone number of the principal investigator, and
  • Identities of other key personnel and participating institutions
Although the notice of intent is not binding and will not be used in the review of the proposal, the Yale University Open Data Access executive committee will use the information it contains to estimate workload and to identify potential reviewers.   The Notice of Intent should be submitted to the following address: yodap@yale.edu

Submission Requirements and Deadline

  • Proposals must be submitted by email as an electronic version (Microsoft® Office Word, with supporting components in Microsoft Office Excel and signed pages as PDF files) to yodap@yale.edu and as hard copy (4 copies). Electronic submission must be received by 11 pm EST, Tuesday, September 27, 2011 and should include “Yale University Open Data Access Project Application” in the subject line. Proposals that are incomplete will be returned to the applicant without further consideration. If a proposal is received after the due date, it will be returned to the applicant without review.
  • Proposals will be limited to 8 double-spaced pages with no less than 11-point font.
  • Page limit excludes cover page, personnel qualifications, institutional letters, or ICMJE declarations.
  • Each page must be single-sided and numbered.
  • No additional materials such as videos are permitted (as these items will not be duplicated or submitted for peer review).

Submit hard copies of proposal addressed as follows:

Yale University Open Data Access Project
Center for Outcomes Research and Evaluation (CORE) at Yale University/YNHH
1 Church Street, Suite 200
New Haven, CT 06510
Attention: Beth Hodshon


Allowable Costs

Applicants must provide a detailed budget, with accompanying justification for all operating expenses. The budget should include costs per task and deliverable and third party expenses (e.g., costs associated with partner organizations) and the applicable Facilities & Administrative (F&A) Costs, if appropriate. The budget must be consistent with the stated objectives and planned activities for the project. Direct costs should not exceed $200,000. Indirect support will be negotiated separately.  Total proposed costs will be considered when evaluating the application.

Award Criteria

The Yale University Open Data Access project awards research funds on a competitive basis. All proposals submitted in response to this RFP will compete for available funds. Only two research proposals passing evaluation (see section below) may be selected per issue/question presented in this RFP. Should all proposals exceed Yale University’s set budget or not receive approval, Yale University reserves the right not to accept any proposals.

Evaluation Process and Criteria for Peer Review

Research proposals submitted to the Yale University Open Data Access project are evaluated in a three-stage process. This process is designed to be scientifically credible, transparent and accountable.

Upon receipt, Yale University Open Data Access project staff will review all applications for completeness and for compliance with the parameters of this competition. Proposals that are not complete, do not provide adequate information in the required sections, exceed length limitations or are otherwise in a form that is unacceptable for peer review, will be returned to the applicant without further consideration. Proposals that are complete and that comply with this RFP will then be evaluated by peer reviewers. Peer reviewers are asked to assess the project‘s scientific merit, feasibility, and budget. All members of the project staff, peer reviewers and the Executive Committee will be publicly disclosed, including their affiliations and disclosures.

Applications that pass peer review will then be reviewed by the Yale University Open Data Access project executive committee for a recommendation on funding.

Final authority on funding approval rests with the Yale University Open Data Access project executive committee.

Overall, the primary criteria for funding will include:

  • Technical soundness of the proposed research project, including feasibility of timelines
  • Experience of the principal investigator and the project team
  • The institutional environment and support for the project
  • Proposed budget/overall value of the project
  • Management of conflicts and bias
  • Timeliness
Notification of Applicants

Yale University Open Data Access project executive committee will notify the successful applicants and the names of the individual researchers and research team members will be posted on the project’s website. We will be unable to provide feedback on unsuccessful applications.

For Further Information

Questions regarding this RFP should be submitted electronically to yodap@yale.edu no later than 11PM EST on Thursday, September 22, 2011. 

Responses to questions will be provided at www.medicine.yale.edu/core/projects/yodap on the RFP Q&A tab. It is the responsibility of inquiring and applying parties to monitor this web address for questions and answers that may be relevant to an application.