FASEB releases report on enhancing research reproducibility

In response to concerns that have been raised about reproducibility in biomedical research, the Federation of American Societies for Experimental Biology (FASEB) invited experts, delegates from its member societies (including GSA), and representatives from the National Institutes of Health (NIH), National Science Foundation (NSF), and other stakeholders to discuss general factors that may impede the ability to reproduce experimental results.

It should be emphasized that the challenges in reproducing research results are not due to research misconduct nor to poor research design; rather, they generally result from the lack of uniform definitions, insufficient reporting of specific experimental details, and gaps in our understanding of biological systems. Indeed, variations in results often provide key insights into previously unrecognized phenomena and enhanced understanding of biology.

The dialog convened by FASEB is captured in Enhancing Research Reproducibility, which provides recommendations to promote the reproducibility and transparency of biomedical and biological research. The 15 recommendations in the report, which address actions for researchers, institutions, professional societies, journals, and federal agencies are listed below.

Recommendations:

FASEB identified three overarching recommendations that it believes will help enhance the reproducibility and transparency of research overall:

1. Scientists, policy makers, and journalists should use precisely defined terms and definitions when discussing research rigor and transparency to promote uniform understanding.
2. The inability to reproduce research findings among laboratories may result from a lack of sufficient detail in the reporting of critical materials or methods. FASEB
   recommends that professional societies, commercial reagent vendors, scientific journals, and funding agencies engage in a dialog to develop expectations for describing reagents, laboratory tools, and protocols in publications and grant applications.
3. Rigorous and transparent research is dependent upon adherence to good research practices by all research team members, including but not limited to principal investigators/faculty members, staff scientists, postdoctoral scholars, graduate and undergraduate students, core facility staff, and institutional leadership.

The FASEB report also includes focused discussion on some of the research areas that have provided some of the most concern: research using mouse and other animal models—and research using antibodies. Recommendations in these areas are below:

4. In 2010, two independent groups—the National Centre for the Replacement, Refinement, and Reduction of Animals in Research (NC3Rs) and the Institute for Laboratory Animal Research (ILAR) published guidelines to improve the reporting of research using animals, maximizing the utility of published studies and minimizing the need for additional studies. NC3Rs’s Animal Research: Reporting of In Vivo Experiments (ARRIVE) Guidelines and ILAR’s Guide for the Care and Use of Laboratory Animals  and Guidance for the Description of Animal Research in Scientific Publications serve as useful references. FASEB recommends that institutional animal facilities, Institutional Animal Care and Use Committees (IACUCs), and professional societies work together to promote awareness and use—as appropriate—of the ARRIVE and ILAR guidelines in animal research.
5. As their research roles expand or change, research team members may require additional training to understand, implement, and report animal husbandry practices and genetic backgrounds of animal subjects. Access to statisticians or additional statistical training may also be necessary to ensure that animal sample sizes are sufficient for the required analyses. FASEB recommends that training resources be made available to the research community on a regular, on demand basis.
6. The organization, daily operation, environment, and staffing of an animal facility can affect the outcomes of experiments. Therefore, animal facilities and staff should be considered extensions of an investigator’s laboratory and research team. FASEB recommends that investigators include animal facility staff in discussions of relevant aspects of experimental design. FASEB also recommends development of checklists to facilitate review of animal care variables and to denote study-specific variations.
7. Animal models provide critical insights into human biology and health. When communicating to the public about preclinical studies, researchers should clearly articulate the rationale for the choice of an animal model as well as its value and limitations in recapitulating human disease and its treatment.
8. Investigators and animal facility staff ensure humane treatment and care of animal subjects by adhering to established guidelines and federal regulations. To minimize environmental effects on experimental outcomes and continue to improve animal care standards, FASEB encourages institutions, veterinarians, and researchers to identify, understand, and promote the adoption of evidence-based husbandry practices. Reporting of animal care practices and study-specific variations will enhance reproducibility.
9. FASEB recommends that researchers, professional societies, and journals use standard nomenclature when reporting animal husbandry practices, breeding practices, and genetic backgrounds of study subjects in grant applications and publications.
10. FASEB strongly recommends that researchers, journals, and funding agencies work together to develop and adopt a standard format for citing antibodies in grant applications and publications. At minimum, this standard format should include the complete product name, catalog number, antibody type (monoclonal, polyclonal, or recombinant), vendor, target, lot number, and dilution/concentration.
11. There is a growing concern that all lab personnel may not fully appreciate the underlying science or limitations of commercially available antibodies or antibody based kits. Although vendor-supplied technical information may help investigators select reagents such as antibodies, this information is insufficient for validation. Therefore, FASEB recommends that stakeholders—including researchers, funding agencies, product vendors, and journals—convene an impartial advisory body to determine information needed for high quality technical bulletins. This might include immunogenic sequence, epitope sequence, cross-species reactivity, and methodologies for which the antibody is validated.
12. To achieve uniform reporting of research findings, FASEB recommends that investigators, funding agencies, and journals adopt best practices for experiments
    using antibodies. Best practices may vary by field and technique.

Finally, FASEB recommended several strategies that it believes will help increase awareness about these issues among scientists:

13. Rigor and reproducibility are critical for the advancement of science. To improve awareness within the scientific community regarding concerns about the translatability of basic research discoveries to clinical applications, strategies to enhance transparent reporting of research findings, and the new NIH policy that will go into effect on January 25, 2016, FASEB recommends that professional societies emphasize proactive efforts demonstrating the research community’s commitment to rigorous and transparent research. This could take several forms, including editorials in scientific journals, press releases, and development of uniform talking points. These efforts should acknowledge the distinction between strategies to enhance scientific rigor and transparency versus identification of scientific misconduct.
14. To facilitate investigator access to resources intended to enhance scientific rigor and transparency, FASEB recommends the establishment of a publicly accessible clearinghouse website. This website would aggregate existing training resources and best practices to enhance scientific rigor and transparency, such as those developed by NIH, professional societies, and research consortia. A resource hosted by a third-party such as FASEB would allow distribution of uniform information regardless of discipline or funding agency.
15. While the goal of FASEB’s rigor and reproducibility roundtables was to develop strategies to assist individual investigators in addressing the NIH policy that will go into effect on January 25, 2016, several opportunities for journals and society publications committees were also identified. Many journals have taken significant actions to address concerns about rigor and reproducibility. FASEB encourages professional societies to continue to enhance scientific rigor and transparency within their specific disciplines and share alternative strategies developed to address these issues.

The full report is available for download on the FASEB website.