Office of Research Click here for the Office of Research Home Page Click here for the UC Irvine Home Page Click here for the School of Medicine Home Page

Enhancing Rigor and Reproducibility

In recent years, there has been a growing awareness of the need for rigorously designed published preclinical studies, to ensure that research findings can be faithfully replicated. In response to this need, the National Institutes of Health (NIH) has established a formal initiative aimed at improving research reproducibility and are implementing significant changes to their policies, grant applications and peer-review process, which took effect Jan. 25, 2016.

UC Irvine is committed to upholding the highest standards of conducting biomedical research and Dr. Oswald Steward, senior associate dean for research in the School of Medicine, is leading a campus-wide effort to ensure that the UC Irvine research community is informed about these important changes and have resources to address the "reproducibly problem" proactively.

This webpage serves as a means to communicate important updates from NIH and post resources to help UC Irvine researchers advance their own research with rigor and transparency.

Resources for enhancing reproducibility through rigor and transparency

May 18, 2017: Workshop led by Dr. Oswald Steward

This workshop is an update to the January 2017 workshop and provides information on how study sections are evaluating the 4 new required sections for scientific rigor and reproducibility.

Jan. 27, 2017: Workshop led by Drs. Oswald Steward, Christine Gall, and Alan Goldin

This workshop is an update to the December 2015 workshop and includes a panel of current study section members who provide information on how study sections are actually evaluating the 4 new required sections for scientific rigor and reproducibility.

Dec. 11, 2015: Workshop led by Dr. Oswald Steward 

In this workshop, Dr. Steward presents a comprehensive overview of the "reproducibility problem," new requirements for NIH grant applications, and suggested best practices to meet the new standards for scientific rigor.

NIH resources for enhancing reproducibility through rigor and transparency

  • Peer Reviewer Criteria for Rigor and Reproducibility — This NIH presentation clarifies the role of peer reviewers when assessing scientific rigor in grant applications. NIH has also issued a decision tree for reviewers when evaluating sex as a biological variable.

  • NIH Guidance and Examples of Reproducibility and Rigor in Grant Applications — NIH has published guidance to assist the extramural community in addressing rigor and reproducibility in grant applications as well as examples of rigorous experimental design from funded applications.

  • NIH Principles & Guidelines — These principles and guidelines are the result of a 2014 workshop held by the NIH, Nature Publishing Group and Science, which focused on identifying common opportunities in the scientific publishing arena to enhance rigor and further support research that is reproducible, robust and transparent.

  • NIH Publications — These are publications from NIH authors on the issue of reproducibility and NIH’s actions to enhance reproducibility. Topics include fixing cell lines and how to balance sex in cell and animal studies.

  • NIH Training Modules — NIH developed four (4) video modules with accompanying discussion materials that focus on integral components of reproducibility and rigor in the research endeavor, such as bias, blinding and exclusion criteria. NIH also released a training module developed for NIH staff that provides a comprehensive overview of NIH revised policy, grant application instructions, and review criteria for rigor and reproducibility.

  • NIH Meetings & Workshops — NIH has hosted several meetings and workshops focused on rigor, reproducibility, and transparency in scientific research and has posted materials from these events.

  • Expanded Guidelines for Reporting Preclinical Research — These guidelines are from external organizations that use the NIH principles and guidelines as a model, but have been expanded to fit the unique needs and challenges of specific fields.

  • NIH Revised Grant Application Instructions — This is an overview of the new instructions and revised review criteria, which will focus on four areas deemed important for enhancing rigor and transparency: (1) scientific premise of proposed research; (2) rigorous experimental design; (3) consideration of sex and other relevant biological variables; (4) authentication of key biological and/or chemical resources. There is also a revised version of the SF424 Application Instructions for applications due on or after Jan. 25, 2016.

  • NIH FAQS about Rigor and Reproducibility — The NIH answers frequently asked questions about changes to its policy and grant applications.

Additional resources

Authentication of key biological and/or chemical resources

  • Colleagues at UC San Diego have published examples of how to authenticate key resources such as cell lines and antibodies to include in NIH grant applications.

Articles about failure to replicate

Articles about sex as a biological variable

Articles about best practices for rigor and reproducibility