Notice Number: NOT-HS-10-002
Update: The following update relating to this announcement has been issued:
- April 17, 2014 - NIH and AHRQ Announce Updated Policy for Application Submission. See Notice NOT-OD-14-074.
Key Dates
Release Date: December 22, 2009
Issued by
Agency for Healthcare Research and Quality (AHRQ), (http://www.ahrq.gov)
Background
The purpose of this Notice is to inform the research community of AHRQ’s plans to implement new procedures in peer review. AHRQ will be adopting many, but not all, of the procedures currently being implemented by NIH under the NIH Enhancing Peer Review initiative. This Notice specifies AHRQ’s policy in the following four areas:
- Enhanced Review Criteria
- Resubmission (Amended) Applications
- Restructured Application Forms and Instructions
- Page Limits and Designation of Early Stage Investigator
This action is similar to procedures currently being implemented at the National Institutes of Health (NIH), which undertook a formal and deliberative process beginning in 2007 to consider options for enhancing review with the following goals:
- Engaging the best reviewers
- Improving the quality and transparency of review
- Ensuring balanced and fair reviews
- Continuous review of peer review
Selected Policy Notices and Additional Information
NIH has detailed their implementation strategy and provided a broad range of detailed background information for Enhancing Peer Review, including a set of Frequently Asked Questions and Answers. These can be found http://enhancing-peer-review.nih.gov/. Additional information has also been provided in a number of NIH Guide policy notices, including:
NOT-OD-09-023 Enhancing Peer Review: The NIH Announces Updated Implementation Timeline
NOT-OD-09-024 Enhancing Peer Review: The NIH Announces New Scoring Procedures for Evaluation of Research Applications Received for Potential FY2010 Funding
NOT-OD-09-149 Restructured Application Forms and Instructions for Submissions for FY2011 Funding
AHRQ Implementation of New Peer Review Processes
AHRQ will follow many of the changes that NIH is currently implementing. This includes implementation of enhancing review criteria for evaluating the scientific and technical merit of research grant applications submitted to AHRQ through the peer review system. The enhanced review criteria (below) will be effective for all research grant and cooperative agreement applications submitted to AHRQ that are intended for due dates on or after January 25, 2010. A side-by-side comparison of the enhanced review criteria described below, and the criteria that will be replaced, is available on the NIH website (http://grants.nih.gov/grants/peer/guidelines_general/comparison_of_review_criteria.pdf).
These enhanced criteria may not be applicable for some other types of applications (e.g., conference applications, fellowship applications, specific Requests for Applications). Review criteria and any additional considerations for these programs will be described within the specific Funding Opportunity Announcement (FOA).
Description of Updated Review Criteria
Enhanced review criteria are generally described below:
Overall Impact. Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following five core review criteria, and additional review criteria (as applicable for the project proposed).
Core Review Criteria. Reviewers will consider each of the five review criteria below in the determination of scientific and technical merit, and give a separate score for each. An application does not need to be strong in all categories to be judged likely to have major scientific impact. For example, a project that by its nature is not innovative may be essential to advance a field.
Significance. Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?
Investigator(s). Are the PD/PI, collaborators, and other researchers well suited to the project? Do the investigators have appropriate experience and training? If the project is collaborative in nature, do the investigators have complementary and integrated expertise? Are the leadership approach, governance, and organizational structure appropriate for the project?
Innovation. Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?
Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?
If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed?
Environment.Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?
Additional Review Criteria. As applicable for the project proposed, reviewers will consider the following additional items in the determination of scientific and technical merit, but will not give separate scores for these items and should not consider them in providing an overall impact score.
Protections for Human Subjects. For research that involves human subjects but does not involve one of the six categories of research that are exempt under 45 CFR Part 46, the committee will evaluate the justification for involvement of human subjects and the proposed protections from research risk relating to their participation according to the following five review criteria: 1) risk to subjects, 2) adequacy of protection against risks, 3) potential benefits to the subjects and others, 4) importance of the knowledge to be gained, and 5) data and safety monitoring, as appropriate, for the proposed research.
For research that involves human subjects and meets the criteria for one or more of the six categories of research that are exempt under 45 CFR Part 46, the committee will evaluate: 1) the justification for the exemption, 2) human subjects involvement and characteristics, and 3) sources of materials.
Budget and Period Support. Reviewers will consider whether the budget and the requested period of support are fully justified and reasonable in relation to the proposed research.
Inclusion of Women and Minorities. Reviewers will evaluate the proposed plans for inclusion of minorities and members of both genders.
Inclusion of Priority Populations. Reviewers will evaluate the proposed plans for inclusion of AHRQ priority populations (see AHRQ policy notice at http://grants.nih.gov/grants/guide/notice-files/NOT-HS-03-010.html for details).
Resubmission Applications. When reviewing a Resubmission application (formerly called an amended application), the committee will evaluate the application as now presented, taking into consideration the responses to comments from the previous scientific review group and changes made to the project.
Renewal Applications. When reviewing a Renewal application (formerly called a competing continuation application), the committee will consider the progress made in the last funding period.
AHRQ Policy on Resubmission (Amended) Applications
Similar to the policy currently being implemented at NIH, AHRQ will modify its policy on resubmission applications. Beginning with applications intended for the January 25, 2010 due date, all original new applications (i.e., never submitted) and competing renewal applications will be permitted only a single amendment (A1). For this and subsequent cohorts of original new and competing renewal applications, any second amendment (A2) will be administratively withdrawn and not accepted for review. Applicants who fail to receive funding after two submissions may resubmit but only if the application is fundamentally revised to qualify as new.
A new application is expected to be substantially different in content and scope with more significant differences than are normally encountered in an amended application. Note that there is no time limit for the submission of the original and subsequent A1. This policy applies to all AHRQ grant applications unless otherwise specified in a particular FOA. Currently no amendments or resubmissions are permitted for applications received in response to an AHRQ Request for Applications (RFAs).
Original new and competing renewal applications that were submitted prior to January 25, 2010 will be permitted two amendments (A1 and A2). For these grandfathered applications, AHRQ expects that any A2 will be submitted no later than January 7, 2011, and AHRQ will not accept A2 applications after that date.
Restructured Application Forms and Instructions
Concurrent with the implementation of Enhancing Peer Review, is an effort to align the structure and content of the grant application with new review criteria. This effort also involves implementing a new application format and transitioning to the most recent version of the OMB-cleared application forms that include the changes required by the Federal Funding Accountability and Transparency Act (FFATA).
With this Notice AHRQ announces its alignment with NIH in using the new, restructured versions of the paper PHS 398 and electronic SF 424 (R&R) application packages.
Applicants must download and use the new, restructured application packages for submissions targeting due dates on or after January 25, 2010. These changes will affect all competing applications submitted to AHRQ, as follows:
- all applications (including individual Career Awards) electronically submitted using application packages that combine the SF 424 (R&R) with PHS 398 components (e.g., PHS 398 Research Plan Component and PHS 398 Career Development Supplement Form);
- all electronically submitted Individual NRSA Fellowship applications using application packages that combine the SF 424 (R&R) with the PHS Fellowship Supplemental Form; and
- all applications using the paper PHS 398 application package.
Applications submitted using incorrect application forms (including applications that have an incorrect mix of old and new forms) will be delayed and may not be reviewed. All new or resubmission applications targeting due dates on or after January 25, 2010 must use the new application forms and instructions (even if the initial submission followed the current application forms and instructions).
AHRQ will post the new application packages in all active FOAs and issue a Guide Notice when the new forms and instructions are available. Please closely follow the instructions detailed in the specific FOA as well as the application package.
Applicants MUST return to the FOA to download the new application forms and instructions for APPLICATIONS INTENDED FOR due dates on or after January 25, 2010.
The current version of the SF 424 (R&R) form is identified with the text ADOBE-FORMS-A in the competition ID field that is visible in the application header information and when downloading the application package from Grants.gov. The new SF 424 (R&R) form will be identified as ADOBE_FORMS_B . The new paper PHS 398 form will be dated June 2009.
Alignment of the Application with Peer Review Criteria. To coordinate with the enhanced peer review criteria, changes will be made to the following sections of the application forms and instructions: 1) Research Plan, 2) Resources, and 3) Biographical Sketch.
- Research Plan. Three sections of the current Research Plan (Background and Significance, Preliminary Studies/Progress Report, and Research Design and Methods) will be consolidated into a new single section within the Research Plan entitled Research Strategy. The new Research Strategy section (a single PDF upload in the PHS 398 Research Plan Component of the SF 424 (R&R)), will be sub-divided into three parts: Significance, Innovation, and Approach. The Approach sub-section will include both Preliminary Studies for New Applications and Progress Report for Renewal/Revision Applications.
- Resources. The Facilities and Other Resources section will be changed to require a description of how the scientific environment will contribute to the probability of success of the project, as well as a description of any other unique features of the environment. The Facilities and Other Resources section is part of the R&R Other Project Information in the SF 424 (R&R) application, and part of the Resources Format Page in the paper PHS 398 application.
- Biographical Sketch
- A new Personal Statement will be incorporated as Part A, changing the parts formerly called A, B, and C to Parts B, C, and D.
- Applicants should limit the list of selected peer-reviewed publications to no more than 15. These 15 publications should be chosen on the basis of most recent, importance to the field, and relevance to the proposed research.
The Biographical Sketch changes are also implemented for the PHS 2590 Noncompeting Continuation Progress Report. Although not immediately required in competing applications, institutions may begin to include a Personal Statement and 15 or fewer publications in Biographical Sketches (as described in this Policy Announcement) prior to January 25, 2010 due dates, if desired.
For more details, visit Details of Application Changes.
AHRQ Policy on Application Page Limits and Designation of Early Stage Investigator
AHRQ will not be using the new, significantly shorter page limits announced by NIH as part of Enhancing Peer Review. In general, AHRQ will maintain the page limits it currently uses. However, each AHRQ FOA will specify the page limit requirements for the Research Strategy section of the grant application.
Also, AHRQ will not be using the designation of Early Stage Investigator on grant applications that NIH is using as a component of Enhancing Peer Review.
Inquiries
Questions should be directed to:
Kishena C. Wadhwani, Ph.D., M.P.H.
Office of Extramural Research, Education, and Priority Populations
Agency for Healthcare Research and Quality
Rockville, MD 20850
Email: EnhancingPeerReview@ahrq.hhs.gov
and Human Services (HHS)
