An important evaluation criterion in the Canada Research Chairs Program (CRCP) is the excellence of the nominee. A key factor in assessing this criterion is the productivity of the individual. When assessing their assigned nominations, reviewers must consider the following:
Career Interruptions and Personal Circumstances
Chairholders in Senior Administrative Positions
Publication conventions by discipline
Collaborative Endeavours and Interdisciplinary Research
Equity and Diversity within the Program’s Peer Review Process
Career Interruptions and Personal Circumstances
The Secretariat acknowledges that certain circumstances may legitimately affect a nominee’s record of research achievement. Nominees are encouraged to explain any personal circumstances (if applicable) in order to allow for a fair assessment of their research productivity. Reviewers and committee members are instructed to give careful consideration to, and be sensitive to the impact of, personal circumstances when assessing the nominee’s research productivity.
Examples of legitimate circumstances that could affect productivity:
- Career interruptions occur when, for health, family or other reasons, a nominee is taken away from his or her research work for an extended period of time (e.g., pregnancy, early childcare, eldercare, illness, etc.).
- Slow-downs occur when for health, family or other reasons (e.g., pregnancy, early childcare, eldercare, illness, etc.) a nominee’s research productivity slows down (i.e., the nominee was not taken completely away from his or her work).
- Publication delays occur when legitimate delays in the dissemination of research results due to circumstances that make it impossible or undesirable to publish important results (e.g., a delay in publication to protect intellectual property).
- Interdisciplinary research achievement and excellence indicators, or similar indicators in emerging areas, may not be as evident as those for research in the mainstream of a given field. Researchers conducting interdisciplinary or multidisciplinary research or research in emerging fields may face additional challenges that can affect productivity.
- Institutional characteristics vary between small, medium and large institutions, and may affect productivity. Opportunities for collaboration and networking, access to graduate programs, access to research space, access to mentoring and coaching, access to professional grant-writing and editing services or research office support may differ between institutions (e.g., a researcher affiliated with a small university that does not have a graduate program in his or her area of expertise will make different contributions to student training than a researcher from a large university that offers an extensive and well-established graduate program).
- Intellectual leadership involvement, such as the stewardship of large initiatives (e.g., leading a network or facility) is important, and may allow researchers to have influence and impact beyond their own institution at national and international levels. These intellectual leadership activities are legitimate activities of a Canada Research Chair. Such involvement may, in some cases, negatively affect their research outputs, as measured by traditional peer review metrics such as the publication record.
Please note: Senior-level administrative responsibilities are not considered acceptable reasons for a reduced level of productivity for Chairs that are being renewed (see below).
Chairholders in Senior Administrative Positions
Some researchers are highly productive and are able to deliver at a level expected of a chairholder while also being in senior level administrative positions. As such, the program does not have a formal policy prohibiting chairholders from holding these types of positions.
However, it is the responsibility of the institution (in collaboration with the chairholder) to ensure that chairholders who hold senior administrative positions have time for research so that they can carry out their program of research at the level that is expected of a Canada Research Chair. Chairholders who, because of a lack of time to devote to research, have difficulty meeting their research objectives, run the risk of not being renewed.
Publications
When assessing the productivity of the nominee, as determined by the quality and impact of their contributions to the field, reviewers should focus on the quality of a publication’s content and not limit their assessments to the number of publications; the quality of journals; or the impact of journals.
Publication Conventions by Discipline
Research publications can vary greatly among disciplines. In addition to the more traditional peer-reviewed journals, researchers also publish in books, chapters of books, articles, monographs, memoirs, special papers, reviews, conference/symposia proceedings and abstracts, patents, creative or artistic works, government publications, book reviews by the nominee or published reviews of his or her work, research reports, papers presented at scholarly meetings or conferences, and other forms of written scholarly expression, including participation in public discourse and debate, which constitute a contribution to research.
Some fast-moving research fields—such as some areas of computer science, genetics or microelectronics—use special means to reach the target audience quickly. Communications, quick-print reports, letters and electronic distribution of pre-prints are important vehicles for disseminating research results. All such contributions should be treated equally when assessing quality and impact, and reviewers should not regard certain types as “second class” or “grey literature.”
Venues with the highest impact (as measured by readership or attendance) may not be the most appropriate for a nominee’s research results; it is up to the candidate to explain their choice of venues for the dissemination of their research results.
When assessing productivity, reviewers should also be sensitive to legitimate delays in research and dissemination of results. Some circumstances make it impossible or undesirable for researchers to publish important results of their research prior to their nomination. For instance, the protection of intellectual property may require a delay in publication.
Collaborative Endeavours and Interdisciplinary Research
Increasingly, research on the most significant problems requires the combined knowledge, expertise and contributions of many researchers often from various disciplines. Such collaborative and concerted activities are encouraged and reviewers should be particularly careful to give adequate credit to effective research interaction. Creativity and innovation are at the heart of all research advances, whether made individually or in groups. The role of collaborative and interdisciplinary work as a means to greater achievement in research must be fully valued by the peer review process.
The indicators of achievement and excellence in interdisciplinary research, or in emerging areas, are often not as evident as those for research in the mainstream of a given field. Therefore, reviewers should recognize and appreciate the additional challenges inherent in interdisciplinary research.
Proposals that relate to interdisciplinary endeavours may appear somewhat unfocused compared to other research programs. Reviewers are asked to take this into account when assessing interdisciplinary work, as well as to keep an open mind to the practices and methodologies of across the wide range of disciplines included in CRC program nominations.
Equity and Diversity within the Program’s Peer Review Process
Commitment to Equity, Diversity and Inclusion
The Government of Canada and the CRCP are committed to excellence in research and research training for the benefit of Canadians. Achieving a more equitable, diverse and inclusive Canadian research enterprise is also essential to creating the excellent, innovative and impactful research necessary to seize opportunities and for responding to global challenges. As such, the program is committed to the federal government’s policies on non-discrimination and employment equity.
Participating institutions administer funds in partnership with the agencies and the Secretariat. Therefore, all institutions that accept agency funding are expected to make concerted efforts to meet their equity and diversity targets, and provide a supportive and inclusive workplace. This supports the goals of equity, diversity and inclusion within the CRCP and the broader Canadian research enterprise.
Reviewers and committee members must reinforce standards of rigor, fairness, respect and equity, diversity and inclusion throughout the peer review process, when assessing nominations, should:
- keep in mind that excellence remains the principal criteria in assessing CRCP nominations, but excellence should not be limited to its narrow and traditional sense and reviewers should ensure that non-traditional career paths and metrics of research (e.g., leadership, applied research, etc.) should be taken into account;
- be aware of the potential for systemic bias in the assessment of “quality” (for example, focusing only on journals ranked in traditional fields may disadvantage scholars in interdisciplinary or emerging fields who are more likely to be from under-represented groups, and therefore reviewers should focus on the quality of a publication’s content and not limit their assessments to the number of publications; the quality of journals; or the impact of journals);
- carefully consider “career interruptions and personal circumstances” as a legitimate explanation for delays in research productivity;
- recognize that all individuals carry assumptions and stereotypes about gender and should avoid unconscious bias when assessing the merit of a nomination;
- recognize that there is evidence of bias in universities in the allocation of resources and support to members of under-represented groups and ensure that the commitment of the university in the nomination package is as strong for members of designated groups as it is for others;
- ensure that they fairly assess the quality of training for internationally educated scholars rather than favoring graduates of domestic institutions who are part of existing networks;
- be aware of the roles that informal networks play in supporting some candidates and excluding others from processes that promote excellence—such as mentoring, coaching, networking, reviewing, invited lectures, etc.—and be aware of unintentional bias toward nominees because of access to or exclusion from these informal networks;
- be aware of the role of support services—professional grant writing, editing etc.—particularly at larger universities, during the CRCP application process (and where such services exist, access to them might not be equal to all candidates; these services may not be available to candidates from smaller institutions);
- provide feedback, where appropriate, that will assist unsuccessful applicants in advancing;
- be aware of the importance of emerging areas, including interdisciplinary, cross-sectoral research, action research, translational research, practice research, etc., which are legitimate and important, and foster excellent scholars who may be less well-established as a result of these new ways of conducting research; and
- be aware of the importance of research on diversity-related issues (gender studies, disability studies, immigration and settlement studies, Aboriginal studies).
In recent years, the Secretariat has taken several actions to ensure that all qualified researchers have access to—and opportunities to participate in—the program through its Equity, Diversity and Inclusion Practices.