Evaluation of Innovation, Science and Economic Development Canada’s (ISED) Contribution to the Canada Foundation for Innovation

From: Innovation, Science and Economic Development Canada

January 2021

Evaluation of Innovation, Science and Economic Development Canada's (ISED) Contribution to the Canada Foundation for Innovation [PDF - 1.18 MB]

Table of contents

• Executive summary
• 1.0 Introduction
  • 1.1 Background
  • 1.2 Governance
  • 1.3 Stakeholders
  • 1.4 Program resources
  • 1.5 CFI funds
• 2.0 Methodology
  • 2.1 Evaluation objective and scope
  • 2.2 Evaluation questions
  • 2.3 Data collection methods
  • 2.4 Evaluation limitations
• 3.0 Findings
  • 3.1 Relevance
  • 3.2 Performance
  • 3.3 Efficiency
• 4.0 Conclusions and recommendations
  • 4.1 Conclusions
  • 4.2 Recommendations
• Annex A: Number of proposals submitted to CFI across Canada by area of application
• Annex B: ISED Performance Measurement Strategy: CFI Logic Model

List of acronyms

AEB

Audit and Evaluation Branch

CAMS

CFI Awards Management System

CIIF

College-Industry Innovation Fund

CFI

Canada Foundation for Innovation

HQP

Highly Qualified Personnel (includes students, technicians, and research assistants but excludes researchers)

IF

Innovation Fund

IOF

Infrastructure Operating Fund

ISED

Innovation, Science and Economic Development Canada

JELF

John R. Evans Leaders Fund

KI

Key Informant

MSI

Major Science Initiatives Fund

PDF

Post-Doctoral Fellow

PL

Project Leader

PPR

Project Progress Report

PU

Project User

SSH

Social Sciences and Humanities

SRS

Science and Research Sector

List of tables

• Table 1: CFI disbursements and operating expenses over the evaluation period

Executive summary

This report presents the results of an evaluation of Innovation, Science and Economic Development Canada's (ISED) contribution to the Canada Foundation for Innovation (CFI). Created by the federal government in 1997, the CFI is a not-for-profit corporation, which strives to build Canada's capacity to undertake world class research and technology development.

Overview

The CFI funds research infrastructure at eligible higher-education institutions (e.g., facilities and equipment) within the framework of five agreements with ISED. The Science and Research Sector (SRS) within ISED is responsible for managing these agreements with CFI, as well as the ongoing management and oversight of the funding. By providing funding for research facilities and equipment, the CFI pursues the following objectives:

• Support economic growth and job creation, as well as health and environmental quality through innovation;
• Increase Canada's capability to carry out important world-class scientific research and technology development;
• Expand research and job opportunities by providing support through research infrastructure for the development of highly qualified personnel (HQPs); and
• Promote productive networks and collaboration among Canadian universities, colleges, research hospitals, non-profit research institutions and the private sector.

Between 2014-15 and 2018-19, the CFI had disbursed a total of $1.8B (or about $360M per year) on infrastructure projects and related operating and maintenance costs through five main funding streams (i.e., the John R. Evans Leaders Fund, the Innovation Fund, the College-Industry Innovation Fund, the Major Science Initiatives Fund, and the Infrastructure Operating Fund).

Evaluation purpose and methodology

The objectives of this evaluation were to assess the relevance, performance and efficiency of ISED funding to CFI in accordance with the Treasury Board Policy on Results.  The evaluation was conducted by ISED's Audit and Evaluation Branch and covered the period from April 1, 2014 to March 31, 2019. The evaluation employed multiple data collection methods: a document and literature review; surveys; interviews; case studies, and administrative data analyses.

Findings

Relevance

The CFI is the primary federal funder for large-scale research infrastructure (up to 40% of the cost) at eligible higher-education institutions in Canada.  On average, the CFI contributed to 36% of the total project amount.  This CFI funding is typically complemented with a reciprocal amount of funding from provinces (35%), contributions from other partners (21% - including in-kind contributions), and 8% from institutions.

In general, the evaluation found that the CFI addresses the needs for new research infrastructure in Canada and upgrading and renewing existing ones. Some gaps were identified between the needs expressed by stakeholders and supports provided through the College-Industry Innovation Fund, where evidence shows that colleges do not request or access all available College-Industry Innovation Fund funding, likely due to fund eligibility requirements (e.g., requiring that colleges have several years of research experience).

In addition, the evaluation found that there is a need identified by stakeholders for the CFI to better respond to the needs of social sciences and humanities projects, particularly with regard to the eligibility of research infrastructure such as database-related infrastructure.

Performance

CFI funding for research infrastructure at eligible higher-education institutions has increased the capacity of institutions to attract and retain world-class researchers from within and outside Canada. In particular, the John R. Evans Leaders Fund has allowed institutions to meet the diverse needs of their researchers at different stages in their careers. The funding has also increased the capacity of institutions to conduct world-class research by improving the overall quality of the infrastructure, equipment and research at funded institutions.

Overall, CFI funding contributes to technology development, especially through the Innovation Fund and the College-Industry Innovation Fund. CFI funding has positively impacted the quality of training environments, enabling cutting edge research and advanced training at institutions and increasing their ability to attract and train highly qualified personnel and technical personnel.  Highly qualified personnels' use of CFI-funded infrastructure led to the acquisition of knowledge and diverse, transferable, and high-demand skills. Highly qualified personnel leverage in-demand skills and expertise to competitively pursue opportunities in the public and private sector, and further training at academic institutions.

The CFI also contributed to fostering more research collaborations in Canada and internationally and enhancing synergies with the private sector and across disciplines. CFI funding contributes to knowledge advancement through several research outputs (e.g., publications) and commercialization successes, such as spin-off companies.  CFI investments contribute to social, economic and environmental benefits for Canadians, including job creation and health protocols.

Efficiency

Overall, researchers and institutions are satisfied with the CFI's program design and delivery, though opportunities for improvement were identified (e.g., application and review process). Although stakeholders confirmed the Infrastructure Operating Fund provides much needed assistance, the fund appears less supportive than similar programs in other countries, particularly over the project's life span.

The CFI's governance structure and administration costs are aligned with similar organizations in other countries, however there is a need expressed by some stakeholders for improvements in coordination and collaboration with other funders (e.g., when making program changes). Furthermore, although the CFI has begun efforts to promote equity, diversity and inclusion practices within its organization and programming, its overall application remains in the early stages of implementation. 

Recommendations

The evaluation findings led to the recommendations noted below.

Recommendation 1: College-Industry Innovation Fund Access

ISED's Science and Research Sector should work with the CFI to identify and examine areas for improvement in the program design and administration of the College-Industry Innovation Fund, including opportunities to improve access (e.g., review eligibility criteria and approval processes) to better meet the needs of colleges and their private sector partners.

Recommendation 2: Eligible research infrastructure

ISED's Science and Research Sector should collaborate with the CFI to review the scope/criteria for eligible research infrastructure costs, particularly activities associated with the development of databases which can be a significant component in many research fields, including social sciences and humanities.

Recommendation 3: Improve Application process

ISED's Science and Research Sector should work with the CFI to examine opportunities to improve program application and review processes.

Recommendation 4: Effectiveness of support to operation and maintenance costs

ISED's Science and Research Sector should work with the CFI to explore opportunities to enhance the efficiency and effectiveness of the Infrastructure Operating Fund, including the timely draw down of available funding.

Recommendation 5: Collaboration

ISED's Science and Research Sector should work with the CFI to examine opportunities to improve collaboration and coordination with stakeholders.

1.0 Introduction

This report presents the results of an evaluation of Innovation, Science and Economic Development Canada's (ISED) contribution to the Canada Foundation for Innovation (CFI). Created by the federal government in 1997, the CFI is a not-for-profit corporation, which strives to build Canada's capacity to undertake world-class research and technology development.  The purpose of this evaluation is to assess the performance and efficiency of the CFI.

1.1 Background

Since its creation, the CFI has operated under agreements with ISED. The Science and Research Sector (SRS) within ISED is responsible for managing the agreements with the CFI, as well as the ongoing management and oversight of the funding.

The CFI funds research infrastructure at eligible higher-education institutions through these agreements and pursues the following objectives:

• Support economic growth and job creation, as well as health and environmental quality through innovation;
• Increase Canada's capability to carry out important world-class scientific research and technology development;
• Expand research and job opportunities by providing support through research infrastructure for the development of highly qualified personnel (HQPs - post-doctoral fellows, graduate students, undergraduates, technicians and professionals); and
• Promote productive networks and collaboration among Canadian universities, colleges, research hospitals, non-profit research institutions and the private sector.

The CFI supports all areas of research and works directly with eligible institutions rather than individual researchers. This is to ensure that the research infrastructure projects that are funded align with the strategic research plans of institutions.

As of March 31, 2019, the CFI has funded 11,314 research infrastructure projects at 160 eligible institutions (i.e., Canadian universities, colleges, research hospitals, and non-profit research institutions) across Canada, which includes advanced equipment, laboratories, databases, scientific collections, computer hardware and software, communications linkages and buildings necessary to conduct research.

In general, the CFI funds up to 40% of a research infrastructure project's cost. Institutions secure the remaining 60% of the required funding in partnership with provincial governments (in general up to 40%) and other public, private and non-profit organizations (including in-kind contributions).

As a result, CFI has awarded over $8.0 billion (as of March 31, 2019) to research infrastructure projects at eligible institutions with $11.4 billion^{Footnote 1} in matching funding from partners.

These institutions submit proposals for financial support and are assessed on three main criteria:

• the quality of the research and the need for infrastructure;
• the project's contribution to strengthening the capacity for innovation; and
• the potential benefits of the research to Canada.

The CFI commits funds to research institutions through a merit-based review process whereby committees comprised of volunteer researchers, research administrators and private sector representatives review proposals.

1.2 Governance

The 13 CFI Members are the highest governing body of the CFI and represents the Canadian public. Members are appointed for a five-year term. They are responsible for appointing external auditors, reviewing financial statements and approving the annual report.

The CFI also has a Board of Directors, which provides overall policy direction, reviews the results of merit-review assessments of applications and makes final decisions on projects to be funded. The Board is composed of 13 individuals, drawn from the private, institutional academic, research and public sectors. Consideration is given to geographical and gender representation.

The Government of Canada appoints six Directors including the Chair, while the remaining Directors are appointed by the CFI Members. Directors are nominated and appointed for three-year renewable terms.

The Board usually meets 3 to 4 times a year. An official from ISED and the department of Finance attend board meetings as observers.

Two CFI strategic roadmaps (2012–17 and 2018-23) were in place during the evaluation period and outlines the strategic direction of the CFI during their applicable period. The most recent strategic roadmap focuses on:

• Convergence, where a problem is identified that requires the deep integration of disciplines;
• Strong and growing emphasis on international collaboration and engagement; and
• Broadening Canada's research community to include a greater diversity of individuals.

This latter point could be seen as an alignment of the CFI with the Government of Canada's commitment to advancing gender equality as well as promoting Equity, Diversity and Inclusion (EDI).

1.3 Stakeholders

Stakeholders in the evaluation are defined as the organizations participating in the delivery of the program and those directly or indirectly benefitting from it. The CFI's stakeholders include:

• Research institutions, and the researchers and HQPs using the infrastructure;
• Other funding partners (provincial governments, private, not-for-profit and voluntary organizations);
• Granting Councils (CFI is a member of the management and steering committee of the Canada Research Chairs (CRC and also a member of the Canada Research Coordinating Committee) along with the granting councils)); and
• End-users (academia, public, private and not-for-profit organizations).

1.4 Program resources

The CFI has received its funding through five agreements with ISED, which specify the period covered, the funding amount, and the conditions under which funds are allocated. 

In January 2019, the CFI signed the most recent agreement with the Government of Canada that will transition the CFI to permanent ongoing funding. The agreement provides the CFI with $763 million over five years starting in 2018-19 and permanent funding at an ongoing level of up to $462 million per year in 2023-24 and onward.^{Footnote 2}

Funding and contribution agreements - Duration

During the evaluation period, the CFI had disbursed a total of $1.8B (or about $360M per year) to ultimate recipients (institutions).  In addition, CFI's operating expenses have traditionally been funded from interest generated on up-front payments from the federal government. Over the evaluation period, the CFI has maintained annual operating costs of $14M, or 4% of CFI disbursements, with about 65 full-time staff.

Sources: CFI Annual reports

1.5 CFI funds

Since its inception in 1997, the CFI has designed and managed several funds to support the capital cost of research infrastructure or the operating and maintenance costs. Over the years, the funds have evolved to reflect institutional and government priorities and the CFI's funding agreements. Between 2014-15 and 2018-19, the CFI had funded projects through four main funding streams that help to support the capital cost of research infrastructure.^{Footnote 3}

Innovation Fund (IF): The IF funds capital costs for new and existing research infrastructure that may lead to scientific breakthroughs and produce socio-economic benefits across the full spectrum of research in all disciplines. It is the largest CFI fund in terms of awarded amount (Figure 2) and has a rigorous review process. Typically, an IF competition has run every 2 or 3 years since 1998, with the last two taking place in 2015 and 2017.

John R. Evans Leaders Fund (JELF): The JELF is designed to attract and retain highly skilled researchers, providing awards on a regular basis (up to four times a year) to fund the capital costs of research infrastructure of individual researchers or small groups of researchers over two streams:

• Unaffiliated stream (S1) allows institutions to provide small groups of researchers with research infrastructure to conduct their research.
• Partnership stream (S2) is a joint process (funding of infrastructure and research support under one proposal) with Tri-agency Research Programs (e.g., Canada Research Chair).

Cyberinfrastructure Initiative (Cyber): The Cyber supported the infrastructure needs of computational and data-intensive researchers in collaboration with Compute Canada. It was introduced in 2015 and curtailed following the 2018 Budget which launched the Digital Research Infrastructure Strategy.

College-Industry Innovation Fund (CIIF): The CIIF supports capital costs of industry-relevant research infrastructure to foster partnerships between colleges and the private sector. There are two competition streams under the CIIF. Until March 2017, colleges were limited to submitting one project proposal per stream of up to $1M each. In response to feedback from institutions, the CFI now allows colleges to submit two proposals per stream, however proposals in the same stream cannot be in the same research area. The proposals are subject to an extensive review process as is the case for IF and Cyber proposals. The first competition was launched in 2012 and takes place annually.

Between 2014 and 2019, the vast majority of projects were awarded through the JELF (1,931) (Figure 1). However, 63% of all CFI funding ($664.8M) was allocated by the IF (Figure 2).

Between 2014 and 2019, the vast majority of projects were awarded through the JELF (Figure 1, Source: CAMS).

Although 87.5% of projects were awarded through the JELF, 63% of all CFI funding was allocated through the IF during the evaluation period.

In terms of area of research application, the bulk of the funding was provided to research infrastructure in Health (41%), Science (25%), and Engineering (25%), which is in line with the volume of applications received by discipline. Nonetheless, projects in Social Sciences and Humanities (SSH) only received 2.3% of the CFI funding awarded, although they represented 8.1% of the awarded projects.

Provinces with the highest volume of awarded projects were Ontario, Quebec, British Columbia, and Alberta (Figure 3). A map of CFI funding requests for all provinces by area of research can be found in Annex A.

Most of the awarded projects were located in Ontario and Quebec during the evaluation period (Figure 3, Source: CAMS).

Overall, the approval rate of projects was 67%, with small variations across provinces. Projects in health had a higher approval rate (70%) compared to projects in engineering and SSH (both 63%).

While projects requesting JELF (S1-S2) funding had an approval rate of almost 80%, the approval rate was 40% for CIIF and Cyber and 30% for IF (the lower approval rate for IF is largely due to a high volume of proposals, inherent to its design). The JELF higher approval rate may be partly explained by its design as an allocation-based fund, in which an institution's researchers are required to meet a standard of excellence, but do not compete with other proposals to the same degree as for other funds. Thus, more extensive merit-review processes (e.g., at least a two-stage process, consisting of Expert Committees and a Multidisciplinary Assessment Committee) are in place for the CIIF, Cyber, and IF.

In addition to funding the capital cost of research infrastructure, the CFI provides support for research infrastructure operating costs.

Infrastructure Operating Fund (IOF): The IOF is a one-time allocation to draw down by institutions from an "account" at the CFI, equal to 30% of the CFI awarded amount from IF, JELF, or CIIF funds to help cover a portion of the operating and maintenance costs of any CFI-funded infrastructure at that institution.

The IOF allocation associated with a project is allotted to an institution's account once all required documents have been received and accepted by CFI (e.g., signed award form). Although it may take one or two years before a specific project has eligible expenses that could be used to draw down the IOF allocation of an institution, the institutions can still draw down the allocation with eligible expenses from other funded projects at that institution.

Each institution decides how the IOF is used to support the operations and maintenance of CFI-funded infrastructure. As such, an institution may use the IOF generated by a project to support any of their CFI funded infrastructure. IOF funding was $318 M between 2014-15 and 2018-19.

Major Science Initiatives Fund (MSI): First launched in 2012, the MSI fund contributes to the operating and maintenance costs of 17 world-class national research facilities. Initially, the MSI funded up to 40% of their operating costs, but, under the 2019 contribution agreement (following the 2018 Budget announcement), a select group of eight national-scale major research facilities funded through the 2017-2022 MSI fund competition had their funding increased to 60%.^{Footnote 4} The MSI funding was $394M between 2014-15 and 2018-19. As well, infrastructure associated with an MSI project may receive or have previously received funding from the IOF.

2.0 Methodology

This section provides information on the evaluation objective and scope, issues and questions, data collection methods, and limitations.

2.1 Evaluation objective and scope

The evaluation of the CFI was required under the Financial Administration Act and was conducted by ISED's Audit and Evaluation Branch (AEB). The objectives of this evaluation were to assess the relevance and performance of ISED funding to the CFI based on the objectives/expected results outlined in the agreements, in accordance with the Treasury Board Secretariat Policy on Results (See Annex B – CFI Logic Model).

The evaluation covered the period from April 1, 2014 to March 31, 2019 and included any projects that requested CFI funding, were approved, or submitted at least a financial report or Project Progress Report (PPR or its equivalent) to the CFI during the evaluation period. Where it was possible, the evaluation collected and analyzed data by socio-demographic factors such as gender, race, ethnicity, age, and disability status in order to incorporate a GBA+ analysis into the evaluation.

The evaluation also includes projects that have started prior to April 1, 2014 and completed their reporting cycle (and have been operational for four to five years) at the beginning of the evaluation period in order to measure longer-term outcomes. For instance, these projects were assessed through case studies (for more details, see the section on case studies).

Previous Evaluations and Audit

In 2014, CFI contracted external consultants to assess three of the five core evaluation issues under the 2009 Directive on the Evaluation Function: continued need, achievement of expected outcomes and efficiency and economy.Moreover, ISED's AEB finalized an evaluation in March 2015 based on the contractor's work and further research to respond to the core issues of alignment with government priorities and federal roles and responsibilities. The key findings were:

• CFI met a continued need for ongoing investment in research infrastructure and it made a unique contribution to the federal government's support for the research ecosystem via its funding for large-scale research infrastructure.
• The CFI was achieving its immediate and intermediate expected outcomes. There was less evidence on CFI's contribution to its ultimate outcome of "social and economic benefits for Canadians". This is one of the challenges for this evaluation (see section 2.4).
• Overall, the CFI was delivered efficiently and economically during the evaluation period. Challenges identified included the lack of long-term stable funding for the CFI's support of Ultimate Recipients and a lack of ongoing operational funding beyond the five-year funding window. Budget 2018 addressed this issue by providing on going funding, starting in 2023-24.

CFI contracted the same firm who conducted its 2014 external evaluation, to conduct a Value-for-Money audit as required by its 2010 agreement. The audit provided a positive overall assessment of CFI's operations including its stakeholder engagement, the use of merit review in project selection, and its risk-based approach to project monitoring. It concluded that: "Overall, the CFI is a robust and mature organization that seeks continual improvement in order to deliver on its mandate…".

2.2 Evaluation questions

The evaluation addressed the following questions:

 Relevance

1. Is there a continued need for research infrastructure as supported by CFI and to what extent is CFI addressing that need?

Performance

2. To what extent has the CFI contributed to the achievement of its expected outcomes?

   Immediate outcomes

   1. 2.1 To what extent has the CFI contributed to the attraction and retention of world class researchers and to the capacity to conduct world-class research?
   2. 2.2 To what extent has the CFI contributed to technology development?
   3. 2.3 To what extent has the CFI contributed to an enriched training environment?

   Intermediate outcomes

   1. 2.4 To what extent has the CFI contributed to the acquisition of skills and expertise by Highly Qualified Personnel (HQP)?
   2. 2.5 To what extent has the CFI contributed to the establishment of productive teams, networks and collaborations?

   To the extent possible, AEB will attempt to measure through quantitative and qualitative methods the CFI contribution over the long-term horizon with regard to the following evaluation questions:

   Long-term outcomes

   1. 2.6 To what extent has CFI contributed to the advancement of knowledge, supported innovation, and commercialization?^{Footnote 5}

   Ultimate outcomes

   1. 2.7 To what extent has the CFI contributed to social, economic and environmental benefits for Canadians?

Efficiency

3. To what extent has the CFI been an efficient model for delivering support for research infrastructure in Canada?

2.3 Data collection methods

Multiple lines of evidence were used to address the evaluation questions.  The data collection methods included a document review, literature review, surveys, interviews, and an administrative data analysis.

Document Review

A document review of foundational documents, governmental reports, CFI plans, annual reports and results reports and studies, and other CFI research and reports (e.g. satisfaction surveys, previous evaluations and audits).

Literature Review

A review of alternative design, delivery and governance models in other countries (e.g., United States, United Kingdom, Germany and Australia) was conducted.

Surveys

Two online surveys were conducted between November 2019 and January 2020. The first survey invited 5,860 project leaders (PLs - individuals mandated by the institution to lead the CFI project) and principal users (PUs – team members and other individuals identified to participate in the CFI project) who led or participated in CFI-funded projects between 2014 and 2019. The second survey of 1,693 HQPs was generated from names and contact information provided by the PL/PUs. A total of 1,998 PLs/PUs completed the survey for a response rate of 34%. For HQPs, 530 surveys were completed for a response rate of 31%. In order to conduct further analyses, including GBA+, the survey results were linked with CFI administrative data (e.g., CFI Award Management System (CAMS) and Project Progress Reports (PPRs)).

Interviews

A total of 41 interviews were conducted across the following stakeholder categories: ISED and CFI officials (including Board Directors and Members), provincial and territorial representatives, funded institutions (president, vice-presidents of research, and liaison officers), other federal granting or funding agencies/partners, private sector and end users, unsuccessful institutions and eligible institutions that did not apply to CFI in the last five years. 

Administrative Data Analysis

Several analyses of CFI administrative data collected through the CAMS database and PPRs (progress and outcome data) were conducted to assess CFI expected outcomes and a financial analysis was conducted to assess efficiency.

Case studies

Eight case studies of CFI-funded projects were conducted.^{Footnote 6} These case studies were selected from across Canada and represent different types of institutions to provide examples of the longer-term outcomes, along with gathering the views of HQPs and end users (i.e., academia, private sector, and public sector).

2.4 Evaluation limitations

Attribution

The presence of other sources of federal funding and other partners (e.g., provincial government, NGOs and private sources) made it difficult to isolate and measure the impact of the CFI's contribution.  Further, research infrastructure is only one of the contributing factors (e.g. direct research funding) to CFI's longer-term outcomes. To alleviate these challenges, data collection methods (particularly survey and interview questions) were designed and articulated in a way that respondents could answer, to the extent possible, the impacts and incrementality of ISED's funding to CFI. 

Limited data on longer-term outcomes

Although there was limited information available on CFI's contribution to longer-term outcomes, the proposed case studies responded to this weakness, to the extent possible, by selecting a portion of the case studies from a sample of projects that were completed at the beginning of the evaluation period. Consequently, case studies assessed some of the program's medium and longer-term outcomes. Moreover, to further facilitate the assessment of longer-term outcomes, where possible, interviews were conducted with longstanding program stakeholders.

Respondent Bias

Another challenge and limitation associated with this evaluation was the potential for respondent bias. For example, many of the people interviewed or surveyed were involved in program design and delivery, or, were direct beneficiaries. As such, the findings may be positively biased.  To mitigate this, the purpose of the interview/survey and strict confidentiality was clearly communicated to participants and responses were cross-validated across stakeholder groups. As a result, interviews were conducted with experts who were not direct program recipients (e.g., researchers from unsuccessful institutions and those that did not even apply) and data collected from interviews was triangulated with other lines of evidence.

3.0 Findings

3.1 Relevance

3.1.1 Is there a continued need for research infrastructure as supported by CFI and to what extent is CFI addressing that need?

Evidence from Key informants (KIs) and the document review indicated that the CFI responded to an ongoing need for new research infrastructure in Canada and upgrading and renewing existing ones over the evaluation period, including multidisciplinary research which has emerged in recent years.

The document review and KIs also revealed that the CFI is unique in Canada as it is the primary federal funder for large-scale academic research infrastructure and is a unique model as it delivers funding not only to eligible postsecondary institutions but to all research institutions, including research hospitals and not-for-profit research institutions.

CFI investments also addressed the needs of the Canadian research community. Survey data indicated that 90% of PLs/PUs (who benefit from CFI funding) thought that the CFI addressed their current research infrastructure needs, with limited variations across all the funds and sectors of application. Many KIs commented that the CFI funds were highly needed as the number of project proposals exceeded the number of approved projects. In the 2017 IF competition, 15.2% of projects considered fundable by the highest review committee were not funded given the available IF budget envelope.

As the ongoing need for research infrastructure funding became increasingly important, the 2017 Fundamental Science Review (FSR) suggested that the CFI be awarded permanent funding. In Budget 2018, the government allocated permanent funding to CFI beginning in 2023-24. Many KIs view this change positively and it is expected to address the absence of predictable funding competitions that limited the ability of institutions to engage in long-term research infrastructure planning issue.

Available evidence from the document review and interviews suggest that the CFI does not duplicate, but rather complements other federal research infrastructure programs such as the Post-Secondary Institutions Strategic Investment Fund (a time-limited program that supported up to 50% of the eligible costs of infrastructure projects at post-secondary institutions and affiliated research and commercialization organizations across Canada). Launched in 2016-17 for three years, this program focused on generating direct economic activity and accelerating the strategic construction, repair and maintenance of infrastructure at post-secondary institutions across Canada in order to strengthen capacity for research and innovation and industry-relevant training and to support environmental sustainability.^{Footnote 7}

Similarly, the NSERC's Research Tools and Instrument grants program funds smaller equipment in science, for example. Some KIs explained that there are other federal programs that provide research infrastructure support (e.g., through Regional Development Agencies), but these programs are either regional or not equivalent in scale and size to the CFI.

The analysis of the CAMS data showed that the CFI funded on average 36% of project infrastructure costs (in percentage of the total disbursement amount). Provinces typically complement CFI's efforts as the main co-funders on CFI projects and data analysis showed that provinces funded 35% of project costs (Figure 4). The PL/PUs survey revealed that 83% of the respondents indicated that provincial support complemented CFI funding. This percentage was relatively constant across provinces and other factors (for example, gender or field of research).

Source of final funding disbursement during the evaluation period (Figure 4, Source: CAMS)

Of note, some KIs and case study interviewees pointed out that, in recent years, securing provincial matching contributions has been more challenging in some provinces (e.g., ON, NB, and SK) due to budget constraints, which has caused delays in initiating some projects. 

The private sector contribution is generally in the form of an in-kind contribution (non-monetary resources) such as a rebate on equipment, professional services, and training, or property transfer. The 2017 FSR noted that it can be more difficult to secure matching funding from the private sector, especially in the SSH discipline. Similar results were also found in the surveys.

Nevertheless, overall, the majority of KIs indicated that the CFI's leveraging ratios are appropriate.

College-Industry Innovation Fund Access

While 93% of PL/PU who received CIIF funds indicated that the CFI addressed their research needs, other lines of evidence suggest that support available through the CIIF is not being fully accessed due to some aspects of program design and administration. Through interviews, several KIs noted that colleges do not request or access all available CIIF funding. The financial data analysis corroborates this finding as only 60% of the CIIF amount allocated in the 2014 agreement ($38.5M) had been disbursed by 2018-19, and none of the amount in 2015 agreement ($45.0M) had been disbursed as of September 2019. The bulk of the expenditures allocated for CIIF in 2015 is expected to be spent from 2020-21 to 2022-23, five to seven years after the agreement. In comparison, CFI started to disburse allocated funding for IF, JELF, and MSI in the 2015 CA as soon as 2017-18.

Evidence suggests that this slow draw down of CIIF funding could be explained by several factors such as:

• A low number of proposals in several provinces (e.g., in BC, NS, NL, PEI and NT, less than 2% of their funding applications were for CIIF), in part due to the eligibility criteria requesting colleges have several years of research experience. Many colleges just recently started undertaking research activities, which prevented their eligibility.
• Only one CIIF competition is launched per year per stream, compared to up to four for JELF per stream.
• As noted, until March 2017, CIIF only offered colleges one project in each stream per year of up to $1M each (two projects per year per college may be considered if they are not in the same strategic applied research area).
• An average low approval rate (41%) during the evaluation period, especially with the small number of CIIF proposals that compete for the funding available.
• A lengthy approval process (6 months to a year) for the size of the awarded amount, especially compared to JELF. The private sector (e.g., SMEs) noted challenges partnering on projects (committing funding) given this length of delay before approval. NSERC's 2018 Evaluation of its College and Community Innovation (CCI) Program found similar issues with the lengthy turnaround time for applications to its college program. Stakeholders noted that long application review processes resulted in lost potential project partners and that the turnaround time for applications should be reduced to align with the needs and speed of industry partners.^{Footnote 8}
• In the same NSERC evaluation, stakeholders noted that although CFI equipment grants are available to colleges, the funds are perceived as difficult to receive or insufficient.^{Footnote 9}

Eligible research infrastructure

Qualitative evidence revealed that over the last 20 years, the need for funding for SSH has increased. However, many KIs indicated that support for SSH projects is often limited in terms of funding. For instance, only 1 SSH project was approved in the 2015 IF competition (out of 13 SSH project applications) and 3 other projects were approved in the 2017 IF competition (out of 16 SSH project application), which is lower than the average approval rate of IF projects (30%). Overall, data analysis revealed that SSH projects accounted for only 2.3% of the total funding.

In general, the approval rate for SSH projects across all CFI funds was lower than the rate for all other areas of research. During the evaluation period, CAMS data showed that the approval rate for SSH projects stood at 64.0% compared to an average of 69% for other areas of research.

In addition, a few KIs indicated that their institutions exclude more SSH projects from their CFI submissions, due to the perceived lower approval rates, and in order to attempt to secure more funding. Some other KIs noted that the CFI has kept a conservative definition of research infrastructure, mostly limiting the funding to physical equipment. Some activities related to the development of databases, often used in SSH research (e.g., research costs to produce a dataset or database, the development of sampling methodologies, fees for accessing a database for research use, and research activities using the database), have not been eligible. As such a social science project could not use CFI funding to develop a database, which is considered a foundational research tool and one that could be used by other researchers in the future.

3.2 Performance

3.2.1 To what extent has the CFI contributed to the attraction and retention of world-class researchers and to the capacity to conduct world-class research?

Attraction of Researchers

Over the evaluation period, approximately 98% of newly recruited^{Footnote 10}PLs reported each year in their project progress reports (PPR) that the availability of CFI-funded infrastructure was an important^{Footnote 11} factor in their decision to join their institution (Figure 5).

Almost all PLs indicated that CFI funding was an important factor in their decision to join or stay at their institution respectively (Figure 5; Source: PPR)

Among newly recruited PLs (2013-2017), about 55% originated outside of Canada, two-thirds of whom were from the United States and one-third from other countries (Figure 6). Of these foreign recruits, 51% were Canadian citizens or permanent residents, indicating that CFI funding encouraged Canadian researchers residing abroad to return.^{Footnote 12}As well, on average, nine of ten were recruited from academia while the remaining came from the public, private, and non-profit sectors (Figure 7).

Among newly recruited PLs (2013-17), approximately 55% originated outside of Canada, two-thirds of whom were recruited from the United States (Figure 6; Source: CFI Focusing on Results: Attraction and Retention, p.5)

Among newly recruited PLs, 90% were recruited from academic organizations while 10% were recruited from the public, private, and non-profit sector (Figure 7; Source: CFI Focusing on Results June 2018 Report: Attraction and Retention, p. 6.)

Case studies showed similar findings, as high-quality equipment enabled advanced research and attracted new faculty members and researchers to institutions. One PL noted that the CFI funded infrastructure contributed to attracting several applications for research positions from researchers across the globe, while another reported that the infrastructure led to several Canadian PhD students returning from the US after they had left to pursue their research.

Retention of Researchers

As shown in Figure 5, PPR data also indicated that around 94% of PLs on average felt that CFI funding was an important factor in their decision to stay at their institution. Similarly, the PL/PU survey revealed that 77% regarded CFI funding as an important factor in their decisions to remain in or change employment positions, only behind the research environment (88%) and availability of research funding (81%) (Figure 8).

Importance of Factors Influencing PL/PUs Employment Decisions (Figure 8; Source: Project Leader/Project User Survey)

In many case studies, interviewees recognized as leaders in their fields indicated that the availability and quality of CFI-funded infrastructure was a key reason they decided to remain at their universities. State-of-the-art equipment allowed researchers to work at the forefront of their fields without having to leave their institution. For example, a U15 (group of 15 Canadian Research Universities) faculty member indicated that having access to her CFI-funded lab was instrumental in retaining her.

CFI funding also helped institutions to increase their reputations and level the playing field with wealthier and more prestigious institutions abroad. The CFI's 2012-2017 MSI Summary Report found that several funded facilities increased their research capacity, leading to a rise in their domestic and international reputations and a greater ability to attract top researchers and users.^{Footnote 13}

Importance of JELF and IF

PPR data and interview evidence suggest that both the JELF and the IF programs contribute to attracting new researchers, but in different ways. 97.9% of newly recruited JELF PLs along with 90.5% of IF PLs reported their CFI funding was important, indicating that both funds contribute to attracting new researchers. However, the former is often used to attract younger researchers, while IF is in general used by institutions for PLs with established research experience.

Interview evidence revealed this distinction as well, as the majority of funded institutions indicated that they use JELF primarily to attract rather than retain researchers.

Interviews also revealed that JELF has allowed institutions to meet the diverse needs of their researchers at different stages in their careers. Some institutions reserve JELF exclusively for new, younger recruits or CRC appointments and to offset smaller institutional start-up grants. One KI described JELF as a "carrot" for recruitment while according to a small institution, the fund is "absolutely critical" to recruit and retain talent because it prevents the loss of expertise. Furthermore, in individual cases, institutions use JELF to support established researchers to acquire new or replace obsolete equipment; help a researcher branch into a new area of research; or to provide start-up equipment.

Some KIs identified a gap in support for mid-career researchers, explaining that while early career researchers have access to JELF and senior researchers with significant research experience have access to IF, mid-career individuals do not necessarily have the same level of support. One university representative explained that these researchers are often individuals who received initial JELF funding, but five to ten years later are using outdated equipment. Despite this issue and given the flexibility of JELF, some institutions have used the fund to address mid-career researchers' needs for instance, by renewing or upgrading the researcher's lab.

Distinction by Groups

GBA+ analysis of PPR data merged with the survey results found that those identifying as visible minorities or having a disability were slightly more likely to report that the CFI funding was important in attracting and retaining them. Additionally, PLs in small and rural communities were more likely to report that funding was important for retaining them in contrast to those in major urban centres, possibly meaning that projects had a greater impact in these communities to attract and retain researchers.

Capacity to conduct world-class research

All institutional and provincial KIs agreed that CFI funding has enhanced institutional capacity to conduct world-class research. Many institutions explained that CFI funding provided: leading-edge equipment that was unique nationally or internationally; attracted partnerships with industry on collaborative research opportunities; allowed for exploration of new areas and avenues of research; and advanced research culture. Furthermore, the infrastructure contributed to the development of institutional capacity in priority and niche areas ranging from immunology to material science.

Key Informants noted that:

• "In terms of manufacturing, everyone has 3D printers. However, we are the only institution in North America that has its own vineyard."
• "The list of companies that we work with in relation to projects that are funded through CFI is quite long. In the AI space, most importantly Google, Deepmind, and IBM [are] key partners."
• "CFI equipment enabled us to enter into new areas of research. We were not doing natural health products and were very focused on biochemical."
• "[CFI equipment] has definitely helped in creating, maintaining, and enhancing the research culture and emergent research culture of the institution."

All but one case study provided evidence that CFI projects had allowed researchers to conduct high-quality research, regarded internationally as leading edge. Institutions developed expertise in diverse areas including ocean and aquatic research, investigations of cardiovascular biomarkers, and autism. In some instances, the research infrastructure contributed to improved data collection capabilities and methodologies; better knowledge dissemination in high impact journals; active participation in collaborative international research initiatives; and a greater ability to leverage other funding sources. In another case, a CFI project led to a new therapy for patients recently experiencing a heart attack that was shown to be more effective and less expensive than current treatments.

Quality of the research infrastructure

Over the evaluation period, PLs rated the quality of CFI-funded highly specialized equipment favorably, with 88% on average reporting them as "state-of-the-art" (Figure 9). Notably, ratings were higher for IF projects (94%) and those in major urban centers, while lower in Atlantic provinces (82%) and for the SSH sector (72%). Furthermore, CFI-funded highly specialized equipment is expected to provide benefits for a long duration, as PLs reported annually an average useful remaining life^{Footnote 14} of 7.9 years. By discipline, projects in SSH reported the lowest average (5.5) in comparison to projects in engineering (9.2) and environment (8.2).

PLs rated the quality of CFI-funded highly specialized equipment favorably, with 88% on average reporting them as "state-of-the-art" (Figure 9; Source: PPRs)

PL/PU and HQP Perspectives

Surveys identified positive PL/PU and HQP assessments of the CFI's contribution to their research capacity (Figure 10). Notably, the proportion of these groups indicating that the quality of their infrastructure, quality of research, and quantity of research was high increased significantly after they received CFI funding in contrast to before CFI funding.

Many HQPs noted that the equipment allowed for more complex experiments and advanced techniques, while some PL/PUs explained that initial CFI funding was critical in enabling them to conduct research specific to their needs. PL/PUs in arts and SSH, women, those in the Atlantic, and those receiving JELF funding were more apt to note a lower quality of infrastructure before CFI funding.

The proportion of PL/PUs and HQPs indicating that the quality of their infrastructure, quality of research, and quantity of research was high increased significantly after CFI funding (Figure 10; Source: PL/PU and HQP Surveys)

Advancement of Research

Over the evaluation period, PPR data submitted by PLs showed that 8,126 internal and 12,336 external researchers per year on average advanced their research using infrastructure funded through a CFI award.

Some KIs noted that contribution to world-class research may depend on the fund, the funding amount, and the type of research. This was reflected in evidence showing that the CIIF and JELF contributed less to world-class research than the IF, which may be partly explained by differences in the funds' objectives and target audiences (e.g., early career and established researchers). PPR analysis corroborated this finding showing that on average, PLs funded through the IF reported between 21.8 and 33.6 external researchers per project per year advancing their research, in comparison to averages between 4.8 and 3.6 for JELF PLs (Figure 11).

On average, IF projects reported a greater number of external researchers advancing their research per project than JELF. (Figure 11; Source: PPRs)

3.2.2 To what extent has the CFI contributed to technology development?

Findings from the PPRs reveal that, over the evaluation period, 7% of JELF-funded projects per year reported technology outcomes such as patents granted, trademarks, copyrights, or industrial designs. This proportion was significantly greater among IF projects (23%) and CIIF (25% of required projects by CFI to report).

In addition to the variation by CFI funds, the presence of technology development outcomes vary by area of research application. For instance, there was a smaller share of projects in SSH (1.9%) and environment (3.7%) that reported technology development outcomes compared to engineering and health (10%). The proportion of female PLs reporting technology developments (1.8%) is almost five times lower than male PLs (8.8%).

Almost all KIs reported that many CFI-funded projects enabled the development of new technologies such as institutional patents, declarations of inventions, technology product or process developments. Some KIs specifically noted that JELF contributed to a lesser extent to technology developments than MSI and IF for instance, likely because research following JELF funding often occurs at a different part of the research spectrum (i.e. more fundamental research versus applied research). KIs also noted that it is difficult to exclusively link CFI infrastructure investment directly to technology development because CFI-funded projects receive funding from several sources.

The vast majority of the case studies also provided examples of contributions to technology development capacity. For instance, an Advanced Genomics project developed eDNA water sampling kits that allowed researchers to identify species and track the impact of aquatic pollutants, while the Resolute Bay Incoherent Scatter Radar project developed new red-line cameras for capturing images of the Northern Lights. In the health sector, the Biomedical Imaging project has led to scientific advancements that are contributing to the development of software for the diagnosis and treatment of cancer and neurological diseases. The Biomarkers project led to the development of new research instruments, methods and bioinformatics tools for predicting cardiovascular diseases.

3.2.3 To what extent has the CFI contributed to an enriched training environment?

Quality of Institutional Training Environments

Evaluation evidence revealed that the CFI has positively impacted the quality of training environments at funded institutions. Overall, PPR analysis showed that approximately 92% of PLs on average reported that CFI funding had a very high (72.3%) or high (20.7%) impact on the quality training environment (Figure 12).

Across the evaluation period, 92% of PLs believed that CFI-funded infrastructure had a high or very high impact on the quality of the training environment (Figure 12; Source: PPRs)

Consistent with these results, 92% of HQPs and 89% of PL/PUs surveyed reported that their institution's training environment was enriched considerably (Figure 13) while 77% of stakeholders, clients and PLs responding to the CFI's satisfaction surveys (2015, 2017, 2019) believe that CFI was enhancing training capacity.

89% of PL/PUs and 92% of HQPs surveyed reported that their institution's training environment was enriched considerably (Figure 13; Source: Surveys)

Case studies and key informant interviews further described how CFI funding significantly improved training environments. They included examples of CFI providing access to the latest advances in equipment, informatics and online data. Case studies described examples of CFI-funded equipment facilitating HQP training in the use of the latest wastewater treatment technologies and state-of-the-art 3D imaging equipment; access to up-to-date open source ocean data; and opportunities to use the newest approaches in genomics and bioinformatics. In one case, CFI infrastructure enabled a level of research complexity rarely duplicated elsewhere in the world.

Key informants noted that:

• "New equipment allowed me to increase the number of samples in my PhD research because analysis was in house."
• "Equipment that allowed for investigation of more sophisticated and important research questions."
• "Advanced techniques were made possible with CFI-funded infrastructure."
• "It helped me to access some cutting-edge equipment for high quality research."

Similarly, all funded institutions interviewed as KIs indicated that the CFI has made a significant contribution to enhancing their training environments. Some explained that funded infrastructure facilitated more complex training and provided HQPs with opportunities to work with external partners and disseminate knowledge. One institution highlighted that CFI equipment "allows new areas of research and more complex types of training to become possible" while another noted that HQPs worked closely with industry partners "to develop and disseminate new knowledge."

Attracting and Training HQPs and Technical Personnel

Improvements in the quality of training environments also positively impacted institutions' ability to attract and train HQPs and technical personnel. From 2014-15 to 2018-19, around 29,600 trainees each year (average of 17 trainees per project) used CFI- funded infrastructure (including those involved in CIIF and MSI projects) as a key resource in their research projects. Notably, MSI, IF and CIIF projects reported a higher average number of personnel trained than JELF projects, likely due to differences in funding amounts  and scope of projects. In 2017, almost 50% of HQPs were PhDs (31%) or Post-doctoral fellows (16%) (Figure 14). To note, there was little variation per year in the percentage distribution by type of HQPs over the evaluation period.

Breakdown of trainees in 2017 (Figure 14; Source: PPRs)

Over the same period, an average of 2,800 technical and professional personnel (1.6 per project) were trained on the use and maintenance of CFI infrastructure. As was the case for HQPs noted above, MSI, CIIF, and IF projects reported a higher average (over 100, 16, and 5.7 respectively) than JELF projects (1.1) which could be explained by differences in the funding size, objectives, and purpose of each fund.

The PL/PU and HQP surveys further supported these findings showing that 71% of PLs/PUs believe that the infrastructure had a significant impact on attracting HQPs, while 67% of HQPs reported that the infrastructure was important in attracting them to the institution (Figure 15).

71% of PLs/PUs surveyed believe that the infrastructure had a significant impact on attracting HQPs, while 67% of HQPs reported that the infrastructure was important in attracting them. (Figure 15; Source: Surveys)

Similar to PLs, some KIs explained that CFI-funded infrastructure attracts foreign HQPs and trainees by increasing the institution's international reputation. One funded institution remarked that funding "had a huge impact on the program's projection in Canada and internationally" while another indicated that her institution attracted many HQPs by becoming well-recognized.

Key informants noted that:

• "A new area (to me) was explored and resulted in international recognition of my expertise after several publications. This in turn attracted new students and resulted in greater numbers of students than had been previously supervised. As a result of the infrastructure, the students received training on state-of-the-art equipment and recognized this great privilege."
• "Having state-of art infrastructure has allowed me to join several multidisciplinary research projects and to attract outstanding HQP. Working with outstanding HQP in multidisciplinary projects has increased the quality and quantity of research output."

3.2.4 To what extent has the CFI contributed to the acquisition of skills and expertise by HQP?

The evaluation's lines of evidence showed that HQPs acquired a number of valuable and high-demand skills and knowledge using CFI-funded infrastructure. For instance, 93% of HQPs surveyed indicated that the infrastructure offered hands-on experience (Figure 16).

93% of HQPs indicated the infrastructure offered hands-on experience and opportunities to manipulate technical equipment (89%) (Figure 16; Source: HQP survey)

Further, 90% of HQPs indicated that the infrastructure had a considerable impact on their skilldevelopment, with less than 10% indicating a moderate (7%) to limited (2%) impact.^{Footnote 15}

Key informants noted that:

• "CFI funded infrastructure provided me excellent opportunities to learn and master new techniques. It also gave me hands-on training on advanced techniques."
• "state-of-the-art infrastructure . . . allowed me to learn and apply advanced forms of data analysis, which was reflected in my research products and conclusions."

90% of all HQPs surveyed gained research and analytical skills and technical and computer skills (Figure 17; Source: HQP survey)

Among the HQPs surveyed, notable distinctions between the groups included:

• PhD students were more likely to report that the skills and expertise gained contributed directly to their degree, thesis or publication(s); and acquiring research and analytical skills;
• HQPs accessing JELF projects, those in the natural sciences and engineering sector, and those working at a hospital or university, were more prone to say their experience contributed directly to their degree, thesis or publications; and
• The majority of female (95%) and male HQPs (90%) reported a considerable impact in developing their skills and expertise across all regions.

Case studies and the document review further expanded upon the skills, knowledge, and expertise that HQPs acquire. Several case studies reported that exposure to cutting-edge equipment and computational technologies helped HQPs to develop highly marketable and in-demand skills. More specifically, all the cases demonstrated that HQPs had acquired valuable new skills and expertise. In one project, HQPs developed specialized skills required for working in genetic labs and in another, HQPs learned quantitative imaging skills.

Similarly, in focus groups conducted by the CFI, students and post-doctoral fellows who used CFI-funded infrastructure in 2016 and 2017 noted that the most common benefits were the opportunity to interact with diverse colleagues which led to advanced research and analytical capabilities and understanding of research methodologies. The most frequently reported knowledge benefit among focus group participants was developing expertise in operating highly specialized equipment.

HQPs and Trainees gain a competitive advantage

Experience using CFI-funded infrastructure provides competitive advantages to HQPs and trainees. All lines of evidence showed that HQPs leveraged their skills and expertise to pursue professional opportunities. In KI interviews, funded institutions explained that enhanced training environments produced trainees who were more knowledgeable, innovative, and productive, resulting in better job prospects after graduation. Some KIs noted that HQPs were exposed to potential career prospects that they would not have considered without access to CFI-funded equipment. Colleges in particular highlighted the opportunity HQPs have to work with the private sector on collaborative projects by learning how to use the most appropriate equipment for applied research tasks. These collaborations often led to industry partners hiring HQPs after graduation.

Key informants noted that:

• "Many [HQPs] are hired by private sector companies that collaborated with us and the job they get is directly connected to the research projects they worked on."
• "Those we hire on specific projects get trained on the equipment . . . now they know how to operate it and they become more desirable to industry."

Survey evidence further showed that HQPs felt experience using the infrastructure provided a competitive advantage for further study (83%) and in the labour market (74%).

Over the evaluation period, PPR analysis showed that on average, HQPs who completed their training each year were most likely to pursue further training at a Canadian or foreign institution, followed by joining Canada's private sector. (Figure 18). Consistent with this finding, trainees participating in the CFI's 2016-2017 focus group discussions noted that through use of the infrastructure, they increased their competiveness to pursue future employment and training opportunities.

Over the evaluation period, PPR analysis showed that on average, HQPs who completed their training each year were most likely to pursue further training at a Canadian or foreign institution.^{Footnote *} (Figure 18; Source: PPRs)

Interviews and case studies provided examples of HQPs leveraging their research experience to launch companies and businesses. For example, one student launched a battery and fuel cell technology company that attracted significant investor interest, while another started a company that provided engineering services to the CFI-funded infrastructure he worked on as an HQP. Case studies also showed that HQPs went on to work for the public and private sector while others became technicians and faculty members at funded institutions.

3.2.5 To what extent has the CFI contributed to the establishment of productive teams, networks and collaborations?

One of the four CFI objectives is to promote productive networks and collaboration among Canadian universities, colleges, research hospitals.

According to the PLs/PUs survey, 78% stated that the CFI was important in fostering collaborations with academic researchers within their institution, especially for those working in the health sector (86%) compared to those in other sectors.

Meanwhile, another 71% stated the same, but with academic researchers outside their institution in Canada. Collaboration with academic researchers outside their institution was particularly present for those who worked on IF (78%) and MSI projects (90%) compared to those working on JELF and CIIF projects (less than 70%).

In addition, a few KI interviews revealed that the CFI also provided opportunities for larger and smaller universities to further collaborate with other Canadian research institutions on research projects. It was noted that the IF fosters team collaboration as it often involves multiple disciplines and institutions from different regions in Canada. Similarly, the MSI Fund is focused on the support of national hubs which also fosters collaboration among institutions, which has in some instances also encouraged more localized/regional collaborations.

Case studies confirmed that the CFI infrastructure allowed for regional and national research networks composed of multidisciplinary researchers from federal or provincial research organizations and Canadian and international institutions to collectively tackle key issues (e.g., health-related and environmental challenges).

Over the evaluation period, the PPR data corroborates these findings by showing that about 69% of projects per year led to at least one external research collaboration. As Figure 19 shows, environment and SSH were the sectors that most benefitted from the infrastructure in terms of research collaborations.

The PPRs also reveal that these collaborations resulted in more than 5,000 peer-reviewed co-publications (or 3.3 per project) and more than 6,000 conference co-presentations (or 3.8 per project) per year.

Environmental and SSH CFI projects are more likely to report research collaborations compared to engineering, health and science projects (Figure 19, source: PPRs)

In addition to fostering collaboration among academics within Canada, a majority of universities were able to develop international research collaborations as a result of CFI projects. The survey data showed that 83% of those involved in MSI projects indicated international collaboration was important, compared to 68% for IF, 63% for JELF, and 40% for CIIF.

More specifically, interviewees suggested that due to the CFI infrastructure, they were able to collaborate with researchers from the European Union, France, England and the USA. It was also noted that as international students who worked with CFI-funded infrastructure return to their respective countries, they enable further international research collaborations.

Most KIs suggested that the CFI enhanced their collaborations with the private sector and in some cases with multinational companies on large artificial intelligence projects (e.g., Amazon, Google, IBM).

Some universities also pointed out that CFI-funded infrastructure has been a magnet for industry as some private companies decided to initially set up their equipment within the institution as a result of CFI contribution. As a result, these companies conduct their R&D at the institution and also benefit from the CFI infrastructure.

3.2.6 To what extent has CFI contributed to the advancement of knowledge, supported innovation, and commercialization?

Based on PPR data, 94% of CFI projects per year (around 1,600 projects) led to research outputs during the evaluation period, resulting on average in approximately 34,000 research outputs per year, such as publications, seminars, and conferences. Close to 50% of these outputs were conferences presentations, 35% peer-reviewed publications, and the remaining 15% were reports, training material, etc.

The baseline/target was established at 1,816 projects per year (based on 2013-14 results) in the Performance Measurement Strategy (PMS) for the Contribution to the CFI.

Figure 20 shows a slight decline in the number of CFI projects with research outputs, consistent with the smaller number of IF projects during the evaluation period, as there was only one IF competition launched and completed between 2010 and 2014. For instance, over the evaluation period, projects funded through IF result on average in more research outputs (107) than those funded through JELF (15) per year. The difference is likely related to the project scope and the associated funding amounts (i.e., larger projects under IF) and the purpose of the funds.

Number of projects with research output (Figure 20, source: PPRs)

Although all interviewed institutions agreed that CFI projects have enabled knowledge advancement (e.g., peer-reviewed publications), colleges explained that publications in journals are not a priority in their mandates (CIIF averaged 21 technical publications after 36 months). Their focus is rather on data and knowledge transfer that can be used by industry to support commercialization

The case studies also demonstrate that the CFI projects contributed to significant advancements in knowledge. All cases resulted in research outputs such as publications, conferences, seminars, and presentations. In two cases, the CFI projects contributed to more than 100 research outputs per year.

CFI projects also led to innovation and commercialization outputs. Overall, the PPRs reveal that approximately 39% of PLs reported research agreements, Intellectual property right, licensing agreements or spinoff companies, with a greater proportion for those funded through IF (66%) than those funded through JELF (38%).

Specifically, each year on average, close to 50 projects reported at least one licensing agreement; nearly 300 Intellectual property rights were set up; and close to 50 spin-off companies.

Both college and university KIs provided examples of CFI projects contributing to innovation and commercialization outcomes.

KIs also noted that CFI is among many organizations and factors that contribute to scientific research discovery that leads to commercialization. Nonetheless, evidence shows that the CFI projects have been instrumental in many cases. KIs and case studies reported that a few spin-offs were launched and owned by former HQPs. For example, out of the eight case studies, three spin off companies were created, two pharmaceutical companies use CFI funded infrastructure to operate, and one company is working to bring new diagnostic tests for cancer to market.

3.2.7 To what extent has the CFI contributed to social, economic and environmental benefits for Canadians?

The PPR data indicated that approximately 48% of projects (or around 800 projects per year) have reported new or improved public policies, programs, or health care protocols and diagnostics over the evaluation period, a similar proportion than the baseline (49% in 2013-14).

A greater share of projects in SSH (69%) and environment (54%) reported socio-economic benefits compared to projects in natural sciences (45%) and health (41%). It is possible that projects resulting in socio-economic benefits in natural sciences and health may take more years to be achieved.

Similarly, the survey revealed that 42% of PLs/PUs indicated that the CFI projects led to a positive impact on public policies (e.g., improvements for Indigenous communities who suffer from a lack of food security) or programs and approximately 40% also indicated that the project led to practical applications in health care protocols and environmental benefits (e.g., diagnostic tests for cancer, effects of aquatic pollutants).  Those working in SSH along with those having access to the MSI funding were more apt to report new or improved public policies and programs.

In five of the eight case studies, the CFI project has contributed to social and environmental benefits including new systems for early warning of tsunamis and better protection for marine mammals. Some KIs also highlighted a number of benefits, including a project on agricultural insecticides (e.g., Neonics) and their toxicity to birds, which led to improvements in public policy and public awareness (i.e., the Canada Pest Management Regulatory Agency "ruled that by 2021, neonics won't be allowed to be sprayed on crops such as apples and tree nuts").^{Footnote 16}

In addition, in the PPR, about 28% of PLs reported jobs created due to their CFI projects, for a total of about 1,400 jobs per year (or 0.8 jobs per CFI project).

• While over 50% of IF projects reported job creation, this proportion was 25% to 30% for JELF projects. In addition, PLs reported on average four jobs created per IF projects compared to less than one job per JELF project.
• A greater proportion of projects in health (35%) led to job creation than in engineering and natural sciences (24%).

3.3 Efficiency

3.3.1 To what extent has the CFI been an efficient model for delivering support for research infrastructure in Canada?

Overall, the PL/PU survey reveal that the levels of satisfaction with the program design and the program delivery were high for those that benefit from CFI funding:  between 75% and 80% for those involved in IF, JELF, and CIIF project implementation. The level of satisfaction was slightly lower for those involved with MSI projects, where 63% of them provided a high rating for the delivery. The 2019 CFI satisfaction survey revealed similar results.

Although most institutions viewed the CFI's application process positively, the PLs/PUs survey revealed that 38% of researchers involved in an IF application process considered the process to be burdensome. In comparison, the process was considered less burdensome by researchers for the other funds (MSI - 35%, JELF and Cyber around 29%, and CIIF - 18%).

KIs identified a few potential improvements such as the application requirements should reflect the size and complexity of the proposal (e.g., application requirements to CIIF – see Relevance section). Case studies and KIs also raised concerns that rapid technological advances (e.g., big data) required institutions to constantly upgrade their equipment. They suggested that CFI could launch more frequent CIIF and IF competitions as CFI transitions to permanent ongoing funding. Many of them indicated that CFI could consider more regular IF competitions (e.g., every 2 years) so that they could better plan and coordinate proposals.

The merit review process was considered fair by most respondents in the survey, with some variation across the funds (89% for those involved in JELF projects, 80% for IF or CIIF, and 74% for MSI). Some KIs suggested to simplify and speed up the review process for CIIF. This could be considered in particular as a part of CIIF accessibility improvements (see Relevance section and recommendation).

In the survey, PL/PUs suggested additional ways to improve the design and delivery of CFI funds such as an increase of support for operations (see section on IOF), less bureaucracy (e.g., too many requirements throughout the processes, more flexibility in these requirements), and a faster process for receiving funds. On this latter point, a few KIs also indicated that the length of time between the application and funding disbursement is too long, which has led some projects to be delayed.

To what extent has the IOF been effective?

The CFI's IOF covers a portion of the operating and maintenance costs (O&M) of CFI-funded projects to ensure their optimal use.^{Footnote 17} All eligible projects generate an IOF allocation of 30% of the maximum CFI amount approved at award finalization.

Flexibility provided to institutions

KIs generally agreed with the IOF's flexibility, which allows institutions to allocate funding as they see fit. Each institution determines how funds are assigned among projects rather than granting the exact amount to the project that generated it. KIs explained that not all projects require their full IOF allocation, and thus, funds can be redirected based on actual O&M needs.

While some institutions allocate each project their full IOF allotment, others grant projects a proportion (e.g., two-thirds). Furthermore, many require an internal application process to access funding. KIs explained that requiring PLs to submit applications to access funds improves accountability by ensuring genuine need and appropriate spending. Other institutions hold a percentage of IOF centrally (e.g., 20%) to fund CFI projects at their discretion (e.g., those facing unexpected O&M costs or requiring emergency support).

Level of O&M support

Although most KIs regarded the IOF's flexibility positively, the evaluation identified challenges in terms of O&M and IOF support. A 2017 OECD report notes that in recent years, the challenge of sustaining research infrastructure has grown in importance in part due to the increasing quantity and diversity of these investments.^{Footnote 18} Many KIs agreed with this assessment and explained that since its inception, CFI investments in research infrastructure have coincided with a rise in O&M costs and hence, the need for long-term support.

Multiple funded institutions explained that the IOF works well to cover some O&M costs, however, some commented that more IOF funding is needed, especially due to the lack of provincial O&M support.^{Footnote 19} In contrast to the CFI's capital fund programs, provinces do not provide matching IOF commitments.

Key informants noted that:

• "The provincial government does not provide O&M so I feel that it's not enough." In some cases, the equipment may not be expensive, but it is expensive for operation. It would be good to increase the amount of IOF a bit more."
• "We have a lot to say about the IOF. It is not enough. It has been the same rate since the beginning.

When measured against comparable international programs, the IOF covers a smaller percentage of annual O&M costs for research infrastructure. The literature review indicates that, in general, O&M costs for research infrastructure varies from 10-30% annually of total project costs in countries (e.g., Germany, U.S., and U.K.). International programs such as the German Research Foundation provide 20% annually in overhead funding for the research projects it funds to cover indirect costs and the US federal government reimburses universities for facilities and administrative costs up to a cap of 26-28% annually.

In contrast, the IOF provides a one-time O&M allocation equal to 12% (i.e., 30% of the CFI contribution) of total project costs, equivalent to 2.4% annually over five years. In order to pay for O&M costs, some institutions charge user fees paid by the private sector and other researchers (e.g., academic users) who utilize CFI infrastructure, with each charged a different rate. In general, user fees were seen as reasonable by KIs, although expensive for small businesses. While user fees may help to offset a portion of O&M costs, in some cases, the revenue generated is not sufficient to operate established platforms and infrastructure.

Although the 2017 FSR noted that some of the indirect costs for research infrastructure are eligible under other federal programs, notably the Research Support Fund (RSF), it underlined: "the total federal contribution for infrastructure operating costs is but a small proportion of expenditures actually incurred by institutions. This contrasts with other countries that provide continuing support for these O&M costs as incurred… the current federal reimbursement rate for indirect costs paid through the RSF… is very low compared with other countries."

In addition, all but one case study identified a need for longer-term support for the O&M of CFI-funded infrastructure. Cases showed that current IOF allocations run out long before the research infrastructure reaches the end of its useful life, calling into question the long-term sustainability of CFI investments. One case study KI explained that while the expected life of the infrastructure was 25 years, IOF dollars would run out in a few years. The 2017 FSR calculated that based on the current IOF funding formula, it provides approximately 10 months of operating support for small equipment and tools.^{Footnote 20}

To note, the 2019 Contribution Agreement allows CFI to determine, in consultation with the Minister, the best allocation of the contribution among its funds (i.e., "in response to evolving context and community needs, in consultation with the minister").

Usage of IOF funding

Nonetheless, with the current IOF allocation, evidence indicates that the CFI has experienced some difficulty in getting institutions to use their available IOF allocations. According to a few interviewees, institutions may be exhausting other resources to cover O&M costs before accessing their IOF support.

IOF data shows that 41 institutions have used more than 90% of their IOF allocation (up to 99.6% in some cases), demonstrating that these institutions quickly draw down and use their IOF, to cover O&M needs.^{Footnote 21} However, 56 institutions still have unused IOF funding available after 5 years (29% of their funding, equivalent of $128M), which contributes to less O&M funding for researchers.

This challenge is also observable at the researcher level as 21% of PL/PUs surveyed indicated they had not received IOF funding. Although institutional discretion to allocate IOF funding likely explains this result, it may also reflect other challenges.

While the CFI does not impose a maximum number of years for which a project's IOF funding can be claimed, some institutions appear to only allow researchers to draw down IOF over 5 years, even though the average useful life of highly specialized equipment is 7.9 years. For example, documents show that at the University of Alberta, the maximum term for an IOF award is five years, after which unspent funds are reverted to a central fund.

The 2017 FSR explained that a potential gap exists in institutional IOF support for researchers maintaining small-scale equipment. The panel explained that these researchers may not secure IOF funding due to their institution's prioritization of large, multi-user capital projects and facilities.^{Footnote 22} As such, although they face higher proportional O&M costs^{Footnote 23}, these PLs do not receive funding which may lead to ineffective use of equipment and limited return on the initial capital investment.^{Footnote 24}

To what extent has the CFI's governance structure been effective?

Governance

The literature review indicated that organizations in other countries similar to the CFI have comparable governance structures, usually consisting of an appointed or elected board or council that ultimately decides which infrastructure proposals receive access to funds. ^{Footnote 25} Similar to the CFI, external and internal review bodies provide assessments that inform the board or council decision.

In interviews, most KIs had positive assessments of the effectiveness of CFI's governance structure. For example, some KIs explained that CFI's unique relationship with the federal government —relative to the tri-councils— allows it to be more independent and nimble. Notably, several KIs suggested that improvements could be made in the process for appointing new directors to prevent a significant number from turning over in a short period (e.g. federal appointments should be staggered).

Administration Costs and Reporting

Over the evaluation period, the CFI maintained annual operating costs of 4% of total expenditures, below the 6% limit stipulated in its 2019 Contribution Agreement with ISED (Figure 21). These costs are comparable to similar organizations in the United States, Australia and the United Kingdom as indicated in the literature review, and according to KIs, indicates the organization is operating efficiently.^{Footnote 26}

Over the evaluation period, CFI maintained its annual operating costs at 4%, below the 6% limit (Figure 21; Source: Financial Data)

Communications and Collaboration

The majority of KIs provided a positive assessment of the CFI's communication processes. In particular, funded institutions explained that the CFI has developed strong relationships and partnerships; responds quickly to inquiries; regularly shares information; and produces publications that clearly show the broader impact of its research infrastructure funding.

Several provinces and a few other stakeholders (e.g., colleges) indicated that CFI could enhance its consultation, coordination, and collaboration on funding decisions, announcements and program changes (e.g., eligible infrastructure expenses). For example, in some cases, changes to eligible infrastructure expenses were made without necessarily consulting the provinces, even though in some instances they were significant co-funders. Furthermore, some KIs noted that better coordination of CFI funding announcements with the provinces could improve consistency and enable more effective planning (e.g., ensuring provincial contributions are secured prior to announcing funding).

There is also an opportunity for the CFI to engage earlier and more systematically with the provinces and ISED regarding new funding programs or potential changes to plans and priorities. One KI explained that early engagement would allow parties to identify similar priorities; align funding sources; and increase the likelihood that the province can provide matching co-funding.

Single Source for Research Support

Canada's research support ecosystem does not have a single, coordinated window for researchers to access funding for infrastructure, equipment, research activities and O&M support. Some KIs remarked that improving coordination and streamlining among federal programs could be explored, such as the feasibility of a single application window for capital, operating, and research funds. In particular, KIs explained that programs in Canada require many applications to different funding sources, resulting in an uncoordinated and inconsistent approach that is challenging for researchers.

Key informants noted that:

• "Among the U15 VPs of Research, we wish we had one place where we could apply for research, equipment, and graduate student funding which is more or less what happens in the United States."
• "We dream of having access to various funding sources such as NSERC, CFI, and SSHRC in one pot."        
• "We could look at creating a bucket of grants that were joint CFI grants as well, so we would be awarding the research and equipment at the same time."         

In contrast, organizations abroad such as the United States' National Science Foundation are responsible for both funding research infrastructure and activities. One KI suggested that a more streamlined approach in Canada, such as that of the National Institutes of Health could enhance research coordination for large funding initiatives.

These findings are consistent with the 2017 FSR, which identified similar concerns from informants and noted that "the interdependence of capital and operating funds has received too little attention from the four agencies."^{Footnote 27} Notably, the Panel highlighted the importance of greater collaboration and coordination among the federal agencies, recommending the formation of a formal coordinating board for CFI, CIHR, SSHRC, and NSERC, chaired by the Chief Science Advisor of Canada, of which one of the priorities would be better aligning capital and operating support.

3.3.2 Extent to which CFI's efforts on equity, diversity and inclusion contribute to an inclusive and efficient delivery model.

The most recent CFI strategic roadmap indicated that the CFI would focus on broadening Canada's research community to include a greater diversity of individuals between 2018 and 2023. In order to do so, CFI committed to "consider policies that provide new researchers, women, Indigenous persons, persons with a disability and members of visible minority groups opportunities to take leadership roles in research initiatives."

As a result, the CFI has also deployed a number of efforts to foster and raise EDI awareness at the organizational level. During the evaluation period, all CFI staff and some governance members received EDI training (including unconscious bias training). The CFI has also enhanced the diversity of the membership on its Board of Directors taking into consideration gender, geographical, and experiential diversity.

The CFI is a member of the Tri-Agency^{Footnote 28} EDI initiative and part of the EDI training plan that was developed. Although not required by their agreements with ISED, the CFI could have been more proactive in respect of EDI by fully participating and signing the EDI Tri-Agency Action Plan that was developed through the initiative.^{Footnote 29} Similarly, although the CFI has two internal EDI working groups, their work has not yet been reported to the board of directors.

At the international level, in the United Kingdom, the UK Research and Innovation (UKRI - which offers research and infrastructure funding) has an External Advisory Group for equality, diversity and inclusion. The group identifies and prioritizes areas where most progress on outcomes on equality, diversity and inclusion can be achieved. Having an EDI coordinating committee at CFI could ensure that the organization adopts a comprehensive organizational EDI strategy and makes further progress in respect of EDI.

As of 2018, the DFG in Germany (which offers research and infrastructure funding) included in its proposal instructions to encourage applicants to consider diversity in the composition of research teams. Similarly, the CFI has started to integrate some EDI considerations in some of its programming. For instance, the 2020 IF program guidelines were adjusted to achieve more diversified research teams by allowing two researchers to act as co-leads, providing more flexibility for researchers that might need to take parental leave and to increase the likelihood that early-career researchers or members of underrepresented groups could co-lead a large initiative.

Tracking for EDI evidence

Although CFI has started to gather some performance data on diversity in March 2017 (as noted by some KIs and in some case studies), CFI has not publicly released EDI results on its webpage or in their annual reports (except limited information in a newsletter in May 2018).

Internal reports indicated that the percentage of female researchers named on project proposals decreased from 33% in 2016-17 to 30% in 2018-19 and slightly increased for visible minorities to 17% (from 15%). Documents also showed that the CFI publicly reported EDI aggregated results in a Canada Research Coordinating Committee report, showing that 33.1% of the 6,738 researchers in CAMS who answered personal information questions were women, 0.8% indigenous, 0.9% people with disability and 15% visible minority.^{Footnote 30}

Findings from PL/PU survey conducted for the evaluation indicated that women generally receive a smaller amount of funding compared to their share of projects. More specifically, as illustrated in Figure 22, women were particularly underrepresented in the IF.

Women generally received a smaller amount of funding compared to their share of projects. Women were particularly underrepresented in the IF. (Figure 22, Source: CAMS and PL/PU survey)

The survey data also demonstrated that visible minorities accounted for 21.7% of respondents, indigenous only represented 0.5% and 2.5% of respondents were individuals with disabilities.

In comparison, the Tri-Agency has released their EDI results on their respective webpages either through a report or in the case of SSHRC, an EDI dashboard.

In addition, CFI also seems to be behind the other Canadian funding programs such as the CRC. The CRC established its EDI Action Plan in 2017. Since then, the CRC has started to track actions to support institutions in making progress towards addressing the underrepresentation of four groups, which led to an increase in Chair awards to the four groups (women, visible minorities, Indigenous people and people with disabilities). For instance, the number of Chairs awarded to women increased by 7.8% between 2017 and 2019.^{Footnote 31}

Similar organizations in other countries have published information on EDI. For instance, the Australia Research Council (ARC - which offers research and infrastructure funding) collects EDI through application data and published the results in its annual report. ARC observed an increase in its share of women receiving funding, from 25.9% in 2015-16 to 31% in 2018-19. The UKRI also published gender data in their 2018-19 annual report, in addition to an EDI Interactive dashboard on competitive funding decisions (e.g., award rate by gender, age, ethnicity, and disability).^{Footnote 32}

4.0 Conclusions and recommendations

4.1 Conclusions

Relevance

• The CFI is the primary federal funder for large-scale research infrastructure (up to 40% of the cost) at eligible higher-education institutions in Canada. On average, the CFI contributed 36% of the total disbursement amount. CFI funding is typically complemented with a reciprocal amount of funding from provinces (35%), contributions from other partners (21% - including in-kind contributions), and 8% from institutions.
• In general, the evaluation found that the CFI addresses the needs for new research infrastructure in Canada and upgrading and renewing existing ones. Some gaps were identified between the needs of stakeholders and support provided through the College-Industry Innovation Fund, where evidence shows that colleges do not request or access all available College-Industry Innovation Fund funding, likely due to fund eligibility requirements (e.g., requiring that colleges have several years of research experience).
• In addition, the evaluation found that there is an identified need for the CFI to better respond to the needs of social sciences and humanities projects, particularly with regard to the eligibility of research infrastructure such as database-related infrastructure.

Performance

• CFI funding for research infrastructure at eligible higher-education institutions has increased the capacity of institutions to attract and retain world-class researchers from within and outside Canada. In particular, the John R. Evans Leaders Fund has allowed institutions to meet the diverse needs of their researchers at different stages in their careers. The funding has also increased the capacity of institutions to conduct world-class research by improving the overall quality of the infrastructure, equipment and research at funded institutions. Overall, CFI funding contributes to technology development, especially through the Innovation Fund and the College-Industry Innovation Fund.
• CFI funding has positively impacted the quality of training environments, enabling cutting edge research and advanced training at institutions and increasing their ability to attract and train highly qualified personnel and technical personnel.  Highly qualified personnels' use of CFI-funded infrastructure led to the acquisition of knowledge and diverse, transferable, and high-demand skills. Highly qualified personnel leverage in-demand skills and expertise to competitively pursue opportunities in the public and private sector, and further training at academic institutions.
• CFI also contributed to fostering more research collaborations both in Canada and internationally and enhancing synergies with the private sector and across disciplines. CFI funding contributes to advance knowledge through several research outputs (e.g., publications) and commercialization successes, such as spin-off companies. CFI investments contribute to social, economic and environmental benefits for Canadians, including job creation and health protocols.

Efficiency

• Overall, researchers and institutions are satisfied with the CFI's program design and delivery, though opportunities for improvement were identified (e.g., application and review process). Although stakeholders confirmed the Infrastructure Operating Fund provides much needed assistance, the fund appears less supportive than similar programs in other countries, particularly over the project's life span.
• The CFI's governance structure and administration costs are aligned with similar organizations in other countries, however there is a need expressed by some stakeholders for improvements in coordination and collaboration with other funders.
• CFI has begun efforts to promote practices within its organization and programming, but its overall application remains in the early stages of implementation.

4.2 Recommendations

The evaluation findings led to the recommendations noted below.

Recommendation 1: College-Industry Innovation Fund Access

ISED's Science and Research Sector should work with the CFI to identify and examine areas for improvement in the program design and administration of the College-Industry Innovation Fund, including opportunities to improve access (e.g., review eligibility criteria and approval processes) to better meet the needs of colleges and their private sector partners.

Recommendation 2: Eligible research infrastructure

ISED's Science and Research Sector should collaborate with the CFI to review the scope/criteria for eligible research infrastructure costs, particularly activities associated with the development of databases which can be a significant component in many research fields, including social sciences and humanities.

Recommendation 3: Improve Application process

ISED's Science and Research Sector should work with the CFI to examine opportunities to improve program application and review processes.

Recommendation 4: Effectiveness of support to operation and maintenance cost

ISED's Science and Research Sector should work with the CFI to explore opportunities to enhance the efficiency and effectiveness of the Infrastructure Operating Fund, including the timely draw down of available funding.

Recommendation 5: Collaboration

ISED's Science and Research Sector should work with the CFI to examine opportunities to improve collaboration and coordination with stakeholders.

Annex A: Number of proposals submitted to CFI across Canada by area of application

Annex B: ISED Performance Measurement Strategy: CFI Logic Model^{Footnote 33}