Evaluation of the Computers for Schools Program

Recommended for approval to the Deputy Minister by the Departmental Evaluation Committee on February 28, 2012

Approved by the Deputy Minister on March 1, 2012

Table of contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Program Context
  • 1.2 Program Overview
  • 1.3 Program Resources
  • 1.4 Expected Results
• 2.0 Evaluation Methodology
  • 2.1 Evaluation Approach
  • 2.2 Objective and Scope
  • 2.3 Evaluation Issues and Questions
  • 2.4 Data Collection Methods
• 3.0 Findings
  • 3.1 Relevance
  • 3.2 Performance
• 4.0 Conclusions and Recommendation
  • 4.1 Relevance
  • 4.2 Performance
  • 4.3 Recommendation
• Management Response and Action Plan

Appendices (Separate document)

• Appendix A: List of Documents Reviewed
• Appendix B: Generic Interview Guide
• Appendix C: Beneficiary Survey Questionnaires
• Appendix D: Beneficiary Survey Tables
• Appendix E: Youth Survey Questionnaire
• Appendix F: Youth Survey Tables

Appendices are available via an Access to Information.

List of tables

• Table 1: Breakdown of Computers Distributed by Beneficiary Type
• Table 2: CFS Approved Budget
• Table 3: CFS Computers Supplied to Target Beneficiaries
• Table 4: Distribution of Computers by Type of Organization and Year
• Table 5: Donations by Year
• Table 6: Partner Contributions
• Table 7: Alternative Delivery Approaches
• Table 8: Operating Expenses as a Percentage of Total Program Expenses
• Table 9: Net Environmental Benefit of Refurbishing CFS Computers
• Table 10: Summary of Leveraged Funds by Fiscal Year

List of figures

• Figure 1: CFS Program Logic Model

Acronyms, Abbreviations and Definitions used in this Report

Executive Summary

Program Overview

Launched in 1993, Computers for Schools (CFS) is a national, partnership-based program that makes use of surplus computers from federal departments, provincial-territorial governments and the private sector. Computers are donated to refurbishment centres, where they are refurbished for use by the program's beneficiaries, which include schools, libraries and Canadian not-for-profit learning organizations.

The program also offers students and recent graduates of information technology programs professional experience refurbishing computers through the Technical Work Experience Program (TWEP), funded by the Government of Canada's Youth Employment Strategy (YES). Candidates are engaged in the refurbishment process, and develop skills in computer repair and software testing while cultivating softer skills such as time management and team building.

In December 2007, CFS received Cabinet approval to operate for an additional five years, leading up to March 31, 2013. The budget for the overall program is $45 million over five years (2008-09 to 2012-13)^{Footnote 1}.

Evaluation Purpose and Methodology

The purpose of this evaluation was to assess the ongoing relevance and performance of the CFS program and results will be used to support program renewal and improvements. The evaluation covers the period of 2008-09 to 2011-12.

The evaluation findings are based on the analysis of multiple lines of evidence. The methodology included document and literature reviews, data analysis, 70 key interviews, a survey of 303 users of CFS and 64 non-users, and a survey of 136 youth participating in the program (interns and co-op students).

Evaluation Findings

Relevance

The consistent donation, refurbishing and uptake of CFS computers over time demonstrates the continued relevance and need for the program. Apart from equipping educational institutions with technology, the program provides an environmentally responsible means to dispose of surplus computers. Schools would be negatively impacted if the CFS program did not exist and federal departments and agencies would have to find alternative ways to deal with surplus computer equipment. Additionally, through refurbishing and extending the useful life of computers, the program helps achieve the objectives of the Federal Sustainability Act.

The CFS program continues to address a number of government-wide priorities that cut across traditional departmental boundaries. It supports the federal commitment to sustainable management practices and initiatives and is consistent with government-wide priorities related to skills development and IC's strategic outcomes. IC is best positioned to continue with the administration of the CFS program as the department has the ability to leverage existing partnerships as well as has a proven track record in managing the program.

Performance

Overall, the evaluation found that the program is successful. Its success is rooted in the reciprocal benefits it offers to governments, the private sector and the general public. These benefits outweigh the costs associated with the program.

This assessment found that the program has largely achieved its medium-term outcomes. Specifically, the program is increasing computer awareness, knowledge and skills among youth. Further, it was found that the program's partners contribute to program sustainability and that the program is increasing corporate social responsibility and the sustainable development of partners, as well as contributing toward increased employability among TWEP program participants.

Evidence also suggests that the program is making progress towards its long-term outcomes. CFS has contributed to increasing computer literacy among Canadian youth by enhancing access to computers in schools and other venues. From an environmental perspective, refurbishing computers has reduced the environmental footprint of the government and other organizations.

While this assessment found that the program has been successful in achieving its expected outcomes, it should be noted that the evaluation faced some challenges in identifying the incremental impact of the program for some outcomes based on the current logic model and performance indicators. The logic model needs to be reworked to better reflect the inherent value of program outcomes, eliminate duplication and ensure that all outcomes are relevant.

Beyond expected outcomes, the program has yielded unanticipated benefits. Specifically, CFS has been used as a model to launch similar initiatives in other countries, recipients indicated that they are moving toward adopting provincial environmental standards, and youth are gaining work-related social skills.

There does not appear to be a more appropriate and efficient means to achieve objectives, relative to alternative delivery approaches. Alternatives would be less efficient, serve different clients, offer a lower level of service, or be less equitable for different provinces and territories.

The program has operated at a greater level of efficiency than what had been anticipated at the time of program renewal. From a cost-benefit perspective, the environmental benefits from eliminating landfill disposals were significantly greater than refurbishment costs. Finally, for every CFS dollar, the program leverages at least an additional two dollars each year from other sources.

Recommendation

An updated performance measurement system for the program is required. It should include:

1. Reworking the logic model and identifying appropriate and measurable program outcomes to evaluate the success of the program;
2. Revising the performance indicators (and including attainable targets where applicable) that can be collected to assess performance; and,
3. Developing a common database for performance information to improve reporting capacity.

1.0 Introduction

This report presents the results of an evaluation of the Computers for Schools (CFS) program conducted between June and November 2011. The purpose of the evaluation was to assess the relevance and performance of the CFS program.

This section of the report describes the program context and objectives. The remainder of this report is organized as follows:

• Section 2 presents the evaluation methodology;
• Section 3 presents the key findings related to the evaluation issues of relevance and performance; and,
• Section 4 summarizes the evaluation's conclusions and provides recommendations.

1.1 Program Context

Computers and the Internet are critical tools for information processing and sharing. The work and lives of Canadians are increasingly undertaken in the context of a global, knowledge-based economy, which demands a high level of skill in the area of information and communications technologies (ICT). The use of ICT contributes directly to labour productivity, and is strongly linked to economic development, jobs, learning and a wide range of business and government services. In order to develop proficiency in ICT skills, access to its tools, especially computers, has become essential for Canadians.

Many schools, libraries, and not-for-profit organizations face financial barriers and depend on donated computer equipment to supplement their equipment budgets. Youth in particular require access to computers in K-12 school settings, where the demand for computers often exceeds the available resources. By delivering refurbished computers, CFS helps fill the gap in demand and helps ensure that more Canadians can better avail themselves of the tools necessary to live and work successfully in the knowledge economy.

1.2 Program Overview

Launched in 1993, CFS is a national partnership-based program that makes use of surplus computers from federal departments, provincial-territorial governments and the private sector. Computers are donated to refurbishment centres, where they are refurbished for use by the program's beneficiaries, which include schools, libraries and registered not-for profit learning organizations.

Across the country, there are 14 organizations operating refurbishment centres, called recipients. Thirteen are not-for-profit organizations, each governed by its own Board of Directors including representation from the respective provincial or territorial government, school boards, the private sector and volunteer organizations. In P.E.I., where the provincial government delivers the program rather than a not-for profit organization, the province has engaged an Advisory Board composed of representatives from the government, school boards, and other organizations.

Recipients oversee over 50 different workshops across Canada, some of which are located within the schools themselves. They are additionally responsible for promoting CFS, developing local partnerships, and overseeing computer deliveries in their province or territory. Generally speaking, recipients have their own networks of partners and volunteers that facilitate the delivery of the program in their area. This support ranges from the free transportation of equipment to and from the refurbishment centre to donations of software. As well, there are national partnerships that provide in-kind support for transportation and operating systems (e.g. CN Rail and Microsoft).

Operational models vary among recipients depending on the resources available/donated. For example, one recipient may have to pay rent and adapt by utilizing more volunteers, while another may have secured donated space and be able to retain more paid staff. Of note are the TelecomPioneers, an organization that provides skilled volunteers—often retired telecommunications personnel – to support CFS with program delivery across the country.

Recipients may also offer students and recent graduates of information technology programs professional experience refurbishing computers through the Technical Work Experience Program (TWEP), funded by the Government of Canada's Youth Employment Strategy (YES). Candidates are engaged in the refurbishment process, and develop skills in computer repair and software testing while cultivating softer skills such as time management and team building.

Each year, Industry Canada consults with recipients to determine the standard level of computer that will be available free of charge to all schools requesting equipment. Computers donated to the program are assessed by recipients to determine whether or not they meet the minimum specifications. If not, the computers are sent for recycling. If they do meet the specifications, they are refurbished for the program's beneficiaries. Schools, as well as other beneficiaries, may request upgrades to the standard, such as additional memory or a new hard drive, and in these cases, the upgrades are provided on a cost-recovery basis.

Since 1993, over 1 million computers have been refurbished and distributed through the CFS program. The breakdown of computers distributed by beneficiary type is as follows:

1.3 Program Resources

The CFS Program is delivered throughout Canada by the Regional Operations Sector of Industry Canada. In addition, CFS reports to Human Resources and Skills Development Canada (HRSDC) as the coordinating body for YES on TWEP.

The budget for the CFS program including TWEP is $45 million over five years (2008-09 to 2012-13).^{Footnote 2} Table 2 provides details of the budget allocation of the program.

1.4 Expected Results

The program logic model was developed as part of the program's integrated results-based management and accountability framework (RMAF) and risk-based audit framework (RBAF) in February 2008. The logic model outlines the program's activities and outputs, as well as the intended short-term, intermediate, and longer-term outcomes. It serves as the basis for assessing the program's achievement of expected outcomes. While the short and medium-term outcomes are the anticipated direct results of CFS activities, these results are expected to contribute to the program's long-term and ultimate outcomes. The logic model is depicted in Figure 1.

Figure 1: CFS Program Logic Model

2.0 Evaluation Methodology

This section describes the overall approach of the evaluation, the objective and scope, the specific evaluation issues and questions that were addressed, the data collection methods, and data limitations.

2.1 Evaluation Approach

The evaluation was managed by Industry Canada's Audit and Evaluation Branch (AEB) and conducted by Performance Management Network Inc. An in-house evaluation manager worked closely with the consultant and was responsible for some elements of the assessment.

2.2 Objective and Scope

In accordance with the Policy on Evaluation and the Directive on the Evaluation Function, the purpose of the evaluation was to assess the core issues of relevance and performance. The results of this evaluation are intended to be used to support program renewal and improvements. The evaluation covered the period of 2008-09 to 2011-12.

2.3 Evaluation Issues and Questions

Based on the CFS logic model, the CFS integrated RMAF-RBAF of February 2008, and consultations with the program, the evaluation addressed the following questions:

Relevance

• To what extent does the program address a current need? Is the program still relevant?
• To what extent is CFS aligned with federal government roles and responsibilities? Is there overlap or duplication with other government programs?
• To what extent do the objectives of CFS continue to be consistent with government-wide priorities and departmental strategic objectives? Is Industry Canada best positioned to administer CFS?

Performance

• To what extent has CFS achieved its short-term outcomes?
• To what extent has CFS achieved its medium-term outcomes?
• To what extent has the program achieved progress toward its long-term and ultimate outcomes?
• Were there any unintended impacts or outcomes that can be attributed to the program?
• Are the most appropriate and efficient means being used to achieve objectives, relative to alternative delivery approaches?
• What has the utilization of resources (expended) been in relation to: a) the production of outputs and b) progress toward expected outcomes?

2.4 Data Collection Methods

Multiple lines of evidence, along with the triangulation of data, were used where possible to address all evaluation questions. The data collection methods included a review of documents and literature, data analysis, key interviews, a survey of CFS users and non-users, and a survey of youth participating in the program.

Document and Literature Review

A review of CFS program documents included various IC reports, the CFS Integrated Results-based Management Accountability Framework and Risk-based Audit Framework, Program Terms and Conditions, Contribution Agreements with recipient organizations, project applications and summary forms, and other reports (i.e. financial and operational data). It also included relevant government-wide strategies and policy documents.

The literature review focused on the environmental impacts of e-waste and benefits of re-use and recycling.

A list of documents and literature reviewed is provided in Appendix A.

Data Analysis

Data from the project summary forms (PSFs) and recipient progress reports were analysed (e.g. project costs, number of computers, reports on youth, partnership donations, cash or in-kind contributions) to assess the achievement of expected outcomes.

In order to report on program-level results, integrated databases were developed from the information in the progress reports, applications and PSFs. This required considerable effort from the evaluation team as data was not stored in a standardized fashion, making the process of compiling information cumbersome.

Interviews

A total of 69 interviews with 70 individuals were conducted, as follows:

• Industry Canada program management (2);
• Other government departments (3);
• Recipients (14 with 15 individuals);
• Workshop managers (15);
• Board of Directors of recipients organizations (27);
• Partners (5); and
• Experts/other stakeholders (3)

Interview guides were provided to each interviewee prior to the interview. Interviews were conducted in the official language identified by the interviewee and each interview was documented.

The generic interview guide is provided in Appendix B.

Survey of Target Beneficiaries

A web-based survey of schools, school boards and NPOs was conducted to assess the ongoing relevance of the program. The beneficiary list was generated either by obtaining contact information from recipients or by researching the internet. As a result, based on the information available, survey results predominantly included school and school board respondents and a limited number of NPOs.

Two jurisdictions did not participate in the survey. CFS Nova Scotia opted out of the survey because it does not deal directly with schools. School board representatives felt that schools would not be able to answer survey questions regarding CFS as they are not aware of where their computers originate. Also, CFS Yukon indicated that it does not supply CFS computers to schools, rather it provides computers directly to teachers and students^{Footnote 4}.

In total, 6,747 target beneficiaries were invited to participate in the survey via e-mail. A total of 303 users (64% were schools and school boards) and 64 non-users completed the survey. This represents a low response rate of 5.4% (or 367 out of 6,747). Nevertheless, a total sample of 367 (users and non-users) provides survey results which are accurate to within plus or minus 5.1%, 19 times out of 20.

Survey questionnaires are included in Appendix C. Detailed survey tables are provided in Appendix D.

Survey of Internship and Co-op Participants

A web-based survey of program interns and other youth participants was also conducted to assess the work experience and its usefulness in terms of practical experience, technical training and skills development (short-term outcomes), as well as the program's contribution to the employability of participants.

In total, approximately 300 youth were invited to complete the survey based on the information provided by recipients^{Footnote 5}. A total of 136 completed the survey, representing a response rate of approximately 45% (or 136 out of 300). Given the total number of youth participating in the CFS program (interns and co-ops) since 2008-09, a sample of 136 provides survey results which are accurate to within plus or minus 8.4%, 19 times out of 20.

The youth survey questionnaire is included in Appendix E and detailed survey tables are provided in Appendix F.

2.5 Limitations

The following were the three main limitations:

1. Data Quality and Systems
   • From the program data, there were observed data inconsistencies in the recipient progress reports making it difficult to assess some of the program outcomes. Numbers provided for computer laptops, monitors, etc. donated per quarter and numbers provided for volunteers and youth had extreme increases or decreases from one quarterly report to the next or were missing data values.
   • In addition, some data was only available through recipient applications (e.g. planned/estimates for future years) and not required in the progress reports. For example, information like cash and in-kind contributions of partners was only available based on planned contributions in the applications and this data was inconsistent across recipients. Some recipients included the estimated value of the computers and other equipment expected to be donated or the value of Microsoft software while others appeared not to. Data was reviewed and appropriate adjustments were made to ensure an adequate information base.
   • Further, data collected from the fourteen recipient organizations was only partially compiled at the outset of this assessment. Large amounts of data were split across various files and the evaluation team had to create a new database in order to assess relevance and performance.
2. Scope of Interviews
   • Interviews mostly represented individuals directly involved with the program. Other stakeholders (e.g. donors vs. non-donors, private sector firms refurbishing computer equipment, etc.) and subject matter experts (e.g. education, IT, environment, etc.) were difficult to identify. The interviews were supplemented with multiple lines of evidence in order to mitigate this.
3. Survey of Past Internship and Co-op Participants
   • A limited number of participants could be surveyed because information on this group was not available directly through IC, but rather through the recipient organizations. Where recipients are not required to maintain current contact information on internship/co-op participants who have completed their work terms, it was difficult to locate and survey past participants. The survey results were supplemented with other information in order to assess the evaluation issues and questions.

3.0 Findings

3.1 Relevance

3.1.1 To what extent does the program address a current need? Is the program still relevant?

In addressing the question of relevance of CFS objectives, the evaluation examined the extent to which CFS addresses the needs of targeted donors and beneficiaries.

Needs of Donors

Evidence suggests that the donation of computers and related equipment offers more than one benefit for donor organizations. Most obviously, the program provides a mechanism for disposing of surplus equipment. Interviewees suggested that the program offers economic and societal benefits via increasing access to computers in schools and the broader community. They also revealed that donors have an ongoing need to dispose of surplus equipment in accordance with environmental regulations pertaining to the disposal of e-waste and the security of information on used hard drives, and that CFS supports their efforts in that regard. Interviewees suggested that donors are motivated to give to CFS because they can be assured that processes are in place to wipe hard drives clean, ensuring the security of any legacy information.

Over the period of the evaluation, more than 620,000 pieces of computer equipment were donated to the program (e.g. desktops, monitors, laptops and printers). Donations from government and non-government organizations have been consistent over this period and meet the program's minimum standards. Program data also shows that the federal government is the largest donor of surplus computer equipment, supplying approximately 40% of all desktop computers, and 50% of laptops.

That said, the extent to which the program can meet the needs of donors depends on the capacity of recipient organizations. Some workshops may be restricted by the amount of physical space they have to store and refurbish equipment. The operational capacity to carry out refurbishment activities is also dependent on availability of human resources, including TWEP funding to hire interns to perform refurbishment activities. Interviewees from recipient organizations indicated that they are generally not looking for additional sources of donations as they lack the resources to increase production (i.e. the number of refurbished computers).

Needs of Targeted Beneficiaries

The consistent uptake of available CFS computers by beneficiaries demonstrates a need for the program. Generally speaking, schools require more powerful computers than other recipients. Interviewees noted that school budgets are limited with respect to information technology and the program has allowed school boards to free up resources to meet other educational needs (e.g. more teaching staff, facilities, specialized computer systems, etc.).

Table 3 shows that CFS delivered 1,092,063 computers between the program's inception in 1993 and March 2011. The table also shows that computers supplied had decreased annually prior to the current funding period (i.e. 2008-09) but has remained relatively stable over the last three years (evaluation period).

It was not possible to determine whether or not the demand for refurbished computers has changed since program inception using program data as the program does not collect this information. The program only collects the number of refurbished computers delivered. The program is reliant on donations and the ability of recipient organizations to supply computers.

However, the survey of beneficiaries conducted for this evaluation demonstrated that the demand for computers is increasing and that a decrease in CFS computers would have a negative impact on their organization. The survey showed an increasing demand for both desktop computers (from an average of 16.2 per organization in 2008-09 to 24.3 in 2010-11) and laptop computers (from 2.4 per organization in 2008-09 to 4.2 in 2010-11) by users.

Additionally, respondents indicated that the program is very important to them, and when asked what would happen if the program was not available, 97% of beneficiaries indicated it would have a negative impact (major impact 59%, minor impact 38%) on their organization. Respondents stated that students would have less access to computers, that it would result in budget shortfalls, and that desktops would not be replaced as fast.

The survey also showed that that a majority of computers used by beneficiaries come from CFS. For example, a total of 28,992 computers^{Footnote 6} were acquired by the 303 user organizations responding to the survey (77% came from CFS, while 23% came from other sources) since 2008-09. Users generally agreed that the types of computers available are well aligned with the type of computers their organizations need (mean of 7.7 out of 10; 58% giving it a rating of 8 or more), and that the program is meeting their needs fairly well (mean of 7.3 out of 10; 53% giving a rating of 8 or more).

Non-users surveyed were asked why they had never used the CFS program to further assess whether the program was meeting its targeted population.^{Footnote 7} Only two responses were noted by more than one individual: no need to use the program (three respondents) and did not use the program due to a decision to use standardized equipment (three respondents).

Finally, a report published by Statistics Canada indicated that two out of every three school principals identified financing the purchase of computers and related electronic equipment as a major concern. In addition, schools find that maintaining an adequate number of computers that are up to date in terms of both hardware and software is a struggle in the face of competing demands for financial resources.^{Footnote 8} The program will continue to be relevant in that there will be an on-going demand for computers, particularly in Canadian schools.

3.1.2 To what extent is CFS aligned with federal government roles and responsibilities? Is there overlap or duplication with other government programs?

The CFS program helps meet federal roles and responsibilities with respect to the disposal of crown assets and sustainable development. The 2006 Treasury Board (TB) Directive on Disposal of Surplus Materiel (Section 4.1) states that Public Works and Government Services Canada (PWGSC) is responsible for disposal of all surplus materiel assets for which authority has not been given either by legislation, or by TB, to another department. Section 5.2 of the Directive states that IC is responsible for the management of federal equipment contributions to the CFS program. Departments must offer right of first refusal of all surplus personal computers, laptops, servers, monitors, keyboards, mice, printers, modems, hubs network cards, hard drives, etc., to IC's CFS program. Material that cannot be used by CFS is then transferred to the Crown Assets Distribution Centre and placed in public auction. Material not suitable for auction is managed by PWGSC's Office for the Greening of Government Operations which is responsible for the management and disposal of surplus electronic equipment through a national Standing Offer or via established provincial identified recyclers.

According to the Federal Sustainability Act of 2008, each Department is responsible for minimizing its own environmental footprint. The CFS program is identified as part of the mandatory implementation strategy for achieving the Federal Sustainable Development Strategy target dealing with reuse and recycling of all surplus electronic and electrical equipment in an environmentally sound and secure manner by March 31, 2014.^{Footnote 9} CFS plays a unique role in supporting this requirement by providing a government-wide approach for ensuring that significant quantities of surplus computer equipment is collected, refurbished and distributed to schools and other beneficiaries on a national level. Without CFS, surplus computers would be sent for recycling rather than extending their useful life through re-use.

The CFS program also plays a role in ensuring the protection of personal and other information from government computers. Although it is the responsibility of federal departments and agencies to cleanse all donated computers of classified and protected information prior to the release to CFS, the program provides additional assurance by also wiping hard drives a second time. The hard drive wiping procedure is done in accordance to standards set by the Royal Canadian Mounted Police and the Communications Security Establishment Canada. In response to a recent audit conducted by the Privacy Commissioner, CFS worked collaboratively with Treasury Board, as well as with all other federal departments and agencies, on the development and implementation of a new surplus certification report to ensure computers are cleansed prior to donating them to the program.^{Footnote 10}

A key objective of the CFS program is to produce ready-to-use computers for schools, libraries and not-for-profit learning organizations. Although the provision of computers to these types of organizations is not a federal responsibility, these types of organizations (especially schools) have been targeted as a means of addressing federal priorities, as discussed above.

Interviewees noted that federal involvement in the program was important because:

• The program is national in scope with a significant proportion of donations originating from federal government departments and agencies. Federal donations represent 41% of desktop computers, 50% of laptops, 54% of monitors, 37% of printers, and 68% of all other equipment donated. This allows the program to redistribute computer assets across the country in a fair way.
• The program demonstrates federal leadership on environmental issues and raises awareness about the benefits of reuse and recycling.
• CFS provides a positive profile for the federal government in its commitment to provide access to technology, skills development and youth employment.
• Recipients would not have the financial or human resource capacity to operate without a combination of federal funding as well as cash and in-kind contributions from other organizations.

The CFS program is distinct. There is no known comparable program (as confirmed by the document review, interviews and the survey of beneficiaries where only 4% indicated that they were aware of similar programs^{Footnote 11}). Interviewees indicated that there may be some organizations that refurbish computers but none operate on a national scale, nor do they have the same objectives as the CFS program. Generally, the organizations mentioned were small local private enterprises that sell refurbished computers, but concerns were raised from interviewees with regard to the quality control and the disposal of environmental waste of some of these companies.

3.1.3 To what extent do the objectives of CFS continue to be consistent with government-wide priorities and departmental strategic objectives? Is Industry Canada best positioned to administer CFS?

Alignment with federal government priorities

The objectives of CFS are aligned with the following government-wide priorities:

• Greening Government Operations: One of the key themes of the Federal Sustainable Development Strategy is aimed at shrinking the environmental footprint, beginning with government. The program continues to meet the ongoing need for an environmentally responsible solution for disposing of surplus government computers, and is therefore well aligned with Greening of Government initiatives. The environmental stewardship aspects of the program have taken on a greater degree of importance in recent years.
• S&T Strategyd – People Advantage: Mobilizing S&T to Canada's Advantage also identifies skills development as a key priority. The strategy states that "Canada must grow its base of knowledge workers by developing, attracting, and retaining the highly skilled people we need to thrive in a modern global economy."^{Footnote 12}
• Youth Employment Strategy: The Youth Employment Strategy (YES), coordinated by HRSDC, is the Government's key labour market program to help young people. Through the utilization of TWEP funding, the CFS program has provided technical employment to thousands of youth over the years. Youth hired by CFS recipient organizations gain experience relevant to future careers in ICT as they are provided with hands-on training in computer refurbishment activities, software testing and other administrative-related work.

Alignment with Industry Canada's strategic priorities

According to the department's Program Activity Architecture, CFS addresses the Strategic Outcome: Canadian businesses and communities are competitive. Currently, CFS is identified as a sub-activity under Community Economic Development, along with Community Access, Community Futures, Northern Ontario Development, and Linguistic Duality and Official Languages programs.

Under its founding legislation, the Department of Industry Act, IC is mandated to not only foster a growing, competitive knowledge-based Canadian economy but also to promote sustainable development which CFS contributes to through some of its program outcomes (i.e. reduced environmental footprint).^{Footnote 13}

The ultimate outcome, identified in the CFS logic model, is to "strengthen public national ICT infrastructure that provides equity of opportunity for Canadians to acquire the ICT knowledge and skills they need to participate in the knowledge-based economy through the reuse of computer assets." The objectives of the CFS program and the intended (long-term and ultimate) outcomes are therefore aligned with IC's mandate to foster a knowledge-based Canadian economy and also to its role in promoting sustainable development.

Interviewees stressed that it is important for the federal government to be involved in a program of this nature. Interviewees commented that CFS is in line with federal priorities related to sustainable development, ICT skills development, youth employment, economic competitiveness and connectivity.

Is IC best positioned to administer the CFS program?

The evaluation considered input from six federal government interviewees, as well as a review of documentation, in assessing whether IC is the most appropriate department to administer the CFS program. Federal government interviewees were consistent in their views and noted that IC and PWGSC have considered this question previously.

Interviewees felt that although PWGSC has federal responsibility for the management of surplus assets and a leadership role for the Greening of Government Operations, IC is best positioned to administer the CFS program because the objectives of the program go far beyond managing surplus assets. As well, it was noted that IC has greater capacity in the administration of grants and contributions (G&C) programs and has a proven record of success in managing the CFS program for more than 17 years.

In addition, interviewees recognized that although HRSDC coordinates the YES and provides funding to support the youth interns, the skills development component and distribution of computers within Canadian schools helps to support IC's mandate to foster a knowledge-based economy to ensure Canadian competitiveness in the global economy. IC's broad industry perspective and contacts with the private sector were also identified as important success factors in leveraging partnerships.

Interviewees noted that the CFS program could not have achieved the same results without its unique network of public and private sector partners, and licensed not-for-profit recipient organizations. The importance of IC's ability to be in regular contact with industry across the country and maintain dialogue with provincial and territorial governments, the private sector and other stakeholders is highlighted in IC's departmental Business Plan for 2011-12.^{Footnote 14}

This provides a further rationale for continuing IC's administration of the CFS program. Other departments may not be as well positioned as IC in this regard.

3.2 Performance

3.2.1 Achievement of Expected Outcomes

This section provides evidence related to the outcomes identified in the program logic model based on the indicators outlined in the integrated program RMAF-RBAF. This evaluation found that the logic model needs to be revisited as some outcomes are less relevant than others and there is a measure of overlap between short and medium-term outcomes. For example, providing practical work experience to participants and increasing their employability are too closely linked to represent short and medium program outcomes, respectively.

Additionally, it should be noted that some indicators are repetitive while others were not collected. However, as these were the approved performance indicators which set the basis for the collection of performance information, they were used as the basis for assessing the program's achievement of expected outcomes. Nevertheless, where possible, the indicators were supplemented with other evidence to strengthen the reported findings.

3.2.1.1 To what extent has CFS achieved its short-term outcomes?

Improved access to computers by Canadian students and adult learners

Table 4 shows the distribution of CFS computers by type of organization for the evaluation period (2008-09 to 2010-11).

The data shows that, in total, 195,298 computers were refurbished and delivered from 2008-09 to 2010-11. The majority of computers (80%) were distributed to schools, thus improving access to computers by Canadian students. The other 20% of computers went to the other beneficiary organizations that may provide access to both students and adult learners.

The total number of computers available in public schools, libraries, and recipient learning organizations is not available. However, based on the survey of users and non-users, CFS computers represent approximately 77% of all computers in surveyed eligible public schools, libraries, and non-profit learning organizations. This suggests that CFS is providing the majority of computers available for use among its beneficiaries.

Interviewees also believed that the program had improved access to computers and noted that "the numbers speak for themselves". For schools in particular, general observations from interviewees stressed the increased number of computers in classrooms due to CFS (e.g. one computer for every two students now instead of one computer for every ten students) and the ability to provide learning and training opportunities to special needs children with the additional CFS supplied computers. Further, interviewees highlighted the improved access to computers available in libraries and NPOs (i.e. community volunteer groups), which has allowed training and workshops for seniors, at risk youth and low-income families to occur. Interviewees also indicated that the program has enhanced access to computers in small, rural, remote and/or poorer communities.

The survey revealed that CFS users believe that:

• the program is easy to access (mean of 8.8 out of 10);
• computers are available to their organization when needed (mean of 8.2);
• computers are reliable (mean of 8.1); and,
• the program has improved access to computers for their organization (mean of 8.8).

The majority of users (97%) also believed that if the CFS program was not available, it would have a negative impact on their organization. The most frequently noted negative impacts associated with the absence of the program related to no and/or reduced access to computers (i.e. 25% noted no access or reduced access to computers for students, children and others; and/or 21% indicated that the organization would have fewer computers available).

Extended computer lifespan

Interviewees noted that the program is contributing to extending the lifespan of those computers simply by refurbishing rather than recycling computers. Surveyed users indicated that they keep their CFS desktop computers for an average of approximately four years and their CFS laptop computers an average of approximately three years.

Increased donations

Donations between 2008-09 and 2010-11 are set out in Table 5. The table shows that there has been a significant increase in the number of desktop and laptop computers, as well as printers, donated to CFS recipient organizations. In fact, the increase in desktop computers and laptops represents a 67% and 134% increase respectively, from 2008-09 to 2010-11. During the same period, donated printers increased by 65%. On the other hand, there has been a slight decrease in the number of monitors provided in 2010-11 when compared to those donated in 2008-09 (12% decrease).

While the data is the most reliable evidence with regard to increased donations, interviewees indicated that donations are increasing. They noted that this was evident by the fact that warehouses are currently at full capacity. They stated that there is currently little capacity to take on more donations because:

1. recipients have limited resources to attract more donors;
2. there are space constraints; and
3. there are limitations as to what can be refurbished with the current staff, youth and volunteers available. Interviewees also noted that more efforts were being expended in increasing the quality of donations in recent years rather than just increasing the quantity of donations.

Increased awareness of the impact of reuse and refurbishment by partners

The program uses an increase or decrease in the number of partnerships as a proxy for measuring awareness of program impacts among partners. It was, however, difficult to assess this indicator as the program receives quarterly lists of partner names for each of the 14 recipients, but does not process this information, nor are changes in the total number of partners monitored over time (even though it is listed as a performance indicator). Further, many other factors likely play into decisions to partner with the program, or to end a partnership, and more robust performance measurement for this indicator is needed.

The program does monitor staged events, conferences, and activities put on by recipients, which is more of an output than an indication of achievement. Nonetheless, recipients reported a total of 51 marketing, public relations and innovative projects, initiatives or studies in the last two quarters of 2008-09; 121 in 2009-10 and 114 in 2010-11. These included, for example:

• A booth and internet café at a conference;
• Provincial pre-budget consultation meetings;
• A quarterly newsletter;
• An open house; and
• Income tax volunteer program.

Similarly, interviewees noted that some of the activities undertaken to increase awareness of the program included: attendance at teacher conferences, trade shows, business expos with city Chambers of Commerce, and the CFS stickers on the refurbished equipment. Some interviewees noted that more could be done to increase awareness on the impact of reusing and refurbishing computers by partners; however, recipients do not have the resources available to undertake these types of activities.

Practical work experience and ICT technical training and skills development for participants^{Footnote 17}

Between 2008-09^{Footnote 18} and 2010-11,^{Footnote 19} volunteers have worked 101,976 hours, TWEP interns have worked 509,807 hours, and co-op students have worked a total of 164,211 hours. This represents approximately 24 full-time equivalents (FTE) per year for volunteers, 121 for TWEP interns and 39 for co-op students.^{Footnote 20}

Interviewees noted that it was hard for youth to get employment when they graduate because they require practical experience. As such, they noted that TWEP internships and youth co-op terms provide a range of youth with the practical experience they need to find permanent employment in ICT. Many interviewees also stated that, through the day-to-day practical work undertaken in the workshops, internships provided an opportunity for youth to work towards obtaining their A+ certification^{Footnote 21}, Microsoft IT certified training, and other certifications provided through provincial programs.

Youth surveyed indicated that they had applied for a TWEP internship or co-op term because they needed the experience (54%). They went on to note that:

• the internships / co-op terms are addressing their needs very well (mean of 8.6 out of 10);
• they have gained valuable practical experience (mean of 8.6); and,
• they have received ICT technical training and skills development (mean of 7.7).

3.2.1.2 To what extent has CFS achieved its medium-term outcomes?

Increased computer awareness, knowledge and skills of youth

The integrated RMAF-RBAF for the program indicates that increased computer awareness, knowledge and skills of youth will be measured through the proportion of interns and co-op students completing their required training. This information is not available in the program data; therefore, this outcome was examined from the perspective of youth participating directly in the program (i.e. TWEP interns and co-op students),^{Footnote 22} as well as those benefiting from computers in schools, libraries and other NPOs.

From the perspective of youth participating directly in the program, interviewees noted that the practical work experience, and ICT technical training, and skills development were contributing to increased computer awareness, knowledge and skills of TWEP interns and co-op students. Interviewees noted that, by working with computers throughout their terms, youth became more aware of computers in general and obtained valuable technical training that helped enhance their computer knowledge and skills.

Additionally, youth participants surveyed agreed that:

• the internship / co-op term has increased their computer awareness, knowledge and skills (mean of 8.3 out of 10); and,
• they are more computer literate as a result of their internship / co-op term (mean of 8.1).

For youth in general, a Statistics Canada 2003 study on computer access in schools notes that access to computers in schools exposes young people to ICT that they can use to expand their knowledge^{Footnote 23}. The study goes on to state that computers give young people skills to face future challenges in the use of rapidly changing technologies in the workplace. The study therefore establishes the linkage between access and knowledge and skills.

From the perspective of the CFS program, since the program has delivered more than one million computers since inception, the majority (85%) of those to schools, it has contributed to increased computer awareness, knowledge and skills of youth. Additionally, interviewees and the users surveyed noted that the program had helped increase the ICT skills and computer literacy of youth by facilitating access to computers.

Increased program sustainability

As per Table 6, the value of partner contributions totaled about $56 million (excluding the value of donated computers) between 2008-09 and 2010-11.^{Footnote 24} Contributions (including CFS funding) for the same period totaled approximately $79 million. Overall, partners contributed 71.5% of the recipient organizations total expenses, while CFS program funding represented 28.5% of recipient funding. It is noteworthy that the proportion contributed by partners and the CFS program has been fairly constant over the three-year timeline for this evaluation.

While partners contribute significantly to recipient operations, interviewees noted that the recipient organizations could not operate without CFS program funding which is required to cover the expenses associated with refurbishing the computers. Additionally, some major partners would likely not contribute without IC / federal government support.

Recipient organizations also indicated that they could not operate without TWEP interns and co-op students who are the main source of labour in almost all the workshops. Most recipient organizations indicated that they would not be able to sustain their operations by relying solely on volunteers.^{Footnote 25}

Increased corporate social responsibility and sustainable development of partners

The CFS program indicators for this medium-term outcome have already been covered in earlier sections as they are similar to the short-term outcomes measures for:

• increased awareness of the impact of reuse and refurbishment by partners (i.e. number of staged events, new releases and articles); and
• increased donations (i.e. increase or decrease in the market value of various donations).

Therefore, no new evidence and/or data pertaining to those two indicators is available from program data for this outcome to be assessed.

Increased ICT skilled workforce and employability among participants^{Footnote 26}

There is no program or recipient data available regarding increased ICT skilled workforce or on the employability of program participants. The information reported in this section is therefore limited to the interviews and the survey of TWEP interns and co-op students.

Interviewees were generally very positive about the quality of youth and their ability to do the work from a technical perspective. Some interviewees noted that they have exit interviews with interns and also provide reference letters to assist them with their future employment. The employment of youth was noted as being especially critical in certain regions of the country (i.e. rural or northern communities) as a way to increase their employability as there are limited opportunities for youth due to the economic conditions in these regions.

The TWEP interns and co-op students surveyed strongly believed that the TWEP internship or co-op term helped them gain the skills they need for the future (mean of 8.0 out of 10) and make more informed employment-related decisions and choices in the future (mean of 8.1). In total, 49% of TWEP interns and co-op students surveyed indicated that they started work in an ICT area since their participation in the program.^{Footnote 27} Those who started work in an ICT area strongly believed that their CFS experience helped them find ICT-related employment (mean of 8.4). Interviewees also noted that many program participants find employment as a result of contacts established through the program (e.g. with schools, school boards, provincial government, etc.).

3.2.1.3 Has the program achieved progress toward its long-term outcomes?

Computer literate Canadian youth

In 2009, the Program for International Student Assessment (PISA) completed a study which highlights the performance of Organisation for Economic Co-operation and Development (OECD) countries regarding computers at school^{Footnote 28}. The report notes that:

• In 2009, on average across OECD countries for which data are available, 93% of students (in the modal grade for 15-year-olds) reported that they have access to a computer at school; Canada compares favourably with more than 98% of students having access to a computer at school (ranked sixth among OECD countries).
• In 2009 across OECD countries, the ratio of computers available for students (in the modal grade for 15-year-olds) was 0.56; again, Canada compares favourably with a computer-student ratio above 0.72 (ranked sixth among OECD countries).

From the evidence presented in the OECD report, it is not possible to assess the extent to which these favourable rankings are attributable to the CFS program. However, the following evidence (discussed in other parts of this report) shows that Canada's performance has at least partially benefited from the CFS program:

• More than 1 million computers refurbished and distributed in total, of which 923,484 were distributed to schools;
• Based on the survey of users and non-users, 77% of computers in these organizations were provided by CFS; in looking at only schools, the percentage is about the same (78%).

Interviewees noted that improved access to computers led to increased knowledge and skills which, in turn, contributed to more computer literate Canadian youth. When asked about some of the benefits of the CFS program, 6% of surveyed users noted increased computer literacy as one of the program benefits (unprompted responses).

Reduced environmental footprint

Recipient data shows that the program's performance with regards to a reduced environmental footprint has been as follows:^{Footnote 29}

• 356,246 computers (desktops and laptops combined) were donated between 2008-09 and 2010-11, and thus diverted from landfills; 40.8% of computers were recycled, the balance presumably refurbished;^{Footnote 30}
• 234,895 monitors were donated between 2008-09 and 2010-11; 44.8% of donated monitors were diverted from landfills to recycling, the balance presumably refurbished;
• 43,434 printers were donated between 2008-09 and 2010-11; 73.1% of donated printers were diverted from landfills to recycling, the balance presumably refurbished.

Studies have shown that reusing (refurbishing) computers has a positive impact on energy use, greenhouse gas reduction, solid and hazardous waste reduction, reduced air and water emissions as well as other environmental benefits and therefore reduces the environmental footprint. For example:

• The cost to produce the average desktop computer is immense from an environmental perspective. Producing one new computer and monitor takes 530 pounds of fossil fuels, 50 pounds of chemicals and 3,330 pounds of water due to the repeated rinsing necessary. These amounts are roughly the weight of a Sports Utility Vehicle or 1.8 metric tons^{Footnote 31};
• The majority (estimated to be 80%) of the life cycle energy for computers is used in the manufacturing phase. Thus, extension of the reuse cycle yields greater environmental benefits than recycling earlier in a product's life.^{Footnote 32} Ongoing energy costs associated with owning a computer are relatively low; and,
• Reselling or upgrading working computers is 5 to 20 times more energy efficient than recycling them over the computer's life cycle.^{Footnote 33}

Many interviewees also expressed the opinion that it is much better for the environment (reduced environmental footprint) and more economical to extend a computer's lifespan by using a refurbished computer for an extra few years, than to buy a new one every few years. From the interviews, it was evident that with federal and provincial e-waste policies, e-waste reduction is becoming more important.

Support for the ICT industry's potential for growth by contributing to Canada's skilled labour capacity

The performance indicator for this outcome is similar to the medium-term indicator: increased employability among participants, therefore the same difficulty arose since there is no program or recipient data available to demonstrate the program's contribution to support the ICT industry's potential for growth. There is limited new evidence that can be presented in this section as it is difficult to measure the attribution factor of the program to this outcome.

There is limited evidence from the findings presented in previous sections that show that the CFS program assists in providing benefits to keep Canada competitive by promoting essential ICT skills. Interviewees noted that progress has been made towards contributing to Canada's skilled labour capacity (see intermediate outcome "Increased ICT skilled workforce and employability among participants"). In addition, the TWEP-only survey suggests this also as 51% of the interns surveyed indicated that they started work in an ICT area since their internships.

Ultimately, the program's outcome is to strengthen public national ICT infrastructure that provides equity of opportunity for Canadians to acquire the ICT knowledge and skills they need to participate in the knowledge-based economy through the reuse of computer assets. The performance indicator for this is the number of CFS refurbished computers as a proportion of total computers in public schools, libraries, and recipient learning NPOs.

The data for this indicator is not being collected by the program (and it is also not a good indicator for this outcome), thus making it difficult to assess. Nonetheless, the evidence presented in the performance section highlight the program's contribution to this ultimate outcome. All categories of interviewees stressed that CFS creates a level playing field in terms of providing computer access to all youth across the country regardless of their location and socio-economic background. The increased access to computers from CFS enables more Canadians to develop the knowledge and skills to participate in an ever growing knowledge-based economy.

As well, survey information introduced in previous parts of the report (i.e. 77% of all computers in user and non-user organizations surveyed were provided by CFS) has been provided to the extent possible to address the indicator.

Overall, it was difficult to assess the long-term outcomes and ultimate outcome due to the fact that performance indicators were either not collected or not appropriate to evaluate that outcome. As well, the attribution effect of the impact of a program decreases as you move from short-term and medium-term outcomes to assessing long-term outcomes which further hampers the ability to assess longer-term program outcomes. Additionally, much of the data collected by the program is kept in a combination of paper files and individual Excel workbooks (168 Excel files were individually reviewed for this assessment) rather than in an electronic database, requiring a file review exercise in order to access aggregate performance information.

3.2.2 Were there any unintended impacts or outcomes that can be attributed to the program?

Due to its positive reputation and success, CFS has hosted tours for various international ambassadors at the National Technology Center (Gatineau workshop) to share the CFS model. Interviewees also noted the adoption of the CFS model for comparable initiatives internationally, including Kenya, Cameroon and Nigeria. For example, in the past nine years, Kenya has deployed 40,000 refurbished computers to over 2,000 schools, trained 9,000 educators, thereby providing ICT to 800,000 students who otherwise would not have had access. CFS Kenya also operates the region's only e-waste management centre.

Program recipients in Ontario and Saskatchewan mentioned the benefits associated with their work to adopt provincial environmental standards. Through these standards, CFS recipients go through an environmental audit to be certified as an approved collector and service provider for reuse and refurbishing operations. In Ontario, it is the Waste Electrical and Electronic Equipment Reuse and Refurbishment (WEEERR) Standard which defines the minimum requirements for reuse and refurbishment operations approved under the Ontario Environment Stewardship program. Recognized as a leader in this area, Ontario's recipient organization played a lead role in developing the provincial standard. Similarly, in Saskatchewan, CFS is working towards getting approval from the Saskatchewan Waste Electronic Equipment Program (SWEEP) to become a certified reuse depot for materials and computers.

Based on the interviews and survey results, some of the impacts of the CFS program which are not identified in the program's logic model relate to the social impacts of the program on youth and others affected by the program. This includes improved personal development through the TWEP internships and co-op terms such as team work and leadership skills, communication, interpersonal and organizational skills, time management skills, and social networking.

3.2.3 Efficiency and Economy

3.2.3.1 Are the most appropriate and efficient means being used to achieve objectives, relative to alternative delivery approaches?

Interviewees could not identify a more appropriate and efficient means of achieving objectives. While some could identify alternative federal, provincial or private sector programs that had similar elements to CFS, interviewees noted that other programs do not serve the same target clients (e.g. schools, libraries and other not-for-profit organizations, and youth), cannot offer the same level of service (e.g. number of computers, price, etc.) or are not national in scope. Consequently, interviewees believed that other programs could not achieve the same results for the same cost as the CFS program.

In terms of alternative delivery approaches, again interviewees had few suggestions. Table 7 identifies possibilities with an assessment of why each alternative approach was deemed less appropriate and efficient than CFS.

3.2.3.2 What has the utilization of resources (expended) been in relation to:
a) the production of outputs and
b) progress toward expected outcomes?

In this section, the utilization of resources expended in relation to the production of outputs (i.e. internal use of resources) and progress toward expected outcomes (i.e. external effect of program resources) is examined.

a) Actual versus Planned Efficiency

When CFS was renewed, the program was allocated funding to support an internal operating capacity that would be sufficient to meet the expected demand for CFS and TWEP grants and contributions. Overall, the annual operating budget was expected to represent 17.2% of total program funding from 2008-09 to 2012-13.^{Footnote 35}

The evaluation used this 17.2% figure to assess the efficiency of the program. It should be noted that this measure for operational efficiency is an indicative, rather than conclusive, measure of the desired efficiency. Further, the figure is based on operating costs allocated to CFS and TWEP, rather than an assessment of the operational efficiency of the individual components.

In order for the program to demonstrate efficiency, the use of resources would be broadly in line with the anticipated design. If actual operating expenses were greater than 17.2% of total program funding, then CFS would be operating less efficiently than anticipated. If actual operating expenses were lower than 17.2% of total program funding, then CFS would be more efficient than anticipated.

Table 8 presents operating costs as a percentage of total program funding after the initial year of operations. Operating costs included salary expenses and operating and maintenance expenses. The total costs of the program included these expenses plus CFS and TWEP grants and contributions.

In all three fiscal years, the operating efficiency percentages were lower than 17.2%, which indicates that the program operated at a higher level of efficiency than anticipated. This can be attributed to the fact that operating expenses were lower than expected. Total operating expenses were approximately $1 million annually compared to the expected level of $1.5 million.

b) Progress toward expected outcomes

With respect to cost per outcome, the evaluation examined the program's efficiency in terms of:

• Cost per refurbished computer (associated with outcomes pertaining to the distribution of computers), and
• Leveraged funds (associated with outcomes pertaining to partnerships).

Cost per refurbished computer

Between 2008-09 and 2010-11, 195,298 computers were refurbished for a total of $25,563,198^{Footnote 36}. This represents a cost of $130.89 per refurbished computer to the Canadian government^{Footnote 37}. While the program does not have a set target to assess this value, the average cost to purchase computers from private sector organizations is $100 for a basic computer or close to $500 for a used duo core computer with features that are comparable to those offered by CFS recipients.^{Footnote 38} Therefore, the CFS cost of $130.89 to produce one refurbished computer appears reasonable based on what private sector organizations charge for comparable computers.

Interviewees noted that they did not believe the computers could be refurbished at a lower cost particularly in light of the in-kind contributions that bolster their efforts (e.g., volunteers, software, space, etc.). As previously mentioned in Table 7, a US study compared the savings from reuse and recycling to disposing of the equipment in a landfill. The study estimated that the replacement costs saved for refurbishing a computer is $668.01 per computer^{Footnote 39}. On the other hand, the same study estimated the costs saved to recycle a computer to be $27.08 per computer. Based on these estimates, the table below shows the net environmental benefit of refurbishing the 195,298 CFS computers.

Leveraged funds

Leveraged funds consist of resources gained by the program through sources other than IC, including both cash and in-kind contributions, such as software, transportation or space. It is notable that these donations come from all levels of government as well as the private sector and that, while the program does not set specific targets for leveraged funds, CFS leveraging is well above the matching funds requirements.

Table 10 outlines the funds leveraged through the program by recipient organizations as well as through national partnerships. The program leveraged $2.13 in 2008-09, $2.37 in 2009-10 and $2.10 in 2010-11 in cash and in-kind contributions. In looking at fifteen other federal programs reviewed recently by the evaluation team or available on-line, this compares favourably as the leverage in these fifteen other programs reviewed ranged from $0.31 in leveraged funds to $4.20, with only one above $3.00^{Footnote 40}.

4.0 Conclusions and Recommendations

4.1 Relevance

• The consistent donation, refurbishing and uptake of CFS computers over time demonstrate the continued relevance and need for the program. Apart from equipping educational institutions with computers, the program provides an environmentally responsible means to dispose of surplus computers (i.e. reuse vs. recycling).
• The program plays a role in fulfilling federal roles and responsibilities with respect to the disposal of surplus crown assets and contributes toward the objectives of the Federal Sustainability Act.
• The CFS program does not overlap with any other government program.
• The program supports the federal commitment to sustainable management practices and initiatives, and is consistent with government-wide priorities related to skills development and IC's strategic outcomes.
• IC is best positioned to continue with the administration of the CFS program.

4.2 Performance

• Overall, the evaluation found that the program is successful. Its success is rooted in the reciprocal benefits it offers to governments, the private sector and the general public.
• This assessment found that the program has achieved its core expected outcomes. However, the logic model needs to be reworked to better reflect the inherent value of program outcomes, eliminate duplication and ensure that all outcomes are relevant.
• The program has yielded unanticipated benefits. Specifically, CFS has been used as a model to launch similar initiatives in other countries, recipients indicated that they are moving toward adopting provincial environmental standards, and youth are gaining work-related social skills.
• There does not appear to be a more appropriate and efficient means to achieve objectives, relative to alternative delivery approaches. Alternatives would be less efficient, serve different clients, offer a lower level of service, or be less equitable for different provinces and territories.
• CFS operated at a greater level of efficiency than what had been anticipated at the time of program renewal.
• From a cost-benefit perspective, the environmental benefits from eliminating landfill disposals were significantly greater than refurbishment costs. In addition, for every CFS dollar, the program leverages at least an additional two dollars each year from other sources.

4.3 Recommendation

The findings of the evaluation led to the following recommendation:

An updated performance measurement system for the program is required. It should include:

1. Reworking the logic model and identifying appropriate and measurable program outcomes to evaluate the success of the program;
2. Revising the performance indicators (and including attainable targets where applicable) that can be collected to assess performance; and,
3. Developing a common database for performance information to improve reporting capacity.

Management Response and Action Plan