Final Evaluation for the Northern Ontario Development Program

Final Report

February 2011

Tabled and approved at the Departmental Evaluation Committee on February 23, 2011

Table of Contents

• Executive summary
• 1.0 Introduction
  • 1.1 Context
  • 1.2 Program Profile
    • 1.2.1 Program description
    • 1.2.2 Program resources
    • 1.2.3 Program expected results
    • 1.2.4 Program reporting
  • 1.3 Report Outline
• 2.0 Methodology/approach
  • 2.1 Evaluation issues
  • 2.2 Evaluation methodology
    • 2.2.1 Document and literature review
    • 2.2.2 Administrative data and file review
    • 2.2.3 Interviews
    • 2.2.4 Case studies
  • 2.3 Methodology limitations
• 3.0 Findings
  • 3.1 Relevance
  • 3.2 Performance
• 4.0 Conclusions and Recommendations
  • 4.1 Relevance
  • 4.2 Performance
• Management Response

Annexes (separate document)

(Note: Annexes are available via an Access to Information Request)

• Annex A Evaluation Advisory Committee members
• Annex B Evaluation matrix
• Annex C File review projects
• Annex D File review template
• Annex E Interview introductory letters and interview guides
• Annex F Case study proponent introductory letter and interview guides
• Annex G Case studies
• Annex H NODP indicators—expected and actual results
• Annex I Examples of non-profit organizations NODP funded projects
• Annex J Concepts from the literature
• Annex K Northern Ontario economic development programs

Program Overview

The Northern Ontario Development Program (NODP) supports economic development and diversification in northern Ontario through contributions to a range of organizations both public and private. The NODP supports projects in six program areas: Community Economic Development; Innovation; Information and Communications Technology; Trade and Tourism; Human Capital; and Business Financing Support.

The NODP is delivered by the Federal Economic Development Initiative for Northern Ontario (FedNor), which serves as the main representative for Industry Canada (IC) in northern Ontario.

Evaluation Approach

The purpose of the evaluation was to assess the relevance and performance of the program as required under the Treasury Board Policy on Evaluation. The evaluation covers the period from April 2006 to August 2010. Projects included in the evaluation were those approved between April 2006 and January 2010, representing approximately $148 million to support 858 projects.

The evaluation findings and conclusions are based on the analysis and triangulation of multiple lines of evidence. The methodology included a review of documents and literature, an administrative data and file review, interviews, and case studies.

Findings

Relevance

There is a continued need for the NODP to strengthen and diversify the northern Ontario economy, which has historically relied on primary resources. The isolation of certain communities in northern Ontario, the distance between these communities, and their reliance on one industry make them vulnerable to outside shocks, such as price fluctuations and variations in commodity exports. The NODP aims to foster economic growth and diversify the regional economy by supporting projects in the six thematic areas.

Its scope has been broad, supporting a wide range of projects. However, some interviewees believe the NODP could better address the needs of the region by investing more directly in businesses and targeting its resources to fewer, larger, and strategically positioned projects located in communities that have good links to outside markets. Much of the recent academic literature aligns with this advice.

The NODP is consistent with federal roles and responsibilities and is an important federal presence in northern Ontario. The program promotes economic growth, diversification, job creation, and sustainable communities in northern Ontario, and is the government's response to regional economic development in that area. Interviewees noted that the federal government has a responsibility to lead the way for all orders of government, non-governmental organizations, and other institutions. Recent academic literature supports interviewees' views.

The NODP also supports the federal government's priorities, as outlined in recent Speeches from the Throne and Federal Budgets, and IC's strategic objectives. Although other economic development programs are available in northern Ontario, evidence shows that they are complementary and result in very little overlap and duplication. Many interviewees and case study participants noted that the collaboration of various programs available in northern Ontario is essential for any project to move forward.

Performance

The Mid-term Evaluation of the NODP (2008) identified ways to strengthen the program's Performance Measurement System and data quality. Program management has made some improvements to the program in response to these issues. However, limited progress has been made on the collection of performance measurement information regarding outcomes beyond the project end date. Changes to address the longer-term measurement of project outcomes are being planned and integrated into the revised NODP Performance Measurement System.

The main factor that supports effective delivery of the NODP is the involvement and experience of FedNor regional staff. Regional FedNor offices in northern Ontario enable direct contact with the proponents, and FedNor staff are knowledgeable of the communities' needs. However, the length of time for project approval has impeded the delivery of the program. These approval times are longer compared to both previous FedNor-administered programs and other regional development agencies in Canada.

The NODP has achieved its immediate outcomes. The Final Project Evaluation indicator database captures several indicators under each program area to measure the achievement of the immediate outcomes of the program. These indicators, as well as interviews and case studies, provide direct evidence of the achievement of immediate outcomes.

However, evidence on the achievement of intermediate and ultimate outcomes is mixed. Qualitative evidence from interviewees and case study participants indicates that the program is achieving these outcomes. The lack of continued tracking and measurement of outcomes by the project proponent after the formal relationship with NODP has ceased limits the evidence that could speak to the achievement of long-term outcomes.

Projects funded under the NODP have led to several outcomes and have contributed to the economic development of northern Ontario. According to interviewees, without funding from the program, many projects would not have proceeded or would have proceeded but at a reduced scope.

Administrative costs appear to be reasonable to support program delivery over a vast geographic area, with requisite support for effective communications and core services. Interviewees stated that the experience and knowledge of FedNor staff in the regions and their high level of availability and support are instrumental to the development of projects and success of the program.

It is difficult to assess whether the cost per outcome is reasonable due to data limitations. From a sample of projects, the total cost to the NODP per temporary and permanent job created and maintained was $5,229. However, creating and maintaining jobs were not the only outcomes expected from these projects.

The NODP has been quite successful at securing financial contributions for projects from other sources. For each dollar contributed to a project by the NODP, $2.77 was contributed to the project from other sources, such as the provincial government, municipalities, the private sector, and the proponents.

Although some interviewees noted that the NODP allocates its resources effectively, they also suggested the program could be more effective if it were refocused. In particular, they felt funds should be allocated more strategically to core economic development activities and should be targeted to projects that impact businesses. These suggestions are consistent with the current direction of the literature.

Recommendations

Recommendation 1: To ensure that expected outcomes are being achieved, the program should improve the measurement of medium- and longer-term outcomes.

Recommendation 2: To enhance program delivery success, the program should consider improving the project approval process to decrease approval times.

Recommendation 3: To better achieve the overall program objective of economic development and diversification in northern Ontario, FedNor should be more strategic in its allocation of program resources and target program funding to core economic development activities. Consideration should also be given to how to increase program impacts on for-profit businesses.

1.0 Introduction

The Northern Ontario Development Program supports economic development and diversification in northern Ontario through contributions to a range of organizations both public and private. NODP staff also coordinate and facilitate consultations among governments, non-governmental organizations, and industry for the purpose of identifying and acting on economic development opportunities.

The Federal Economic Development Initiative for Northern Ontario (FedNor) administers the program. FedNor is Industry Canada's regional development organization in northern Ontario.

The purpose of the evaluation is to assess the relevance and performance of the program under the Treasury Board of Canada Policy on Evaluation. The evaluation covered NODP activities between April 1, 2006 and August 18, 2010.

1.1 Context

Northern Ontario covers over 806,000 square kilometres (almost 90% of the province's total land area) and has a population of 843,853, which accounts for about 7% of the total provincial population.¹ The region consists of 9 cities, 26 towns, 93 townships, 18 municipalities, 5 villages, 129 Indian reserves and settlements, and 17 unorganized areas. Five major urban centres—Timmins, Thunder Bay, Sault Ste. Marie, Sudbury, and North Bay—together account for over half (52%) of the population of northern Ontario.²

Historically, the economy of northern Ontario was concentrated in resource-based industries, including agriculture, forestry, and mining.³ These goods-producing activities remain an important source of economic activity. Almost one-quarter of the jobs in northern Ontario are in goods-producing sectors (22%) such as construction (7%), manufacturing and mining (7%), and fishing and forestry (5%). The remaining 78% of jobs are in the service-producing sectors, such as health care and social assistance (16%), wholesale and retail trade (15%), and education (9%).⁴

Gross Domestic Product (GDP) by sector in northern Ontario has a distribution similar to the employment figures. Goods-producing industries contribute 27% to the northern Ontario GDP, with manufacturing (10%) and construction (7%) contributing the largest amounts. The remaining 73% of the GDP is contributed by services-producing industries, with non-commercial services (23%) and wholesale and retail trade (14%) providing the largest amounts.⁵ Relative to the province as a whole, the northern economy is much more oriented toward forestry, fishing, mining, oil and gas, and the public sector, and less oriented toward manufacturing, finance, insurance, real estate and leasing, and professional, scientific, and technical services.⁶

The mining and forestry industries in northern Ontario have changed significantly in recent years. In the mining sector, automation has reduced available employment opportunities, whereas rising commodity prices have tended to have the opposite effect.⁷ In forestry, cost pressures and turmoil in the US housing market (an important consumer of the region's forestry products) have adversely affected employment in that sector. This has, in turn, seriously affected manufacturing in the region, much of which is tied to the pulp and paper and sawmill industries.⁸

Southcott (2007) notes that because of the way northern Ontario developed, the region depends on natural resource exploitation and externally induced incomes in the form of public administration. He argues that the implications associated with this pattern of development are economic vulnerability to factors outside the region's control, such as world commodity prices, and low levels of entrepreneurship, which has served as a barrier to "the cultivation of an entrepreneurial culture."⁹, ¹⁰

1.2 Program Profile

1.2.1 Program description

The Federal Economic Development Initiative for Northern Ontario

Established in 1987, FedNor serves as IC's main representative in northern Ontario.¹¹ It "is a regional development organization […] that promotes economic development, diversification and job creation and encourages sustainable, self reliant communities in" the region.¹² FedNor delivers two core programs: the NODP and the Community Futures Program. Other programs delivered by FedNor include the Community Adjustment Fund and the Economic Development Initiative.¹³

FedNor has three main offices in northern Ontario located in Sudbury, Sault Ste. Marie, and Thunder Bay. It also has four satellite offices in strategic locations—Kenora, North Bay, Parry Sound, and Timmins—to serve proponents; officers are located in areas where they can stay in close contact with the communities.

Over 80% of FedNor's resources are dedicated to "supporting a decentralized field network to deliver its programs."¹⁴ This provides communities with access to FedNor officers who have established networks of local contacts and working relationships, and can guide strategic project proposals.¹⁵

The Northern Ontario Development Program

The NODP provides support in the form of financial contributions to projects in northern Ontario in six program areas:

• Community Economic Development (CED) – Northern Ontario communities assisted to develop their capacity to undertake strategic planning in response to decreased industry levels and to implement related CED projects.
• Innovation – Knowledge commercialized; investment in innovation, productivity improvement and value-added product development in key sectors such as biotechnology, mining and forestry.
• Information and Communications Technology (ICT) – High-speed Internet provided to communities; support for ICT applications, particularly in the areas of health, education and business; access to advanced communications services through central telecommunications hubs referred to as 'Points-of-Presence.'
• Trade and Tourism – Trade missions sponsored and trade networks assisted to help businesses access U.S., European and interprovincial markets. Northern Ontario's tourism industry (including nature-based tourism) effectively promoted.
• Human Capital – Youth internships, entrepreneurships and business skills promoted, and more skills development opportunities made available through Northern Ontario educational institutions.
• Business Financing Support – Enhanced small- and medium-sized enterprise (SME) access to capital, including patient and venture capital, with more capital providers and leveraged capital.¹⁶

The contributions in these six areas address the NODP's mission to "promote growth, economic diversification, job creation, and sustainable, self-reliant communities in Northern Ontario, through a range of initiatives aimed at improving business access to capital, information, and markets."¹⁷

NODP assistance is limited to SMEs with fewer than 500 employees. Key proponents include:

• SMEs that are unlikely to obtain financing through a loan/investment fund because the risk is too high;
• Non-profit organizations engaged in research and development;
• Aboriginal capital corporations, Aboriginal non-profit organizations, Aboriginal-controlled financial organizations, and similar groups able to increase the economic activity of Aboriginal people;
• First Nations and legal entities including incorporated not-for-profit organizations such as municipalities and municipal organizations; community development organizations; post-secondary institutions; hospitals and regional health care centres and other not-for-profit consortia of partnerships; sole proprietorships; corporations; or associations;
• Businesses including corporations, partnerships, co-operatives, proprietorships, trusts, or consortia; and
• Community Futures organizations.

Table 1 shows the number and dollar value of projects supported under each thematic area of the NODP between April 1, 2006 and January 31, 2010.

In total, the NODP approved approximately $148 million to support 858 projects between April 1, 2006 and January 31, 2010. About half of the funding supported CED and Innovation projects. The thematic area that comprised the most projects was Human Capital; however, most of these projects (271) were youth internships, which are generally low-budget projects with contributions below $30,000. Innovation had the highest funding per project.

1.2.2 Program resources

Table 2 sets out the funding profile of the NODP, split between Grants and Contributions (Gs&Cs) and Operating and Maintenance (O&M) expenses.

Actual spending for the program appears in Table 3.

Total expenditures were $21 million less than expected between 2005–06 and 2009–10 as a result of government-wide reduction initiatives over the past few years. This reduction in available funding would have been greater had FedNor not received additional one-time funding from the fiscal framework for two specific projects as noted in Table 2.

1.2.3 Program expected results

The NODP has the following expected immediate, intermediate, and ultimate outcomes. The six immediate outcomes align with each of the six program areas (noted in brackets) that, in turn, support a single intermediate and single long-term outcome:

Immediate outcomes

• Improved community capacity for socio-economic development (CED)
• Enhanced markets and investment attraction (Trade and Tourism)
• Enhanced innovation capacity and improved infrastructure (Innovation)
• Enhanced information and communications technology applications and infrastructure (ICT)
• Improved access to capital and leveraged capital (Business Financing Support)
• Attraction, retention, and development of human capital (Human Capital)

Intermediate outcome

• Improved competitiveness and/or sustainability of communities and key sectors

Ultimate outcome

• Long-term, sustainable socio-economic development in northern Ontario.

Figure 1 presents the NODP logic model and demonstrates how the activities, outputs, and outcomes of the NODP align.¹⁸

Figure 1¹⁹: NODP Logic Model

[Description of Figure 1]

1.2.4 Program reporting

Throughout the life of a project, proponents are responsible for providing progress reports (Project Activity Reports) with a financial claim on a monthly or quarterly basis. In addition, a Final Project Evaluation (FPE) report must be completed at the end of the project.

Annually, FedNor is required to report on the NODP for IC's Departmental Performance Report (DPR) and Report on Plans and Priorities (RPP). FedNor also provides information on the NODP for its annual business plan.²⁰

1.3 Report Outline

The final evaluation report is broken down as follows:

• Section 2 provides a list of evaluation issues and describes the methodologies used to gather information for this evaluation.
• Section 3 is divided into two subsections: findings on relevance, and performance. It also provides findings of the evaluation by evaluation issue.
• Section 4 provides conclusions and recommendations.

2.0 Methodology/Approach

IC engaged Prairie Research Associates (PRA), an independent research company, to conduct a Final Evaluation for the NODP. The purpose of the evaluation was to assess the relevance and performance of the program as required under the Treasury Board Policy on Evaluation.

An Evaluation Advisory Committee (EAC) was established to provide advice and guidance during the evaluation. The committee reviewed and provided input on the methodology, as well as on the findings, conclusions, and recommendations presented in the evaluation report. A list of NODP EAC members is provided in Annex A.

This section lists the issues addressed by the report and describes the methodologies used to gather and analyze the information for this evaluation.

2.1 Evaluation issues

This evaluation meets the requirements of the Treasury Board Policy on Evaluation and addresses the themes of relevance and performance. The five core issues of the Treasury Board Policy on Evaluation and the specific questions for the evaluation included:

Relevance

• Continued need for the program
  To what extent does the program address a demonstrable need and respond to the needs of Canadians?
• Alignment with government priorities
  To what extent are NODP objectives linked to (i) federal government priorities and (ii) departmental strategic outcomes?
• Alignment with federal roles and responsibilities
  To what extent are the roles and responsibilities of the federal government being met in delivering the program?

Performance

• Achievement of expected outcomes

  To what extent does program management regularly identify and act on required improvements in delivering the program?

  What factors have facilitated or impeded the delivery of the program?

  To what extent has the program achieved progress toward immediate outcomes, intermediate outcomes, and ultimate outcomes in terms of performance targets, program reach, and program design?

  What are the linkages and contributions of outputs to outcomes?

• Demonstration of efficiency and economy

  What has been the utilization of resources in relation to the production of outputs and progress toward expected outcomes?

  To what extent is the NODP cost-effective?

The complete evaluation matrix appears in Annex B.

2.2 Evaluation methodology

The study methodology specifically ensured that multiple lines of evidence were available for all issues and questions. The methodology included a review of documents and literature, an administrative data and file review, interviews, and case studies. In addition, rather than duplicating the efforts of previous studies, the study team used pre-existing research such as case studies and literature reviews, where feasible, to minimize the burden on individuals consulted.

The evaluation findings and conclusions are based on the analysis and triangulation of multiple lines of evidence (see also Annex B, evaluation matrix). This subsection describes these various methods.

2.2.1 Document and literature review

The document review included materials provided by the program, including past evaluations to minimize duplication of work. The literature review incorporated and extended work undertaken as part of the Northern Ontario Development Fund (NODF)²¹ summative evaluation. This review focused on rural economic development and included recent studies conducted by the Organization for Economic Co-operation and Development (OECD).

It is important to note the unique situation in northern Ontario and to use the literature as indicative of current trends in regional/community economic development rather than as a policy template.

2.2.2 Administrative data and file review

The administrative data review included excerpts from the FPE/indicator database and the Grants and Contributions Reporting System (GCRS) provided electronically by the program.

The file review included the selection of 40 projects to be analyzed in more detail using Access software. These projects comprised the 20 largest projects in terms of funding plus a random selection of 20 additional projects. Some of the 20 randomly selected projects were replaced with other projects also selected randomly to ensure representation of all program areas as well as representation of the two main regions of northern Ontario (northeast and northwest).

Annex C presents the list of projects included in the file review. See Annex D for a list of the fields that were captured in the Access database.

For each project selected, the program forwarded the application, contribution agreement, final report, and progress reports. These files contained all the information necessary to complete the fields in the Access database.

2.2.3 Interviews

A total of 42 interviews were completed with the following groups:

• External stakeholders (29)
• Program managers (6)
• Project officers (5)
• Academics (2).

Interviewees received an invitation by email from IC/FedNor in both official languages to inform them that PRA would contact them to participate in an interview. A copy of the letter in both official languages appears in Annex E.

Each interview group received a tailored interview guide (in both official languages) prior to the interview, to support preparation of responses. A pretest with two interviewees in each group was conducted to ensure clarity and comprehension. Only minor changes were necessary. Copies of the finalized interview guides in both official languages also appear in Annex E.

The interviews were conducted by phone, in the official language of the participant's choice. Interviewees received assurance of confidentiality and anonymity prior to the interview. With their permission, interviews were audio-recorded and, to ensure the accuracy of responses, interviewees received notes of their responses.

The findings of the interviews use the following scale:

[Description of Figure 2]

It is important to note that, in analyzing the findings from the interviews, two situations can occur.

First, there may be agreement across all groups of interviewees, but a single respondent may offer a perspective, based on experience or a unique position, that materially clarifies an aspect of the program or context. A single interviewee may raise a unique point that helps clarify some other aspect of information.

Second, since not all groups of interviewees were asked to respond to each evaluation question, and since different groups may have diverging opinions, findings are linked to the interview group that provided the information. Where these situations arise, they are noted in the findings. Of course, a single view needs corroboration from other information to ensure its validity.

2.2.4 Case studies

The evaluation completed eight case studies. Three were selected from the prior summative evaluation of the NODF (2006) to support a longer follow-up and update. Five were selected (in consultation with FedNor) based on:

• Size of budget
• Strategic goal
• Region
• NODP supported area
• Special populations (e.g., Official Languages Minority Communities, Aboriginals)
• Type of recipient organization (e.g., SME, not-for-profit, municipal).

The projects listed in Table 4 served as the case studies for this evaluation.

Case study data comprised an interview with the project proponent and a review of the NODP project file. In some cases, the project proponents provided additional documentation and information not available in the FedNor files.

Project proponents received an invitation letter from IC/FedNor by email saying PRA wanted to interview them concerning their NODP-supported project. Copies of the letter and the interview guide in both official languages appear in Annex F.

Phone interviews with project proponents were in the official language of the proponent's choice. The information collected from the interviews was combined with the information in the project files to develop case studies of about five pages. Case study interviewees received a draft of the case study for their review to ensure accuracy. Annex G presents the finalized case studies.

2.3 Methodology limitations

The following are the three main limitations of this evaluation.

Outcomes are not independently verified – Information regarding immediate outcomes came from administrative and self-reported data. These data appear in an indicator database. Although the indicator database provides an opportunity to measure the success of the program at achieving its immediate outcomes, the data have not been independently verified and rest on the representations of the project proponent in the FPE. Project officers, who flag issues for clarification, review all these reports. FedNor has now started to conduct edit checks using an automated system to ensure data quality. Any issues identified during the checks are referred to an officer for clarification.

Measured outcomes are short-term – As per the NODP Results-Based Management and Accountability Framework, the program collects quantitative data on short-term outcomes at the end of every project and relies on qualitative data collected at the time of program evaluation to measure intermediate and long-term outcomes. The program does not followup on projects to collect quantitative information on the achievement of intermediate and ultimate outcomes.

Attribution of outcomes to NODP is difficult – The majority of projects supported by the NODP are jointly funded. This makes it difficult to attribute the success of outcomes achieved specifically to the NODP. That said, all evaluations of cost-shared programs would face this issue.

3.0 Findings

This section provides the evaluation findings by issue, using information generated by the methods described in Section 2.0.

3.1 Relevance

Issue 1: Continued need for the program
To what extent does the program address a demonstrable need and respond to the needs of Canadians?

Interviewees agreed that northern Ontario's vulnerability to price fluctuations and variations in exports will continue to lead to out-migration and an overall shrinking economy. The isolation of certain communities, the distance between these communities, and their reliance on one industry make them extremely vulnerable to these outside shocks. Interviewees noted that an effort to diversify the economy and create jobs and businesses is essential to building a sustainable economy in northern Ontario.

The NODP aims to foster economic growth and development and diversify the regional economy by supporting projects in six thematic areas. Table 5 shows the NODP support provided to projects by thematic area and provides examples of the types of projects funded under each area.

As the table demonstrates, the NODP supports several different areas and project types, such as investments in infrastructure, public services, ICT, Innovation, and the development of local amenities to support tourism and attract high-skill industries. The expectation is that such diverse support will "accelerate Northern Ontario's movement to a knowledge-based economy and build more globally competitive businesses"²² a nd mitigate the effects of the external variations in demand for their primary resources.

In addition, based on the output and outcome indicators in the database, it appears that many of the NODP projects within the thematic areas create and maintain immediate jobs, and about half lead to immediate business support.²³ All of the NODP immediate outcomes for each NODP thematic area appear in Annex H.

Table 6 shows that the NODP has invested the majority of its funding in non-profit organizations (57%) and municipalities (25%), whereas only 3% of the funding has gone to for-profit businesses. Some interviewees noted that the NODP could better address the needs of Canadians by investing more directly in for-profit businesses.

However, a closer examination of funding for non-profit organization projects shows that much of the spending classified as "non-profit" really comprises investments in infrastructure such as broadband, health care facilities, and other public capital (see Annex I for more detail). Much of these investments can arguably be linked to support for economic development. Increased broadband connectivity, for example, is often viewed as a critical infrastructure investment that supports the knowledge economy.

In addition, some interviewees suggested the program could better address diversification goals and regional economic performance by targeting investments in fewer, larger, and strategically positioned projects. Much of the recent academic literature aligns with this advice. Some key concepts in the literature included:

• Place-based policy – "Focus on policy issues as they play out in concrete geographic and community settings, as a means to address challenges and opportunities where the impacts are directly felt. In the context of regional economic development, it means drawing on the assets and opportunities of a 'place' to develop policies/strategies that support economic development."²⁴
• Comparative versus competitive advantage – Invest based on competitive advantage (e.g., firms and communities that will be successful in market exports outside the region) instead of comparative advantage (e.g., firms and communities that specialize in standard northern Ontario resources).
• Strategic investment – Focus investments on high potential areas and businesses that will stimulate economic development by building on local strengths and opportunities.²⁵

Many interviewees and case study participants noted that the various programs available in northern Ontario for regional economic development complement each other and that collaboration is essential for any project to move forward. Project data show that the provincial government is often a joint funder on NODP-supported projects. Such joint federal-provincial funding also mitigates the risk to any single funder.

Of the 40 projects included in the file review, the most common funding partner is the Province of Ontario, especially the Northern Ontario Heritage Fund Corporation (NOHFC).²⁶ Just under half (18 projects) were jointly funded by the province, including about one-third (12 projects) through the NOHFC.

Recent consultations with NOHFC management reached the same conclusion, that there is no overlap and duplication between the economic development programs available in northern Ontario. They noted that stacking funds for projects was permissible under the NOHFC, and that complementary programs were necessary to move projects forward in northern Ontario.

Annex K lists some examples of available programs to support economic development projects in northern Ontario.

Issue 2: Alignment with government priorities
To what extent are NODP objectives linked to (i) federal government priorities and (ii) departmental strategic outcomes?

Recent Speeches from the Throne and federal Budgets provide evidence that the NODP has supported and continues to support federal government priorities. From 2006 to 2010, Speeches from the Throne have discussed the importance of maintaining and growing Canada's economy. The 2010 Speech identified the importance of creating new jobs and supporting rural industries.²⁷

Federal Budgets support the Speeches from the Throne by earmarking funding for economic development and job creation. The 2010 Budget Address included a focus on regional economic development titled Leading the Way on Jobs and Growth. It also highlighted the important role that Regional Development Agencies (RDAs) play "by promoting innovation and the commercialization of research in communities throughout Canada." ²⁸

Interviewees concurred that NODP objectives link to and support federal government priorities. Many noted that it is now more the case than ever because of the economic downturn and the importance of economic development in Canada.

Within the IC Program Activity Architecture, the NODP contributes directly to the strategic outcome: Canadian businesses and communities are competitive. Table 7 shows how the NODP advances IC's strategic outcomes.

Issue 3: Alignment with federal roles and responsibilities
To what extent are the roles and responsibilities of the federal government being met in delivering the program?

The Government of Canada is committed to promoting equal opportunities for the well-being of Canadians and furthering economic development to reduce disparity, as set out in the Constitution Act, 1982. The Department of Industry Act, particularly section 4(2), sets out how the Minister supports the Government of Canada in its commitment to equal opportunities and reducing regional disparities. Sections 8 and 9 of the Act set out the Minister's responsibility for regional economic development in Ontario. FedNor is the sole federal presence in northern Ontario supporting the Minister in fulfilling these responsibilities.

The NODP promotes economic growth, diversification, job creation, and sustainable communities in northern Ontario, and is the government's response to regional development in that area. The NODP complements the Government of Ontario's initiative known as the NOHFC, which works with northern entrepreneurs, communities, and business organizations to support vital infrastructure and economic development projects to create jobs and stimulate the economy.

There is a general understanding by both levels of government regarding the need to harmonize federal and provincial approaches to northern Ontario economic development to allow for coordination and potential collaboration in areas of mutual interest and where circumstances permit. Ongoing communication, particularly among program officers, ensures duplication does not occur and that initiatives from both levels of government build on each other's successes.

Interviewees noted that the federal government has a clear role and responsibility to deliver a program such as the NODP in northern Ontario. Although the respondents agreed there is a role for all levels of government in this area, they view the federal government as having a responsibility to lead the way for the province, municipalities, non-governmental organizations, and other institutions.

The recent academic literature confirms interviewees' views. The theme of governance figures prominently in the OECD's The New Rural Paradigm and the Canadian Policy Research Networks' (CPRN) review of rural and regional development policies and programs.

The OECD argues that The New Rural Paradigm "requires important changes in how policies are conceived and implemented to include a cross-cutting and multi-level governance approach," adding that "traditional administrative hierarchical structures" are unlikely to suffice for this purpose.²⁹ Developing a governance structure supportive of the new approach to rural policy will require "adjustments … along three key governance dimensions: horizontally at both the central and the local levels and vertically across levels of governance."³⁰

Similarly, the CPRN review finds that novel approaches to policy in the area of rural and regional development incorporate new governance models characterized by:

• "partnerships among governments, the private sector, communities, voluntary organizations and others;
• greater devolution to the regional and local level; new roles for the state as an 'enabler' and 'convenor'; and
• the sharing of power and authority with the community itself."³¹

3.2 Performance

Issue 4: Achievement of expected outcomes

Issue 4A: To what extent does program management regularly identify and act on required improvements in delivering the program?

Previous evaluations have identified the need to improve certain aspects of performance measurement. The most recent of these evaluations—the Mid-term Evaluation of the NODP (2008)—identified ways to strengthen the program's Performance Measurement System and data quality:

1. FedNor should determine the benefit and cost of collecting performance measurement information on project outcomes beyond the end date of projects. If appropriate, FedNor should extend its Performance Measurement System to capture these longer-term results.
2. FedNor should strengthen mechanisms and formalize ongoing edit checks to ensure data quality, including:
   1. consistent comparison of intended and actual outcome data for each project
   2. both program officers and proponents report on final results independent of intended outcomes captured at the project launch stage
   3. analyze minimum, maximum, and most frequent values for indicators
   4. create separate sub-indicators based on project grouping with unique outcome characteristics
   5. address concerns and anomalies with data and indicators as required.³²

In a follow-up report (January 2010) to the Mid-Term Evaluation of the NODP, FedNor developed several actions to address these recommendations. Table 8 sets out these actions, as well as an assessment of progress.

As is shown in the table, progress is being made on most fronts. For example, a separate form for intended and final outcomes is being used for projects that started later in the file review.

However, limited progress has been made on the collection of performance measurement information on project outcomes beyond the project end date. Changes to address the longer-term measurement of project outcomes are being planned and integrated into the revised NODP Performance Measurement System.

Issue 4B: What factors have facilitated or impeded the delivery of the program?

The main factor that supports effective delivery of the NODP is the involvement and experience of FedNor regional staff. Regional FedNor offices in northern Ontario enable direct contact with the proponents and FedNor staff are knowledgeable of the communities' needs. Interviewees noted that the level of FedNor support and staff's knowledge of northern Ontario facilitates the delivery of the program. The direct involvement of FedNor staff in project design and planning guides investment decisions, mitigates risks associated with projects, and enhances the likelihood of realizing NODP's outcomes.

FedNor's continued presence and experience in northern Ontario appears also to have resulted in linkages with other federal departments, provincial and local governments, and non-governmental organizations. Building such relationships can be important in supporting the delivery of the program since FedNor officers can coordinate support with outside partners to move projects forward.

The main issue impeding program delivery is the time required for project approval, according to interviewees and case study respondents. They noted that project approval times for the NODP are longer compared to previous FedNor-administered programs.

An analysis of the 858 NODP projects approved between April 1, 2006 and January 31, 2010 showed that the average approval time was about 140 working/business days. About one-quarter of the projects were approved in three months or less, and about 60% were approved in six months or less. More than one-third took more than six months for approval and more than 10% took more than a year.

RDAs comparable to FedNor, such as Canada Economic Development-Quebec and Western Economic Diversification Canada (WD), have processing time commitments in working/business days of 25–46 days and 60 days (in 90% of cases), respectively.³³ Canada Economic Development-Quebec meets its committed approval time in 50% of its cases.³⁴ Although the Atlantic Canada Opportunities Agency (ACOA) does not have a predetermined commitment time, it has measured its approval times at between 88 days and 8 months, depending on the program.³⁵

Issue 4C: To what extent has the program achieved progress toward immediate outcomes, intermediate outcomes, and ultimate outcomes in terms of performance targets, program reach, and program design?

Immediate outcomes

The FPE indicator database captures several indicators under each NODP thematic area to measure the achievement of the immediate outcomes of the program. Indicators from each project are entered into the indicator database. In total, 90 indicators are captured across the 6 thematic areas. A full list of the indicators and the results expected and achieved by the projects can be found in Annex H.

In response to the NODP Mid-term Evaluation, FedNor collapsed certain indicators into 10 indicator groups, which are common across thematic areas. The most commonly reported indicators across thematic areas are businesses supported; jobs created and maintained; studies/plans developed; events held; and number of participants at events.

Table 9 presents the results of these indicator groupings; it includes data for 858 projects ending between April 1, 2006 and March 31, 2010. The table includes three results columns:

• Count – The number of projects that contributed to the indicator
• Sum – The total number from the projects that contributed to the indicator (e.g., one project may benefit a number of businesses and organizations)
• Mean – The average number per project that contributed to the indicator (Sum/Count = Mean).

The indicators provide direct evidence for the achievement of the NODP's immediate outcomes. For example, the number of events held and number of participants at events lend themselves directly to the enhancement of markets and investment attraction under NODP's Trade and Tourism thematic area. The number of studies and plans developed directly supports improved community capacity for socio-economic development, enhanced innovation capacity and improved infrastructure, and enhanced ICT applications and infrastructure.

The case studies provided corroborating evidence that the NODP is achieving its immediate outcomes. Table 10 lists the five case studies examined in this evaluation, the expected NODP outcome, and the quantitative measures that led to the achievement of that outcome.

Table 10 shows that there is an overall benefit to businesses, particularly the creation and maintenance of jobs. Other important indicators included the development of a community plan and increased revenue into a community.

In addition, Table 10 highlights the case studies' achievement of their intended outcomes. For example, the Revitalization of Main Docks in Little Current project was expected to enhance markets and investment attraction. According to the case study information, revitalization of the docks resulted in stores opening along the docks and in the community, which has led to additional employment. The Development of a Five-Year Strategic Plan led to the development of a community plan, which improved the region's opportunities for socio-economic development.

Interviewees supported the indicator and case study evidence that the NODP is achieving its immediate outcomes. However, it is difficult to determine whether the program has been as successful as expected, as targets were only set for those indicators reported on in the RPP.

Although proponents are required to identify indicators and estimate their expected contribution to those indicators at the start of projects, these goals only provide targets for the individual projects. With respect to overall targets, some are reported on through IC's RPP, but they are limited in scope.

In particular, the RPP provides annual targets for the number of businesses created, expanded and maintained, and dollars leveraged by NODP investments. However, targets for number of jobs created and maintained and several other outcomes were not developed at all.

FedNor has indicated that although it would be difficult to set targets for several of its measured outcomes, it is refining its performance reporting for 2011–12 and will report on jobs created and maintained in the RPP and DPR.

Intermediate and ultimate outcomes

Most interviewees and all case study proponents agreed that the NODP has achieved progress toward its intermediate and ultimate outcomes. Case study proponents provided direct examples of how their projects are achieving these outcomes. Interviewees mentioned the following indicators to show that the program is progressing toward these outcomes:

• The development and sustainment of jobs and businesses
  • development of long-term employment for youth interns
  • increase in specialized research positions
• Support provided to various industry initiatives
• Innovation and infrastructure work that has contributed to the health sector and the economy.

Intermediate and ultimate outcomes can be problematic to measure. Currently, little data on intermediate and ultimate outcomes are collected and there are few resources allocated to performance measurement. The indicator database provides several "indications" of immediate outcome achievement, but does not capture any quantitative measures in the longer term of jobs continuing to be maintained, increased sales, or other measures of business expansion.

In an effort to measure longer-term outcomes, the evaluation conducted three case studies of projects that had been studied by the preceding NODF. Table 11 provides the longer-term outcomes retrieved from the websites and those mentioned by proponents for the three "updated" case studies.

The NODF case studies conducted as part of this evaluation suggest that the NODP is achieving longer-term outcomes. Specifically, they provide examples of jobs, created during a project, that were maintained and of expected future employment that has been realized and could be measured by the program.

For example, the Extend Telemedicine Services to Remote First Nations project, implemented by KO, created seven part-time jobs during the project. As of 2005, those jobs became full-time, and individuals were still employed as of November 2010.

The Development of a Marketing Strategy to Attract Potential Business to Timmins Ontario project, implemented by the Timmins Economic Development Corporation, expected to create approximately 300 new jobs in the retail and service sector by the end of 2007. As of April 2010, a total of 281 new jobs had been created.

Table 11 provides mostly qualitative evidence of success of the proponents in their fields, but the lack of continued measurement of outcomes by the proponent and the program provides limited quantitative evidence of these successes.

Issue 4D: What are the linkages and contributions of outputs to outcomes?

The NODP's "most direct and measurable output is contributions for projects to proponents to support activities related to socio-economic development of Northern Ontario."³⁶

It is evident from the findings presented that the NODP-supported projects have led to several outcomes, including socio-economic development in northern Ontario. Many interviewees and case study proponents indicated that NODP-supported projects would not have proceeded or would have proceeded at a reduced scope if NODP funding had not been received.

Table 12 displays the outputs and achieved results for five case studies.

In each case, it is evident that outputs directly contributed to the achievement of results for the project, which contributed to the immediate outcomes of the NODP. For example, the Revitalization of Main Docks in Little Current project improved the community's dock, which led to the opening of businesses along the dock and in the community.

However, the major barrier to measuring outcomes is that follow-up beyond the end date of the projects is not systematically completed to assess whether the intermediate and longer-term outcomes have occurred. For example, the Assist with Launch and Production of the MultiStations OS2 Product Line project was expected to result in the creation of five additional permanent jobs following the completion of the project; however, aside from the data collected at the time of program evaluation, no follow-up was conducted to verify these jobs had been created.

Issue 5: Demonstration of efficiency and economy

Issue 5A: What has been the utilization of resources in relation to the production of outputs and progress toward expected outcomes?

Between April 1, 2005 and March 31, 2010, FedNor distributed $196.55 million in NODP Gs&Css at a total O&M cost of $54.22 million. Therefore, under the NODP, FedNor has an administrative cost of less than 22%.

The cost to administer a community-based Gs&Css program, such as the NODP, can vary significantly depending on the nature of the program. For this program, these costs appear to be reasonable to support FedNor's capacity to deliver its programming over a vast geographic area, with requisite support for effective communications and core services.

The program's operations include a total of 121 FTEs, with decentralized delivery in 7 offices in northern Ontario and travel to monitor projects, provide a high level of service to proponents, and maintain close contact with local communities. Furthermore, the actual administrative costs for the NODP are in line with program and funding approvals.

Interviewees noted that the experience and knowledge of FedNor staff in the regions and their high level of availability and support are instrumental to the development of projects and success of the program. FedNor staff are also involved in a full range of services beyond the administration of contributions that do not result in projects funded by the NODP (e.g., policy advice, coordination, facilitation, referrals). This suggests the administrative costs to deliver the NODP, as currently defined, are reasonable.

The administration costs of other regional development agencies were reviewed to help determine whether NODP administration costs were reasonable. According to a report produced by Canada Economic Development-Quebec (2010), overall administration costs range from about 16% for Canada Economic Development-Quebec to 26% for the ACOA. ³⁷ The report does not separate the budgets for individual programs delivered by the RDAs. However, this does show that NODP's administrative costs are within the range of other Canadian RDAs.

With respect to NODP costs per outcome, 23 of the 40 projects reviewed as part of the file review resulted in temporary and/or permanent jobs created and/or maintained. Those 23 projects created or maintained a total of 1,637 temporary and permanent jobs at a total NODP funding of $8,560,499, and a total project cost of $30,138,317. Therefore, based on the sample files, the cost to the NODP per temporary and permanent job created and maintained was $5,229.³⁸

It should be noted that creating and maintaining temporary and permanent jobs were not the only outcomes expected from these projects. Many of the projects also expected to achieve other outcomes, such as creating, maintaining, and benefiting businesses; distributing loans; and improving access to capital and leveraged capital. However, since these outcomes are not aligned with specific elements of the contribution, it is not possible to separate the outcomes individually.

Issue 5B: To what extent is the NODP cost-effective?

Between April 1, 2006 and January 31, 2010, the NODP approved Gs&Cs support of $148,000,097 for projects totalling $557,548,777. Therefore, for each dollar contributed to a project by the NODP, $2.77 was contributed to the project from other sources, such as the provincial government, municipalities, private sector investment, and proponents. Table 13 shows the overall leverage effect by thematic area.

This table shows that Innovation, CED, and Trade and Tourism are the most successful at attracting other funds whereas Business Financing Support and Human Capital are least successful.

Business Financing Support is the least successful at attracting funds because several of the projects funded under this thematic area are Community Futures Development Corporations (CFDC) capitalization projects, which are 100% funded by the NODP. Although the initial NODP contributions to the investment funds of CFDCs do not leverage funds, the NODP contribution enables CFDCs to make loans, which, in turn, leverage funding. This type of leverage is not identified at the project approval stage, (as is the case for other types of projects), and as a result is not shown in Table 13. However, the number of loans and the funding leveraged as a result of capitalization projects is tracked elsewhere, and reported as final project outcomes.

Similarly, the majority of projects funded under the Human Capital thematic area are Youth Internship Program projects, which are normally funded up to 90% by the NODP.

Interviewees and case study proponents indicated that the NODP is a catalyst in attracting outside sources of funding and that projects would not have gone ahead at all or with the same scope without the NODP funding; however, it is difficult to imagine this is the case for projects where NODP funding was minimal.

Although some interviewees noted that the NODP allocates its resources effectively, they also noted the program could be more cost-effective if it were refocused. Two suggestions were made: the program should allocate funds more strategically instead of scattering small amounts of funding across a broad range of projects, and funds should deal more directly with and support businesses. These suggestions are consistent with the literature.

The literature indicates that past approaches to rural policy focused on redistribution of economic activity. However, the OECD (2006) argues that increasing pressure on public expenditure, combined with the general failure of redistributive approaches to reduce inter-regional disparities, suggests new approaches to rural policy are required. Instead of subsidies, the OECD's The New Rural Paradigm focuses on identifying local economic opportunities and making strategic investments to improve regional competitiveness.³⁹

References to this approach are common in the literature. For example, Goldenberg (2008) argues that "regional and rural policy should be based on strategic investments—in people, in communities, in local resources and assets, and in technology—seeking to stimulate economic development by building on local strengths and opportunities."⁴⁰

Similarly, Bradford (2010) notes that "one of the key evolutions [in federal regional development activity in Canada] has been the progression away from redistributional spending to investments in knowledge and innovation."⁴¹ Markey et al. (2008) assert that "the ascendency of place …tells us that abandoning communities and regions to the vagaries of the market are short-sighted …strategic investments are necessary to establish a foundation upon which diversified economic and community development can flourish."⁴² The alternative approach of spreading support widely is to fund larger projects in communities that have the infrastructure to nurture these investments.

In terms of direct support to business, this approach carries the risk that the business directly supported through the projects may not succeed. One of the factors in the past shift in regional development programming away from direct investment was that business failures were common, which fuelled criticism of government mismanagement. Another impetus to the withdrawal of government from direct investment was the fear that such spending would displace or crowd out private sector lending. This latter concern is less germane in northern Ontario and in the current recession where a shortage of finance capital appears to exist.⁴³

As noted in Section 3.1.1, literature on rural economic development has evolved and increasingly aligns with the view to focus impact and spending on business. Potentially cost-effective regional development concepts mentioned in the literature include:

• Focus on policy issues as they play out in concrete geographic and community settings, as a means to address challenges and opportunities where the impacts are directly felt⁴⁴
• Investment in competitive advantage versus comparative advantage, which simply means identifying activities that are likely to grow through building market share⁴⁵
• Focus on strategic investments that link to other industries or create essential support services along the value chain.⁴⁶, ⁴⁷

4.0 Conclusions and Recommendations

This section provides conclusions and recommendations on the evaluation issues of relevance and performance for the NODP.

4.1 Relevance

Evidence shows that there is a continued need for economic development and diversification in northern Ontario. The region's dependency on primary resource exports makes it vulnerable to price fluctuations and variations in exports that lead to periodic economic cycles of out-migration and high unemployment throughout the region. The objective of the NODP is to promote economic development, economic diversification, job creation, and sustainable, self-reliant communities in northern Ontario.

Although other programs to diversify the northern Ontario economy are available, especially from the province, the literature and interviewees agreed that a federal presence is required. The federal government, through IC, offers leadership and financial resources that complement those of the province, municipalities, non-government organizations, and other institutions to support joint investment in economic development in northern Ontario.

Providing a catalyst for joint activity, leadership among federal departments, and coordination of public resources are the roles identified by interviewees as appropriate and essential for the federal government in regional development programming.

The NODP supports the federal government's priorities and IC's strategic objectives and is consistent with federal roles and responsibilities.

4.2 Performance

Overall, the NODP has been successful at achieving its immediate outcomes and has made progress in this area since the Mid-Term Evaluation. However, evidence in the achievement of intermediate and ultimate outcomes is mixed. Interviewees and case study proponents provide some qualitative evidence of achievement of these outcomes. However, these outcomes are not tracked systematically or measured. FedNor recognizes the need to improve measurement of longer-term outcomes as this has been an issue identified in previous evaluations.

Recommendation 1: To ensure that expected outcomes are being achieved, the program should improve the measurement of medium- and longer-term outcomes.

The NODP costs appear to be reasonable to support program delivery over a vast geographic area and to provide support for effective communications and core services. Interviewees and case study proponents noted the importance of the knowledgeable and experienced FedNor staff in the regions, and indicated how the staff has facilitated the overall success of the program. However, they felt the lengthy project approval process is hampering the success of the program.

Recommendation 2: To enhance program delivery success, the program should consider improving the project approval process to decrease project approval times.

Several factors have contributed to the success of the program. However, one theme emerging from the interviews and recent literature suggests that funding a diverse array of projects is not the most effective way of achieving the program objective. Instead, funds should be allocated more strategically to core activities and targeted more to businesses.

Recommendation 3: To better achieve the overall program objective of economic development and diversification in northern Ontario, FedNor should be more strategic in its allocation of program resources and target program funding to core economic development activities. Consideration should also be given to how to increase program impacts on for-profit businesses.

Management Response and Action Plan

Footnotes of Final Evaluation for the Northern Ontario Development Program

1 – Industry Canada. (2010). Northern Ontario Economic Overview: Fall 2010, p. 4. Provided by program November 26, 2010. Return to text

2 – In 2006, the populations of North Bay, Sudbury, Sault Ste. Marie, and Thunder Bay were 63,424; 158,258; 80,098; and 122,907, respectively (Fillion. [2009]. Growth and decline in the Canadian urban system: the impact of emerging economic, policy and demographic trends. GeoJournal. doi: 10.1007/s10708-009-9275-8 Return to text

3 – Bollman, R., Beshiri, R., & Mitura, V. (2006). Northern Ontario's communities: economic diversification, specialization and growth (Working Paper No. 82). Agriculture and Rural Working Paper Series. Statistics Canada. Retrieved from http://www.statcan.gc.ca/pub/21-601-m/21-601-m2006082-eng.pdf (PDF Format), p. 7. Return to text

4 – Industry Canada. (2010). Northern Ontario Economic Overview: Fall 2010. Provided by program November 26, 2010. p. 3. Return to text

5 – Industry Canada. (2010). Northern Ontario Economic Overview: Fall 2010. Provided by program November 26, 2010, p. 3. Return to text

6 – Ontario Ministry of Infrastructure, & MNDMF. (2009, October). Proposed growth plan for Northern Ontario. Retrieved from https://www.placestogrow.ca/images/pdfs/Northern_Proposed_Growth_Plan.pdf (PDF Format), p. 3. Return to text

7 – Fiscal Architecture and Economic Competitiveness Working Table. (2008). Partnering to compete: strengthening Ontario's economic competitiveness (Working Paper). Retrieved from http://www.amo.on.ca , pp. 11–12. Return to text

8 – North Superior Workforce Planning Board. (2009). Building a superior workforce: 2009–2010 Labour Market Action Plan. Retrieved from http://www.nswpb.ca/assets/files/nswpb_lmap_2009-2012.pdf (PDF Format), p. 68. Return to text

9 – The North Superior Workforce Planning Board makes very similar observations in its discussion of the changing labour force needs in the Thunder Bay District, arguing that the District "has developed a greater dependence on larger employers and a more limited entrepreneurial culture which is key to developing opportunities," adding that, "in essence, entrepreneurism in Thunder Bay has had fewer opportunities to develop, making economic downturns more damaging as businesses are lost" (2009, p. 15). Return to text

10 – Southcott, S. (2007). Aging population trends in Northern Ontario: 2001 to 2006 (No. 3). Census Research Paper Series. Retrieved from http://www.awic.ca/ (PDF Format), pp. 4–5. Return to text

11 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, p. 2. Return to text

12 – IC. (2010). Highlights of the 2009–2010 FedNor Business Plan. Retrieved May 21, 2010, from http://fednor.gc.ca/eic/site/fednor-fednor.nsf/eng/h_fn02348.html Return to text

13 – IC. (2010). Highlights of the 2009–2010 FedNor Business Plan. Retrieved May 21, 2010, from http://fednor.gc.ca/eic/site/fednor-fednor.nsf/eng/h_fn02348.html Return to text

14 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, p. 2. Return to text

15 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, p. 2. Return to text

16 – NODP Terms and Conditions. Return to text

17 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, p. 2. Return to text

18 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, p. 10. Return to text

19 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, p. 12. Return to text

20 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program, pp. 27–28. Return to text

21 – The NODF was the predecessor to the NODP. The NODP name was adopted in July 2006. Return to text

22 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program. Return to text

23 – The application information prepared by project proponents and recorded in the indicator database itemizes the expected immediate outcomes. Most projects identify job creation as an immediate outcome, the exception being funding for business planning and other "informational" activities. Return to text

24 – Government of Canada. 2010. Policy Research Initiatives – Sustainable Places. 10(4). Retrieved from http://www.policyresearch.gc.ca/doclib/2010-0022-eng.pdf. Return to text

25 – For a full description of these concepts, see Annex J. Return to text

26 – See the table in Annex K for a description of the NOHFC. Return to text

27 – Government of Canada. (2010). Speech from the Throne. Retrieved October 3, 2010. Return to text

28 – Department of Finance Canada. (2010). Budget 2010. Retrieved July 29, 2010, from http://www.budget.gc.ca/2010/pdf/budget-planbudgetaire-eng.pdf. Return to text

29 – OECD. (2006). The new rural paradigm: policies and governance. Paris: OECD. p. 106. Return to text

30 – OECD. (2006). The new rural paradigm: policies and governance. Paris: OECD. p. 106. Return to text

31 – Goldenberg, M. (2008). A review of rural and regional development policies and programs. Canadian Policy Research Networks Inc. Retrieved from http://www.cprn.org/documents/49496_EN.pdf, p. iii. Return to text

32 – Industry Canada. (2008). Mid-term evaluation of the Northern Ontario Development Program (NODP): Final Report. Provided by program May 18, 2010. Return to text

33 – AECOM Tecsult. (2010). Analysis of provincial, national and international regional development organizations and programs: Final Report: Canada Economic Development, p. 57. Provided by program November 26, 2010. Return to text

34 – Information on whether WD was meeting its funding application time commitment was not provided. Return to text

35 – AECOM Tecsult. (2010). Analysis of provincial, national and international regional development organizations and programs: Final Report: Canada Economic Development, p. 57. Provided by program November 26, 2010. Return to text

36 – IC/FedNor. (2007). Results-Based Management and Accountability Framework for the Northern Ontario Development Program. p. 6. Return to text

37 – AECOM Tecsult. (2010). Analysis of provincial, national and international regional development organizations and programs: Final Report: Canada Economic Development, p. 66. Provided by program November 26, 2010. Return to text

38 – Total cost per temporary and permanent job created and maintained = $8,560,499 (total NODP assistance) / 1,637 (number of temporary and permanent jobs created and maintained). Return to text

39 – OECD. (2006). The new rural paradigm: policies and governance. Paris: OECD, p. 56. Return to text

40 – Goldenberg, M. (2008). A review of rural and regional development policies and programs. Canadian Policy Research Networks Inc. Retrieved from http://www.cprn.org/documents/49496_EN.pdf, p. 26. Return to text

41 – Bradford, N. (2010). Regional economic development agencies in Canada: lessons for southern Ontario. Mowat Centre for Policy Innovation. Retrieved from http://www.mowatcentre.ca/pdfs/mowatResearch/15.pdf, p. iii. Return to text

42 – Markey, S., Halseth, G., & Manson, D. (2008). Challenging the inevitability of rural decline: Advancing the policy of place in northern British Columbia. Journal of Rural Studies, 24(4), 409–421. doi:10.1016/j.jrurstud.2008.03.012, p. 419. Return to text

43 – For a review of Canada's regional development programs, see Savoie, D.J. (1992). Regional economic development: Canada's search for solutions [2^nd ed.]. Toronto: University of Toronto Press; Polèse, M. & Shearmur, R. (2006). Why some regions will decline: A Canadian case study with thoughts on local development strategies. Papers in Regional Science, 85(1), 24–46; and AECOM Tecsult. (2010). Analysis of provincial, national and international regional development organizations and programs: Final Report: Canada Economic Development. Provided by program November 26, 2010. Return to text

44 – Government of Canada. 2010. Policy Research Initiatives – Sustainable Places. 10(4). Retrieved from http://www.policyresearch.gc.ca/doclib/2010-0022-eng.pdf. Return to text

45 – The standard reference on competitive advantage is Porter, M. (1985). Competitive Advantage: Free Press. Return to text

46 – OECD. (2006). The new rural paradigm: policies and governance. Paris: OECD. Return to text

47 – See Section 3.1.1 and Annex J for additional explanation of these concepts. Return to text

Description of Figure 1: NODP Logic Model

Figure 1 presents the NODP logic model and demonstrates how the activities, outputs, and outcomes of the NODP align. The NODP has two key activities: to provide advice and funding support. The program has one output which is to provide contributions for projects. These contributions are focussed in six key program areas. They are as follows: community economic development; trade and tourism; innovation; information and communications technology; business financing support; and human capital. The immediate outcomes for NODP stem from the outputs and are specific to the six program areas listed above. They are as follows: community economic development leads to improved community capacity for socio-economic development; trade and tourism leads to enhancement of markets and investment attraction; innovation leads to enhanced innovation capacity and improved infrastructure; information and communications technologies leads to enhanced ICT applications and infrastructure; business financing support leads to improved access to capital and leveraged capital; and human capital leads to attraction, retention and development of human capital. Intermediate outcomes result from the culmination of immediate outcomes over time and can be considered as medium-term outcomes. These outcomes are not typically under the complete control of the program, but could be considered to be in the realm of influence, in terms of program effort. NODP has been designed to achieve the intermediate outcome of improved competitiveness and/or sustainability of communities and key sectors. Ultimate outcomes are longer-term and subject to influences beyond the program itself. They are considered to be at a more strategic level. The ultimate outcome of NODP is long-term, sustainable socio-economic development in Northern Ontario.

Return to figure 1

Description of figure 2

The scale is represented by a long, horizontal rectangle. The long horizontal rectangle starts as transparent on the left edge and progressively becomes darker and darker until it is solid black at the end on the right edge. There are five response categories and the five categories are equally spread out across the long, horizontal rectangle. On the left, the rectangle is totally clear for the "Few" (about 20% or less) category. It is mainly clear for the "Some" (about 20% to 40%) category. It has become light grey for the "About half" (about 40% to 60%) category. It is dark grey when it reaches the "Most" (about 60% to 80%) category. By the end, it is solid black for the "Many/all" (about 80%) category.

Return to figure 2