Formative Evaluation of the Eastern Ontario Economic Development Fund (EODF)

This page has been archived on the Web

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Final Report

Audit and Evaluation Branch
Industry Canada

November 2005

Tabled and approved by DAEC on with the exception of Recommendation No. 1.
Recommendation No. 1 was not supported by DAEC members since evaluation findings were deemed to be insufficient to support this recommendation.


Table of Contents


List of Acronyms

CFDC–Community Futures Development Corporation

EODF–Eastern Ontario Development Fund

FedNor–Federal Economic Development Initiative for Northern Ontario

RBAF–Risk Based Audit Framework

RMAF–Results-based Management and Accountability Framework

Audit and Evaluation Branch
Industry Canada

November 2005

Executive Summary

Introduction

The Eastern Ontario Development Fund (EODF) is a new initiative, launched in October 2004, to promote socioeconomic development in rural Eastern Ontario by building local capacity for development and creating the necessary conditions for increased business and employment opportunities. This program is an initial response to the Government's commitment in Budget 2004 to review the role played by Community Futures Development Corporations (CFDC) in regions without a regional development agency, such as Eastern Ontario, and examine the possibility of broadening and deepening their mandate.

This study is a formative evaluation of the EODF, which assesses the design, delivery and implementation of the program. The evaluation results are to provide evidence critical to FedNor program planning and design in light of the current March 31, 2006 sunset for the program.

For more details, please refer to Section 1.0 of this report.

Methodology

The evaluation addressed a series of research questions categorized according to the following issue categories:

  • relevance;
  • design and delivery;
  • success;
  • monitoring and accountability;
  • alternatives and cost-effectiveness; and,
  • lessons learned.

The evaluation used a range of methodologies to ensure that multiple lines of evidence were available for each issue. The methodologies were as follows:

  • document and database review;
  • 27  key informant interviews with Industry Canada / FedNor management and staff, representatives of the CFDCs in Eastern Ontario and other stakeholders;
  • a telephone survey with 250 EODF funding recipients; and,
  • five case studies which provided an in-depth analysis of the implementation of a specific program element in a particular CFDC or of a particular project.

While there are some strengths and limitations to the methodology, overall, the approaches and sample sizes used for this study resulted in a strong and reliable formative evaluation.

For more details, please refer to Section 2.0 of this report.

Findings – Relevance

A grassroots approach was used to initially design the EODF programming. As such, it is not surprising that the evidence gathered during this evaluation reveals that the EODF programming is relevant to the needs of the community. This is evident in the documents and all interviewees agreed that there was a need for this type of programming and that the way in which the EODF was designed was relevant to this need.

The program is designed around five priorities: business development, access to capital, skills development, retention and attraction of youth, and technological enhancements. Given that the program was initially designed by the 15 Eastern Ontario CFDCs with FedNor involvement, it is not unexpected that the FedNor management and staff, CFDC representatives and stakeholders who were interviewed all agreed that the priorities were relevant. It was noted that there was a greater need for some of the priorities in certain communities and that the need could change over time. However, all five priorities are relevant, overall, and are general enough to be adaptable to the evolving economic development needs of communities across Eastern Ontario. This is evident in the early program data that shows that as of March 31, 2005, 48% of the applications received were for business development (this included three funding streams which explains the relatively high proportion of applications), 25% were for skills development, 15% for youth internships and 12% for access to capital. The recipients surveyed were also highly satisfied that the EODF was able to address their needs and, regardless of their own particular needs, that the five EODF priorities were extremely important (mean importance of 9.4 for business development – 10 meaning extremely important; 9.3 for access to capital; 9.2 for skills development; 8.9 for retention and attraction of youth; and, 8.8 for technological enhancement).

Documents show that proper care was taken to ensure that the EODF does not overlap or duplicate other government programs. The interviewees all agreed that the EODF complements existing programming, which was noted as being very limited in Eastern Ontario. In fact, only 16% of surveyed recipients indicated that they were aware of programs or services of the federal or provincial government which are comparable or similar to the EODF.

For more details, please refer to Section 3.1 of this report.

Findings – Design and Delivery

The EODF service delivery model involves a blend of CFDC and FedNor decision-making. Projects that are funded through secondary recipients (i.e., Youth, Skills Development, Business Development and Access to Capital) are delivered by the CFDCs at various funding levels. The exceptions are Community Capacity and Regional projects which are approved by FedNor / Industry Canada (IC), regardless of the funding level. The CFDCs are then responsible for the negotiation and ongoing monitoring of all local projects, although FedNor is closely involved in monitoring the larger local projects. If a project is regional, the application is reviewed by a committee comprised of representatives from each of the 15 CFDCs. Each CFDC gets one vote on whether the project should be recommended for funding. If the majority of CFDCs agree on a regional project, the application goes to FedNor for review and approval, regardless of the size of the project. If approved, one CFDC is assigned as the lead for the project; as such, this CFDC is responsible for the negotiation and ongoing monitoring of the project as well as for ensuring proper sharing of progress and results with the other CFDCs.

While there were some concerns noted with parts of the service delivery model, in general, all interviewees believed that, when weighing the pros and cons, the model was sound and appropriate. The comments made by interviewees were supported by those made in the survey. Overall, the surveyed recipients were very satisfied (mean of 8.9 our of 10) with the EODF.

While the EODF is a relatively new initiative, all sources (documents, interviews, survey) point to the fact that it was widely promoted and that it is well known. While there is considerable variation among the 15 CFDCs in the use of their website to promote the program, 13 of the 15 have information on the EODF on their website. In addition to the website, interviewees noted announcements, press releases, newspaper and radio advertisements and articles / interviews, meetings, mailings, word of mouth, and several other means to promote the program. Interviewees generally believed that, even though some people were not aware of the EODF, most of those who were potential recipients were well aware of the program. The surveyed recipients were also highly likely to indicate that they believed that the EODF was well known (62% responded very or fairly well known).

Interviewees generally believed that the selection process for local projects was effective. The actual process may differ from one CFDC to another, however, the process was deemed to work for the individual needs of the communities. However, several interviewees had concerns with the project selection process for regional projects. Concerns were raised because of the belief that some of the regional projects were not truly regional, that some could have been done with less EODF funding, and / or that some of the regional projects were not that good in terms of their potential impacts.

The project selection criteria (i.e., consistency with objectives, quality of supporting plans, project feasibility, incrementality, job creation and other lasting community economic benefits, leveraging, and track record of partners) were deemed to be clear, appropriate and applied, for the most part by those interviewed. Recipients were highly or fairly satisfied with the clarity of the eligibility criteria (8.4 out of 10), the fairness of the selection process (8.9 out of 10), their understanding of the decision-making process (7.7 out of 10), and the speed of decision-making (8.4 out of 10).

According to interviewees, the program design, promotional efforts, and selection process have resulted in projects that are relevant.

FedNor interviewees were, in general, satisfied that they had provided adequate support to the projects and that this support was important. CFDC representatives were also very satisfied with the support provided by FedNor. They noted that the support was appropriate and sufficient. Several commented on the outstanding job that FedNor had done with respect to this initiative, particularly in light of the limited time available for implementation and the limited FedNor human resources dedicated to the EODF.

For more details, please refer to Section 3.2 of this report.

Findings – Success

While the EODF is in its very early stages and many projects are not yet completed, there is already evidence that it is successful in achieving its intended outcomes. The case studies completed during the study were all indicative of the wide range of actual and potential impacts of the program and its projects. Additionally, the table below highlights the early results of the projects, based on the survey of recipients.

EODF Project Results
Result Has occurred Will occur Will / may not occur / did not expect
Development of a new business 13% 12% 75%
Retention of an existing business 43% 8% 49%
Improved competitiveness 55% 20% 25%
Attraction of people to your organization or region 52% 19% 29%
Retention of people within your organization or region 58% 11% 31%
Development of people within your organization 65% 9% 26%
Development of business skills 58% 8% 34%
Development of trade skills 42% 6% 52%
Increased use of technology 44% 5% 51%
Development of an innovation 32% 10% 58%

What is particularly important about these results is that they are highly attributable to the EODF because the program is incremental. That is, 34% of all surveyed recipients indicated that they could not have undertaken the project and that this would have had a major negative impact; another 41% would have been negatively affected in a major way in that the project would have been delayed, would have taken more time, would have been of lower scope or quality, or for some other reason.

For more details, please refer to Section 3.3 of this report.

Findings – Monitoring and Accountability

Even though the EODF is in its early days, a lot of progress has been made in putting in place a performance monitoring and reporting system. What is currently in place includes:

  • project recipients are required to report to the CFDCs;
  • CFDCs compile the individual project results and provide reports to FedNor for each large or regional project and for each of their contribution agreements (one for each program priority); and,
  • FedNor inputs this information into a database for further analysis and reporting.

The document review and the interviews revealed that recipients and CFDCs have been fairly diligent in providing timely, complete and accurate report, to the best of their capability. However, the document review and interviews identified some challenges with reporting requirements. The principal challenges are:

  • the reporting requirements apply to all projects regardless of the size of EODF funding, the objectives of the projects, their intended impacts, etc. – therefore, the recipient of a very small project with limited albeit important objectives must report according to the same requirements as the recipients of larger projects with complex objectives;
  • the reporting requirements, while similar, do not match the program structure;
  • CFDCs roll up the results of several projects into one report for each of the contribution agreements they have with FedNor for this initiative; unfortunately, the report is structured for reporting with respect to only one project as opposed to several projects within one contribution agreement;
  • there is confusion with reporting on FedNor target groups in terms of what to report and why it needs to be reported (i.e., it is not necessarily important for some communities to target certain groups, yet they still need to report on the reach of those target groups); and,
  • the key indicators identified in the Results-based Management and Accountability Framework (RMAF) for the EODF are not all covered through the reporting requirements or do not directly meet the requirements.

For more details, please refer to Section 3.4 of this report.

Findings – Alternatives and Cost-Effectiveness

There were three service delivery approaches presented by the interviewees. The first is the status quo (blend of CFDC and FedNor decision-making), the second was to give more (or full) control to the CFDCs for making decisions on projects in their communities, and the third was to give more control to FedNor so that the program is administered the same as the Northern Ontario Development Fund, where FedNor deals with more than just CFDCs as primary recipients. In general, interviewees agreed that the current approach was the best alternative as demonstrated by the results of the program, to date.

In terms of cost-effectiveness, the evidence shows that the program is cost-effective in the following key ways:

  • The FedNor operational component of this initiative represents less than 4% of the total budget for this initiative. The CFDC operational / administrative component represents approximately 6% or 7% of the budget. The total operational is therefore approximately 10% of the total budget.
  • A lot of the work done at the CFDC level is done by the volunteer Boards of Directors. Across the 15 CFDCs, this could represent as much as 180 person-days of work, in total, for each application deadline.
  • The program funding encouraged the financial contributions of other partners on the project. As of March 31, 2005, for close to $9 million in approved EODF funding, the program leveraged an additional $28.6 million. The program therefore funded less than 25% of the total project costs, yet was highly incremental to the organizations' ability to undertake the projects.
  • As of March 31, 2005, 643 projects were approved for close to $9 million in approved funds. This represents average projects of less than $14,000 in EODF funding. As reported previously, these projects are already reporting significant economic development outcomes and impacts (see Success summary).

For more details, please refer to Section 3.5 of this report.

Findings – Lessons Learned

The key lessons learned are based on evidence presented throughout this report as well as on suggestions made by interviewees and recipients. These are summarized in the conclusions and recommendations table in the next section.

For more details, please refer to Section 3.6 of this report.

Conclusions and Recommendations

Conclusions and Recommendations
Conclusions Recommendations
Relevance

Conclusion 1

The EODF programming is relevant to the economic development needs of the communities in Eastern Ontario. To a large extent, this is due to the fact that the program was designed by the 15 Eastern Ontario CFDCs based on their familiarity with the socioeconomic challenges faced by the communities they serve.

Recommendation 1

The EODF should be renewed.

Note: the balance of the recommendations are based on the assumption that the EODF will be renewed.

Conclusion 2

The five program priorities (business development, access to capital, skills development, retention and attraction of youth, and technological enhancements) are relevant, again due to the fact that the program was designed by the 15 CFDCs. While the specific need for funds across these priorities may have differed across CFDCs, these are legitimate and critical priorities to help Eastern Ontario address its economic development gaps. Additionally, while none of the projects were classified under the technological enhancement priority, this is because of the way some projects were classified and not because it is not an important priority.

Recommendation 2

The five program priorities should continue to form the basis of EODF programming. These encompass current priority needs and are broad enough to address evolving economic development needs.

Conclusion 3

The EODF does not duplicate or overlap with other government programming. The EODF is unique in Eastern Ontario.

See Recommendation 1

Design and Delivery

Conclusion 4

The service delivery model for the EODF is appropriate given the program objectives. While the CFDCs need to ensure that they involve other important economic development groups, the network of CFDCs is an excellent delivery mechanism. This is due to the fact that CFDCs have an existing relationship with FedNor, are recognized players in their communities at the economic development table, and provide complementary services through their existing programming. Additionally, CFDCs are unique in their regional rather than local perspectives.

Recommendation 3

The EODF should continue to use the existing delivery model which involves FedNor contribution agreements with each of the 15 CFDCs who, in turn, provide funding to secondary recipients.

Conclusion 5

Even though the program was implemented within a very short time frame and is in its infancy, the CFDCs used a wide range of effective means to promote it to ensure that it is well known. While there is still an ongoing need to continue to promote the initiative and make more organizations aware of its availability, the CFDCs received more applications than they could approve within their budgets.

Recommendation 4

The CFDCs should continue to use a multitude of promotional tools to ensure that their communities are appropriately aware of the EODF. However, promotional strategies should consider the limited availability of EODF funds.

Recommendation 5

While the promotional efforts of the CFDCs have been fruitful, a process should be implemented to ensure that promotional materials are shared and to thus avoid duplication of efforts and build on best practices.

Conclusion 6

The project selection process for small local initiatives is effective. While the exact process differs from one CFDC to the next, each was able to receive and review a large number of applications within a very short time frame and approve projects that were successful.

Recommendation 6

The current project selection process for small local initiatives does not need to change. However, since CFDCs have implemented different processes for the review and selection of projects, the different processes should be shared among CFDCs to help in the sharing of best practices and lessons learned.

Conclusion 7

The project selection process for larger local initiatives where FedNor needed to be involved in the approval process is also effective. The initial review and recommendations of the CFDCs was effective in that few recommended projects were not approved by FedNor. Additionally, FedNor was very effective in reviewing and approving these applications.

See Recommendation 6, but for larger local initiatives.

Conclusion 8

The project selection process for regional projects is not as effective as it could be. The definition of regional projects is unclear. However, the process involving all CFDCs in the review and recommendation of projects is appropriate, as are the role of FedNor in approving the applications and the consultations with other regional economic development stakeholders.

Recommendation 7

The project selection process for regional projects needs to be clearly defined and agreed upon by FedNor and the CFDCs. This should include a clear definition of the scope of regional projects. It could also include a series of outcomes that are regional in nature. The improved project selection process should, however, continue to involve consultations with other regional economic development stakeholders with respect to needs for regional EODF projects. In refining the project selection process, FedNor and the CFDCs will need to clarify the following:

  • definition and scope of regional projects.
  • how regional projects should be approved.
  • how project leads should be identified (one CFDC, several CFDCs, others).
  • how regional projects should be monitored;
  • etc.

In refining the project selection process, care should be taken to ensure that the factors that have made this initiative successful (such as its grassroots aspect) are preserved or built upon, and that other recommendations are taken into account.

Conclusion 9

The project selection criteria are clear, appropriate and applied. The relevance of the projects selected (as demonstrated by the appropriateness of the early results) are indicative of the fact that the criteria are applied and appropriate given the program objectives.

Recommendation 8

The current project selection criteria should not change.

Conclusion 10

The role of FedNor in support of the projects is important given its experience with other programming and with similar types of projects in Northern Ontario. This support is appropriate and sufficient.

Recommendation 9

Notwithstanding other recommendations, FedNor should continue to provide the same level and type of support to CFDCs and to ensure that adequate resources are allocated to provide this support.

Success

Conclusion 11

The EODF has been successful, even in its infancy, in achieving its intended immediate and intermediate outcomes. Even though many projects were still ongoing at the time of the evaluation, all projects have resulted in at least one of the program's intended outcomes.

See Recommendations 1 to 9

Conclusion 12

The EODF is helping organizations undertake projects that they could not otherwise undertake or would have to complete differently (lower quality, smaller scope, delayed, etc.). The program is therefore highly incremental. As such, the success of the projects is highly attributable to the program.

See Recommendations 1 to 9

Monitoring and Accountability

Conclusion 13

FedNor has made good progress in implementing the performance measurement strategy outlined in its Results-based Management and Accountability Framework (RMAF). In addition, the CFDCs have been very diligent in respecting their commitments regarding progress reporting. As such, they have taken the steps required to ensure that they have the information they need to meet these commitments. However, some of the reporting requirements are not clear. Additionally, it is not evident that all RMAF commitments are met through the existing reporting structure. Performance measurement is still a work-in-progress.

Recommendation 10

FedNor should continue to improve its performance measurement system with respect to the EODF. This should include updating the RMAF in consultation with the 15 CFDCs to ensure that they are in agreement with the commitments outlined in this document. It should also include revising the reporting structure to ensure that it is consistent with the program's structure as well as the RMAF indicators. The revised reporting requirements should include clear definitions and instructions.

Recommendation 11

Once the performance reporting requirements are finalized, FedNor should give due consideration to providing CFDCs with more effective tools for reporting. This could include the grants module of The Exceptional Assistant (TEA).

Alternatives and Cost-Effectiveness

Conclusion 14

There are no alternative service delivery approaches for the EODF that would produce the same results at a lower cost. Currently, the administrative portion of this initiative is reasonable, the CFDCs rely on the large contribution of its volunteer Boards of Directors, a significant proportion of the projects costs are leveraged through other sources, and the projects are contributing to the achievement of the intended outcomes. The program is therefore cost-effective.

See Recommendations 1 and 3 and:

Recommendation 12

FedNor and the CFDCs should monitor the appropriateness of the operational budgets for the EODF. It is important to ensure that, with the continuation of the program, the burden on FedNor and CFDC staff is not unreasonable. It is also important to ensure that the program requirements do not conflict with other FedNor and CFDC staff priorities.

Recommendation 13

CFDCs should continue to require that, where possible, additional sources of funding be sought by the secondary recipients.

Lessons Learned

Conclusion 15

The relationships among all players involved in this initiative was key to the success of the EODF. These included the relationship: 1) across the 15 CFDCs who worked extremely well in reaching consensus on the design and delivery of this initiative; 2) between the CFDCs and FedNor where the pre-existing relationship helped to contribute to the smooth implementation of the initiative and the commitment of both parties to the initiative helped make it a success; 3) within the CFDCs among staff and with the Boards where staff demonstrated a high commitment to make this initiative work and the Board members worked with staff to ensure that the project applications were reviewed and approved in a timely fashion; and 4) between the CFDCs and the community where the fact that the CFDC was a recognized credible economic development organization which knew the community and vice versa made it easier to roll out the program within a very short time frame.

See all previous recommendations

Conclusion 16

Flexibility was also key to the success of this initiative, yet ongoing and increased flexibility may help ensure the continued success of the initiative. The program was set up by the CFDCs to provide each CFDC the same amount for each of the program elements. However, enough flexibility was incorporated into the guidelines so that the CFDCs were able to redistribute some of their budgets across components in order to better meet the needs of their communities. While some CFDCs would have liked more flexibility right from the start, the combination of guidelines and flexibility within those guidelines helped ensure that the intended program results were achieved while ensuring that community needs were considered. Nevertheless, increased future flexibility, within set parameters, may contribute to a more effective decision-making process and ensure that the needs of the communities are better met. Additionally, more flexibility regarding bureaucratic requirements for very small projects could help make the program more cost-effective.

Recommendation 14

FedNor should consider the feasibility of putting in place a process to have only one contribution agreement with each of the 15 CFDCs. This agreement would include all elements of the program for local initiatives (i.e., excluding regional projects and community capacity building, but including business planning, access to capital, skills development, retention and attraction of youth, and technological enhancements) with guidelines as to the minimum and maximum proportion of funds to be allocated to each of these elements. This would provide the flexibility required at the local level while ensuring that all priorities are addressed.

Recommendation 15

FedNor should consider implementing a risk-based approach for its monitoring and reporting strategy. This should be integrated in the considerations for Recommendation 10.

Conclusion 17

In order to ensure good ongoing performance reports, it is key to make sure that the performance measurement system is reflective of the program and that performance reporting requirements are clearly defined. The RMAF for the EODF was developed by FedNor. Because of extremely tight deadlines, CFDCs were not directly involved in the development of this RMAF. While FedNor did attempt to consult with some CFDC representatives, these consultations were limited and most CFDCs were not familiar with the document. This presented challenges in that CFDCs did not fully understand (or agree with) the program logic model and performance indicators. As such, they did not understand some of the reporting requirements. Additionally, some of the reporting requirements were not clear and were therefore interpreted differently from one CFDC to the next. Finally, the key indicators identified in the RMAF were not fully integrated into the performance reporting requirements.

See Recommendation 10

Introduction

1.1 Introduction to the Study

The Eastern Ontario Development Fund (EODF) is a new initiative, launched in October 2004, to promote socioeconomic development in rural Eastern Ontario by building local capacity for development and creating the necessary conditions for increased business and employment opportunities. This program is an initial response to the Government's commitment in Budget 2004 to review the role played by Community Futures Development Corporations (CFDC) in regions without a regional development agency, such as Eastern Ontario, and examine the possibility of broadening and deepening their mandate.

This study is a formative evaluation of the EODF, which assesses the design, delivery and implementation of the program. The evaluation results are to provide evidence critical to FedNor program planning and design in light of the current March 31, 2006 sunset for the program.

1.2 Profile

Origin and Rationale for EODF

Rural Eastern Ontario faces socioeconomic challenges including a dependency on declining resource industries, out-migration of youth, insufficient telecommunications capacity, economic infrastructure deficiencies and the need for collaborative strategies and implementation capacity to achieve the area's sustainable potential in the global knowledge-based economy.

In May, 2004, the Government approved the creation of a $10 million fund for 2004–05, which could be accessed by the 15 CFDCs of Eastern Ontario as a short-term response to addressing the stated challenges of the area and the commitment made in Budget 2004 to review the role played by CFDCs in areas without a regional development agency.

The major role played by Eastern Ontario CFDCs is justified in part by a program evaluation in 2003, confirming that Ontario CFDCs are relevant, successful and cost-effective with the potential to accomplish more given increased resources. In addition, experience has demonstrated that CFDCs can be effective in implementing regional economic development initiatives, where resources are available.

Community Futures Development Corporations in Eastern Ontario

The Community Futures Program is specifically designed to support rural communities in meeting their own socioeconomic development needs. In Eastern Ontario, the program supports a network of 15 CFDCs, which are incorporated, non-profit organizations governed by local volunteer Boards of Directors. As such, the program offers an appropriate mechanism to enable rural communities in Eastern Ontario to address socioeconomic challenges by undertaking a variety of community economic development initiatives.

The Community Futures Program, administered by FedNor, provides financial contributions towards the operating costs as well as investment capital to support CFDC activities at the community level. There are 61 CFDCs in rural Ontario (37 in Southern Ontario which includes the 15 CFDCs in Eastern Ontario, and 24 in Northern Ontario).

Objective

The objective of the EODF is to promote rural socioeconomic development in Eastern Ontario leading to a competitive and diversified regional economy and contributing to the successful development of business and job opportunities and sustainable self-reliant communities.

Eastern Ontario CFDCs are eligible to receive additional funding in support of community-based initiatives which stimulate business development opportunities; attract and retain youth; support skills development; provide access to capital for new and existing businesses; and support technological enhancements. This program has been designed to support Industry Canada's Strategic Outcomes of an innovative economy and competitive industry and sustainable communities.

Stakeholders and Beneficiaries

For the purpose of the EODF, Eastern Ontario is considered to be the area which extends east from Durham Region, south of Algonquin Park and is bounded by the Quebec border. This geographic area encompasses the service areas of 15 CFDCs and excludes the cities of Kingston and Ottawa.

The 15 CFDCs in Eastern Ontario are the primary clients of this initiative.

CFDCs in turn undertake a range of activities in support of economic renewal in Eastern Ontario and provide funding to secondary recipients. The secondary recipients represent a broad cross section of community stakeholders which can include, among others, the private sector, non-profit organizations, municipalities, education, labour and First Nations.

Management and Administration

EODF is managed regionally by a FedNor Manager of Program Delivery who reports to FedNor's Director General. FedNor Program Officers work closely with the Boards of Directors and professional staff of the CFDCs to effectively administer the EODF.

CFDCs must establish separate revenue and expense records for EODF activities. Each agreement must be accounted for separately from all other funding.

CFDCs are required to track and report on the use of EODF funds as outlined by the accountability framework, and more specifically according to the Performance Measurement Strategy.

Logic Model

The logic model for the EODF is provided on the following page (source: EODF RMAF).

top of page

1.3 Report Structure

Performance Management Network Inc. was contracted to undertake this formative evaluation of the EODF. This report addressed the formative issues included in the Results-based Management and Accountability Framework (RMAF) and Evaluation Plan and is organized as follows:

  • Section 2 provides an overview of the methodology used to address the evaluation issues.
  • Section 3 describes the evaluation findings on issues of relevance, success, cost-effectiveness and results-based management.
  • Section 4 summarizes the conclusions made throughout the report and provides FedNor management with recommendations for improvement.

A list of Steering Committee members is provided in Annex A. A list of documents reviewed is provided in Annex B. A sample of the interview guides is provided in Annex C and a list of interviewees in Annex D. A sample of the survey questionnaires is found in Annex E and the detailed survey tables in Annex F. Finally, the case study write-ups are provided in Annex G.

Eastern Ontario Development Fund Logic Model
Eastern Ontario Development Fund Logic Model
Larger view
[Description of logic model]

2.0 Methodology / Approach

2.1 Evaluation Issues

Table 1 over the next several pages identifies the specific issues and how each of the approaches contributed to each issue.

2.2 Methodology

2.2.1 Document and Database Review

A series of documents were reviewed to help address some of the issues. The document review consisted of program documents obtained through Industry Canada / FedNor and CFDCs throughout the course of the study. Information from these documents were incorporated, as relevant, into the various sections of the report. Some of the documents served as background information to some of the issues, whereas others provided factual evidence to help reach conclusions on some of the issues. A list of the documents reviewed is provided in Annex B.

Additionally, data on the program was obtained, analyzed and integrated into the appropriate sections of this report.

2.2.2 Key Informant Interviews

A total of 27 key informant interviews were completed. The list of potential interviewees was determined in close consultation with Industry Canada / FedNor. These interviews were 30 to 60 minutes in length and were all completed by telephone at a time and date that was convenient to the interviewee, in the language of choice of the respondent. Once the interview was scheduled, the interviewee was forwarded a copy of the interview guide (see Annex C) to help him / her better prepare. In one case, one interview involved two individuals (therefore 28 individuals were interviewed). Interviews were completed with Industry Canada / FedNor managers and staff, CFDC representatives and other stakeholders. Table 2 provides a summary of the distribution of the individuals interviewed. A list of interviewees is provided in Annex D.

Table 1: Evaluation Issues by Source
Evaluation Issues Document Database Review IC / FedNor CFDC Stakeholders Recipient Survey Case Studies
Relevance
1. Is EODF programming relevant to the community? Are EODF priorities relevant? Medium Low High High High High Low
2. Are there any overlaps with other government programs? Is FedNor collaborating with other departments and governments to reduce overlaps? Low Low High High High Medium Low
Design and Delivery
3. Is the FedNor service delivery model appropriate ( CFDCs as only recipients)? Low Low High High Medium Low Low
4. Is the EODF well promoted? Well known? Medium Low Medium Medium Medium Medium Low
5. Is the EODF project selection process effective? Are the selection criteria appropriate? Medium Low Medium Medium Low High Low
6. Are EODF projects relevant? Do they meet the program criteria? High Low Medium High Medium Low Medium
7. Are project clients satisfied with counselling, advice, training or workshops provided? Low Low Low Medium Low High Low
8. Is FedNor playing an appropriate role in support of the projects? Low Low Medium High Low Low Low
Success
9. To what degree has the EODF contributed to the development and retention of new and existing businesses? To what degree had the EODF improved competitiveness of rural Eastern Ontario firms? Medium Medium Medium Medium Medium High High
10. To what degree has the EODF helped attract, retain and develop human capital, including special client groups? (e.g., women, Francophones, First Nations and other Aboriginals, youth). Medium Medium Medium Medium Medium High High
11. To what degree has EODF contributed to the economic development of rural Eastern Ontario communities? Medium Medium Medium Medium Medium High High
12. To what degree has EODF helped develop business and trade skills? Medium Medium Medium Medium Medium High High
13. To what degree has EODF helped develop external markets for rural Eastern Ontario firms? Medium Medium Medium Medium Medium High High
14. To what degree has EODF increased the use of technology? Has contributed to the development of innovation? Medium Medium Medium Medium Medium High High
15. What is the incremental impact of EODF projects? Low Medium Medium Medium High High High
Monitoring and Accountability
16. Is reporting timely, reliable and complete? Are reports providing FedNor useful information for management purposes? Low Low Medium Medium Low Low Low
17. Is the reporting system allowing FedNor to meet accountability requirements? Medium Low High Low Low Low Low
Alternatives and Cost-Effectiveness
18. Are there alternative service delivery approaches? Are there alternatives to increase the cost-effectiveness of EODF programming? Low Low High High Medium Medium Low
Lessons Learned
19. What lessons have been learned from EODF programming for the future? Low Low High High Medium Medium Medium
20. What factors have facilitated and / or impeded implementation of EODF? Low Low High High Medium Medium Medium
Table 2: Number of Interviews Completed by Type of Interviewees
Type of Interviewee # of Interviews # of Individuals
Industry Canada / FedNor 7 7
CFDC representatives 11 12
Other stakeholders 9 9
Total 27 28
top of page

2.2.3 Survey of Secondary Recipients

A telephone survey was completed with 250 secondary recipients of EODF funding. For the purposes of the survey, secondary recipients included those organizations that had received funding under EODF to complete a specific project or endeavor. Secondary recipients excluded the CFDCs which are considered primary recipients.

The survey interviews were completed with secondary recipients from all 15 CFDCs and from the five program components, as per Table 3 below. The table shows that the sample represents fairly well the actual distribution of secondary recipients by program components as well as by CFDC.

Table 3: Survey Distribution Versus Actual Distribution of Projects
Description Survey Actual
# % # %
Program Component
Access to capital 14 5.6 40 7.1
Business development 94 37.6 230 40.9
Retention and attraction of youth 45 18.0 93 16.5
Skills development 96 38.4 199 35.3
Technological enhancement 1 0.4 1 0.2
Total 250 100.0 563 100.0
CFDC
1000 Islands 14 5.6 31 5.5
Frontenac 20 8.0 28 5.0
Greater Peterborough 13 5.2 32 5.7
Grenville 21 8.4 30 5.3
Haliburton 22 8.8 45 8.0
Kawartha Lakes 23 9.2 29 5.2
North Central Hastings 14 5.6 47 8.3
Northumberland 9 3.6 25 4.4
Prescott Russell 20 8.0 41 7.3
Prince Edward / Lennox and Addington 7 2.8 25 4.4
Renfrew County 13 5.2 37 6.6
South Lake 16 6.4 32 5.7
Stormont, Dundas and Glengarry 24 9.6 69 12.3
Trenval 13 5.2 52 9.2
Valley Heartland 21 8.4 40 7.1
Total 250 100.0 563 100.0

The survey questionnaire is included as Annex E and the detailed survey results are provided in Annex F.

2.2.4 Case Studies

Five case studies were completed. The case studies involved a review of the relevant case-related documents provided by Industry Canada / FedNor, interviews with the Industry Canada / FedNor Project Officer, the CFDC representative and the project recipient(s). Brief case study write-ups were prepared. The specific cases are as per Table 4 below.

Table 4: Case Study Distribution
Title CFDC Component $
Youth Internship Initiative 1000 Islands CDC Retention and Attraction of Youth $182,000
Skills Development Initiative Northumberland CFDC Skills Development $319,000
Local Initiatives and Business Planning CFDC of Stormont, Dundas & Glengarry Business Development $289,425
Implement Phase 1 of Broadband Network Plan South Lake CFDC Business Development $75,000
Test / Evaluate Three-Stage Electronic Waste Diversion Facility Frontenac CFDC Business Development $29,000

The case study write-ups were verified with the recipient organizations for accuracy. The final write-ups are provided in Annex G.

2.3 Strengths and Limitations

Overall, the approaches and sample sizes used for this evaluation resulted in a strong and reliable formative evaluation, which provided the evidence to conclude on all issues. Additionally, the overall evaluation methodology is strong because multiple lines of evidence were used for all issues. Nevertheless, there are some strengths and weaknesses associated with each approach which are important to recognize. These are summarized in Table 5.

Table 5: Methodological Strengths and Weaknesses by Approach
Approaches Strengths Weaknesses
Review of Documents A wealth of documents were provided by program management as well as from CFDC representatives. Where applicable, these provide documented evidence related to specific issues The documents are limited to those provided by Industry Canada / FedNor and the CFDCs. It does not involve a thorough literature review on program-related issues. Additionally, it is difficult to assess if all relevant documents were provided.
Data Review This provides quantitative factual information on project results. The information in progress reports was inconsistent and the completeness of these reports varied from one project to the next. This made some of the data unreliable and therefore unusable for evaluation purposes.
Interviews This provides an opportunity to obtain in-depth, qualitative information on how the program was managed and on how CFDCs delivered the program The only limitation is that, in order to reduce burden, some people were not interviewed specifically on the program, overall, because they had already been interviewed in the context of case studies. Additionally, due to budgetary constraints, the number of interviews was fairly limited, albeit sufficient in the context of a formative evaluation.
Survey of Secondary Recipients This provides an opportunity to obtain quantitative information on how the program was implemented as well as on early results. The survey approach did not provide as much opportunity for probing into some of the responses and is therefore limited from a qualitative perspective, although some qualitative information was obtained.
Case Studies Case studies are useful methods for assessing some of the project impacts / results in greater depth. Case studies can be used to highlight lessons learned and best practices related to the specific contributions of projects to community economic development. Because of the extensive resources involved in completing case studies, only a limited number could be completed. While not meant to be representative, it was still not possible to cover all program components using this approach.

3.0 Findings

3.1 Issues – Relevance

3.1.1 Is EODF programming relevant to the community? Are EODF priorities relevant?

Evidence from the document review shows that there have been extensive efforts to ensure that the EODF program elements are well aligned to address the needs of Eastern Ontario rural areas. Experience in the delivery of regional development services by FedNor in Northern Ontario and the CFDCs throughout Ontario has shown that the challenges faced by rural Eastern Ontario are similar to those of other rural areas in Ontario. Eastern Ontario CFDCs have been involved in the identification of the EODF program focus and setting of priorities. As outlined in Table 6, the RMAF for the EODF identifies the socioeconomic challenges faced by rural Eastern Ontario and describe the program elements which address those challenges.

Table 6: EODF Response to Community Challenges
Challenge EODF Response
Reliance on declining resource industries Support for business development and diversification
Out-migration of youth Retention and attraction of youth through youth internship initiative
Insufficient telecommunications capacity Support access to broadband services and the use of ITC technologies
Deficiency in economic infrastructure Support training and skills development and provide access to capital
Insufficient capacity to build partnerships to identify economic development strategies and implement them Support development of community strategic plans

All interviewees agreed that the EODF programming was relevant to the economic development needs of communities in Eastern Ontario. This was generally due to the fact that a grassroots approach was used to initially design the programming. All 15 CFDCs have been involved in community strategic planning and were therefore familiar with the economic development needs of their individual communities.

The five program priorities or components are:

  • Business Development – To enhance the community's capacity for economic development and diversification, and support the growth of existing businesses and the establishment of new businesses by local entrepreneurs, outside investors and potential exporters.
  • Access to Capital – To improve access to capital funds for new and existing businesses in Eastern Ontario.
  • Skills Development – To support the development of a labour force in Eastern Ontario that will meet the needs of the business community, encourage new local entrepreneurs, and attract external investment.
  • Retention and Attraction of Youth – To support initiatives which stem the out-migration of youth from Eastern Ontario and encourage young people to move to Eastern Ontario.
  • Technological Enhancements – To support initiatives which ensure that all residents and businesses in Eastern Ontario have equitable and affordable access to broadband services and enhanced capacity to realize the benefits of information and communications technologies.

Table 7 shows the distribution of applications of approved projects, as of March 31, 2005, indicating that there is a need for all priorities.

Table 7: EODF Applications and Approvals (as of March 31, 2005)

Note: no technological enhancement projects were reported as of March 31, 2005. This is due to the fact that: 1) many technological enhancement projects were, in fact, categorized as community capacity projects because of their size; 2) the program is in its early days; and / or 3) the projects initially funded were shortterm in nature and therefore not conducive to technology enhancements, which generally are longer-term projects.

Priority Applications Received Projects Approved
# % # %
Business Development  
Business Planning 108 9% 50 8%
Local Initiatives 364 30% 161 25%
Community Capacity 119 10% 47 7%
Access to Capital 150 12% 95 15%
Skills Development 301 24% 175 27%
Retention and Attraction of Youth (Youth Internship) 190 15% 115 18%
Total 1,232 100% 643 100%

FedNor management and staff agreed that these priorities were relevant, again because the program had been developed by the 15 Eastern Ontario CFDCs. However, it was noted by some that there could have been more flexibility in terms of the financial distribution across communities, as there was lower demand in some communities for some of the components. Additionally, it was noted that some of the projects were categorized into a particular priority but really would have better fit into another.

CFDC representatives also made similar observations. That is, all agreed that the priorities were very appropriate to the economic development needs of their communities. However, some would have liked more flexibility in terms of how much was budgeted for some of these priorities.

Finally, the stakeholders who were interviewed also all agreed that the priorities were appropriate. Again, some stakeholders believed that some of the priorities were more significant in their communities (for example, retention and attraction of youth was more important than technological enhancements). Nevertheless, all agreed that the needs of their communities fit into those categories; nothing was missing.

The survey of secondary recipients also revealed that the program was well suited to address the needs of these recipients. On a scale of 1 (not at all) to 10 (fully), when asked to what extent the CFDC was able to address their needs through the EODF, 44% indicated fully with another 19% saying 9 out of 10. Overall, the mean was 8.7 out of 10. However, 7% gave it a rating of 5 or less. When asked about the reasons for their rating, the most frequently mentioned positive responses were:

  • we got the money (25%);
  • it was a very good service, very helpful that addressed my needs well (23%);
  • they were supportive (12%); and,
  • they were easy to deal with (11%).

There were few negative responses. The single most frequently mentioned response was that it was not flexible enough (4%).

When asked about the importance of the five priorities of the EODF to support community economic development, the average ratings (out of 10 for extremely important) were as per Figure 1. It is important to note that these high importance ratings are irrespective of the type of funding accessed by the secondary recipient. For example, those who received funding for business development rated access to capital as highly important as those who received funding for access to capital.

Figure 1: Importance of EODF Priorities
Figure 1: Importance of Eastern Ontario Development Fund Priorities
[Description of Figure 1]

3.1.2 Are there any overlaps with other government programs? Is FedNor collaborating with other departments and governments to reduce overlaps?

Other federal regional development initiatives serving Eastern Ontario include Industry Canada's Community Access Program, and the Broadband for Rural and Northern Ontario Pilot Program, as well as the regular CFDC programs. Others include the National Research Council's Industrial Research Assistance Program, and the programs and services of the Business Development Bank. Many of these initiatives have good linkages with the local CFDCs.

The interviewees all agreed that the EODF complemented existing programming within the CFDCs as well as the limited existing programming that was available in Eastern Ontario through other federal or provincial government programs.

Some CFDCs noted that no other programs were available. A few mentioned the Ontario Small Town and Rural (OSTAR) program but noted that this program does not have the money, has too many applicants for the limited money, and that the process to get this money is extremely slow. In terms of complementing existing CFDC programming, it was noted by several that the EODF gave the CFDCs the needed resources to implement the community economic development plans that the CFDCs have worked with their communities to develop.

Stakeholders also all agreed that there was no duplication or overlap, but rather that the EODF complemented existing provincial programs as well as CFDC programming.

In the survey of secondary recipients, 16% indicated that they were aware of programs or services of the federal or provincial government which are comparable or similar to the EODF and available in their region. Those who mentioned programs noted the Trillium Foundation (30% of those aware of others) and Human Resource and Skills Development Canada (28%) the most frequently. They believed that the EODF was much better (22%) or better (34%) than these other programs for its easy of application (22%), ease of access (22%) and faster process (22%).

3.1.3 Conclusions – Relevance

  1. The EODF programming is relevant to the economic development needs of the communities in Eastern Ontario. To a large extent, this is due to the fact that the program was designed by the 15 Eastern Ontario CFDCs based on their familiarity with the socioeconomic challenges faced by the communities they serve.
  2. The five program priorities (business development, access to capital, skills development, retention and attraction of youth, and technological enhancements) are relevant, again due to the fact that the program was designed by the 15 CFDCs. While the specific need for funds across these priorities may have differed across CFDCs, these are legitimate and critical priorities to help Eastern Ontario address its economic development gaps. Additionally, while none of the projects were classified under the technological enhancement priority, this is because of the way some projects were classified and not because it is not an important priority.
  3. The EODF does not duplicate or overlap with other government programming. The EODF is unique in Eastern Ontario.

3.2 Issues – Design and Delivery

3.2.1 Is the FedNor service delivery model appropriate (CFDCs as only recipients)?

There are three aspects related to the service delivery model for the EODF. FedNor signs up to five different contribution agreements with each of the 15 CFDCs. There is a separate contribution agreement for each element of the program. Within the terms and conditions of those agreements, each CFDC can then reach its own agreements with secondary recipients for local projects at various funding levels. The secondary recipients submit their claims, progress and final reports directly to the CFDC.

The first aspect of the service delivery model involves local projects that can be funded through secondary recipients. In these agreements, FedNor / IC provides a contribution to cover a broad range of activities within a specific element (i.e., Youth, Skills Development, Business Development and Access to Capital). Upon approval, the local CFDC then negotiates a contribution agreement with a secondary recipient. FedNor / IC ensures that the CFDC has an appropriate approval and monitoring process in place.

The second aspect of the service delivery model involves Community Capacity projects which are all approved by FedNor / IC, regardless of the funding level. In such cases, once the CFDC has reviewed the project application and deemed it worth pursuing, the application goes to FedNor for review and approval. Even though the local CFDC monitors these larger local projects, FedNor is also closely involved.

Finally, the thirds aspect related to service delivery involves the regional projects. For regional projects, a representative from each of the 15 CFDCs reviews the applications. These are discussed at a meeting of the Eastern Ontario CFDCs and each representative gets one vote. If the majority rules in favour of the project, the application then goes to FedNor for a review and approval process which is comparable to that described for local projects requiring FedNor approval. However, one CFDC takes the lead on administering the project and ensuring proper sharing of progress and results with the other CFDCs.

There is considerable evidence that the CFDCs are an appropriate mechanism to deliver the EODF. The CFDCs are not-for-profit organizations, each governed by a local Board of Directors aware of the specific needs of their community. A recent (2003) evaluation of the CFDCs, demonstrated that they are a relevant, successful and cost effective means to deliver regional economic development programming. Other documents report that CFDCs have "an established rapport with the federal government through FedNor and credibility with the public…".

According to FedNor management and staff, there were two options for the service delivery model: using CFDCs as the primary recipients or having FedNor deal directly with other primary recipients. According to FedNor management and staff, if FedNor dealt directly with other primary recipients (as it does for the Northern Ontario Development Fund), the advantages would include:

  • better control and accountability;
  • the fact that FedNor is more experienced in program delivery of this nature;
  • ability to deal with a broader range of eligible recipients; and,
  • ability to support more projects as well as larger ones.

On the other hand, FedNor management and staff also recognized that there were significant benefits to using the CFDCs as the primary recipients for the EODF. The advantages noted by FedNor management and staff included:

  • CFDCs are the ones who designed the program based on their knowledge of the needs in their communities;
  • the grassroots aspect of the initiative;
  • priorities are established and decisions are made at the local level; and,
  • particularly given the short time frame to implement the initiative, dealing with a network established community organizations who already know how to deal with FedNor was the best way to go (particularly in light of the fact that the history with Community Futures Program is good).

Overall, FedNor management and staff agreed that the blend of CFDC and FedNor involvement for managing and administering the program worked well.

Not surprisingly, the CFDC representatives all agreed that the CFDCs were the most appropriate groups to deal with FedNor for this initiative and that they should remain the primary recipients. Comments made by some of the interviewees in support of this decision include:

  • CFDCs have intimate knowledge of the community, are grassroots, have the infrastructure in place (Community Futures Program) to deliver the EODF;
  • our Board, who knows the needs of the community, is the one that decides who get what and what needs are best served;
  • CFDCs are well positioned, less territorial, politically neutral, and can ensure less duplication and more integration to ensure funding goes forward; and,
  • others may be able to do it, but not as quickly as the CFDCs can in terms of both design and delivery.

All stakeholders agreed that CFDCs did an exceptional job and, as such, are very appropriate primary recipients. Reasons given were similar to those given by CFDC representatives.

The secondary recipients were asked about their overall satisfaction with the EODF. The survey results show that those surveyed were highly satisfied with the EODF. On a 10 point scale (10 being extremely satisfied), the average rating was 8.9; only 4% gave a rating of 5 or less. The key reasons for satisfaction include:

  • the staff was very good (26%);
  • the money is good and useful (23%);
  • it is a generally very good program (22%);
  • the processes were clear and efficient (20%);
  • it is locally driven (15%); and,
  • the projects are successful, have good results (11%).

This provides an indication that CFDCs are efficient as primary recipients in dealing with the secondary recipients.

3.2.2 Is the EODF well promoted? Well known?

The document review addressed this issue only through an examination of CFDC websites. There is considerable variation among the 15 CFDCs in the use of their website as a communication mechanism. The majority (13 of the 15) has information about the EODF on their website. In some cases, this information is not easily accessible, as there is no linkage to EODF on the home page, or the information is very limited. The majority of CFDCs (8 of 15) has a direct link to the EODF information from their home page, and provide extensive information about the EODF as well as application forms. There is the question of the need to match applications with the available funding and not build expectations beyond what the fund can deliver. One of the CFDCs adjusts the level of information about the EODF on its website, reducing the information when funding has been exhausted.

Based on the interviews, there was a wide range of mechanisms used to promote the EODF. These include:

  • websites (CFDC websites and EODF overall website);
  • program announcements;
  • individual press releases;
  • newspaper / radio advertisements;
  • newspaper articles / radio interviews;
  • meetings / presentations / information sessions;
  • mailings;
  • newsletters;
  • word of mouth (Board members);
  • trade shows;
  • e-flyers / emails;
  • through community partners;
  • via Members of Parliament (MPs).

FedNor interviewees generally noted that there were two key challenges associated with the promotion of the program. First, the delays in publicly announcing the program caused delays in CFDCs being able to do their own local promotion. Second, the interest of politicians in being involved in public announcements also caused some delays. The CFDCs also expressed similar concerns. However, they generally agreed that the program was well promoted and that prospective secondary recipients were generally aware of the initiative. It was noted by some that certain promotional mechanisms were less effective than others, but these were more locally driven than of a general nature.

The stakeholders all agreed that a wide range of mechanisms were used and that the promotional efforts of their CFDC were effective.

The secondary recipients who were surveyed were asked how they had become aware of the program. Three methods were most frequently noted: directly through the CFDC (30%); through the newspaper (24%); and through a friend, colleague, word of mouth (10%). The majority of surveyed respondents indicated that they believed that the EODF was very well known (14%) or fairly well known (48%) by the business community and others in their community. However, 25% indicated that the program was not very well known and 6% that it was not well known at all. It is, however, important to recognize that the initiative is still in its infancy and therefore the high level of perceived awareness is encouraging.

The data on enquiries and applications is also indicative of the effectiveness of the promotional efforts in ensuring appropriate awareness of the program. As of March 31, 2005, the CFDCs had received 1,731 enquiries regarding the EODF and 1,232 applications.

3.2.3 Is the EODF project selection process effective? Are the selection criteria appropriate?

Based on a meeting between the 15 CFDCs and FedNor managers, there is no indication that the funding categories need to change or that there are problems with the project selection process, either for smaller projects or for the larger projects that require FedNor approval. The project selection process is depicted in Figure 2.

Different FedNor representatives commented on various aspects of the selection process. Some noted that the local decision-making for the smaller projects was effective since the CFDCs had some experience as a result of the Community Futures Program and they knew their communities and their needs. Others noted that it was important to have FedNor involved in the decision-making for the larger and regional projects, particularly since FedNor officers have a lot more programming experience than the CFDCs and can ensure proper due diligence as well as compliance with the process.

Most CFDCs believed that the local decision-making process was effective. In some CFDCs, the role of staff was more extensive than in others (e.g., rating and prioritization of applications). In some CFDCs, the entire Board reviewed and approved the smaller applications, whereas in other CFDCs, sub-committees were formed, whereas in others still, a combination of the two approaches were used (i.e., full Board for some decisions and a sub-committee of the Board for others). Regardless of the specific process used by the CFDCs, all believed that the process was effective for their community.

Figure 2: Project Selection Process
Figure 2: Project Selection Process
[Description of Figure 2]

* FedNor approval includes review by the CED manager and approval by the Director General or her designated official.

CFDC representatives noted that the efforts made by their volunteer Boards of Directors were significant. The Boards spent additional days discussing and approving the applications, over and above the time spent on their normal CFDC Board meetings. Even with the significant time constraints involved with this initiative, no one disputed the fact that the Boards had all been extremely effective in reviewing and approving projects in a timely fashion.

CFDC representatives also noted that FedNor had done an outstanding job in reviewing and approving the projects that needed FedNor approval. Approvals were made very quickly and few were not approved. The program officers were noted as having played an important role in ensuring that CFDCs understood what needed to be done for effective approval and in also ensuring FedNor understood the importance of the project to the community.

Finally, several CFDC representatives noted that the approval for regional projects was somewhat flawed. Some of the issues included:

  • there is no clear definition of what a regional project consists of (e.g., it is not just a project that spans the regions served by two CFDCs);
  • while each CFDC has an equal vote on projects that are brought forth, there is little negotiation / discussion on the scope or financial aspects of the projects, which is different from what happens at the local level; and,
  • some of the projects that were approved were questionable in terms of their potential regional impacts.

The project selection criteria were identified as:

  • consistency with EODF funding objectives;
  • quality of supporting plans;
  • project feasibility;
  • incrementality;
  • job creation and other lasting community economic benefits;
  • investment leverage; and,
  • track record of partners.

FedNor interviewees noted that these criteria were clear, appropriate and applied, for the most part. It was noted that, for the first round, there were more projects approved with 100% EODF funding than normal, but that this was due to the short time line. CFDCs were now questioning this more when reviewing applications. One FedNor interviewee indicated that there was some difficulty in getting some CFDCs to recognize the intricacies of the terms and conditions. Another interviewee indicated that it is difficult to gauge incrementality (4th project selection criteria) since its definition is not clear.

All interviewed CFDC representatives indicated that the project selection criteria were clear, appropriate and applied. Some noted that they had adapted the criteria to their need while others indicated that they had expanded on the criteria given their experience with, and knowledge of, their community. Some of the reported good practices regarding the project selection criteria include:

  • developed a framework around the selection criteria – added sustainability (for youth to be hired), realistic schedule ("pre-deadline" deadlines), and adjusted others to ensure that the impacts were measurable;
  • built the criteria into the application form which makes assessment of the applications easier; and,
  • developed a scoring model that would rank projects relative to one another for use as a tool by the committee from the Board that was tasked with approving projects; the decisions were therefore based on quantitative as well as qualitative information.

From the perspective of the approved secondary recipients, they were highly or fairly satisfied with:

  • the clarity of the eligibility criteria with a mean rating of 8.4 out of 10; respondents indicated that they were satisfied because the criteria were clearly outlined (53%) or because staff were very helpful throughout the process in ensuring that the required clarifications were given and answering all questions (12%);
  • the fairness of the selection process with a mean rating of 8.9; respondents indicated that they were satisfied because they were successful in getting the funding (35%), because it seemed fair and transparent (21%) and / or because a wide range of projects and organizations were approved, there was no discrimination (11%);
  • their understanding of the decision-making process (7.7 out of 10); and,
  • the speed of decision-making (8.4).

3.2.4 Are EODF projects relevant? Do they meet the program criteria?

According to FedNor interviewees, the projects that have been approved to date are quite good. Interviewees noted that this was the first real injection of funds of this kind in these communities and that the EODF resulted in an ability to advance priorities in tourism, agriculture, technology and other areas. According to staff, some projects have had significant results, even in the short-term and others will have lasting impacts.

Regarding meeting the program criteria, the reader is referred to the previous section (3.2.3).

CFDC representatives generally agreed that the local projects were very good and relevant. The program was deemed to be a "godsend to get things done that the community knew needed to be done but could not get done without additional financial assistance". Several interviewees indicated that they received a lot more applications than they could fund, many of the rejected ones were also excellent, but that, overall, the ones that were approved were very relevant and to a large extent, the best ones.

There were mixed feelings regarding the regional projects. Some indicated that the regional projects were, in some cases, weak. Others believed that some of the regional projects were very interesting and strategic. Some also noted that a few of the regional projects were more localized.

The stakeholders indicated that, based on their familiarity with the projects, they were generally very satisfied with the projects that had been approved. They believed them to be very relevant.

The case studies revealed that the projects were very relevant to specific needs, for example:

  • The need to make broadband accessible to the businesses and residents of the Town of Georgina to ensure that people had options for work, education, health, social and other government services, etc.
  • Several organizations in the Northumberland area could not be as productive, competitive or profitable without specific changes that required specific training (e.g., LEAN manufacturing, web design, etc.).

3.2.5 Are project clients satisfied with counselling, advice, training or workshops provided?

There is limited information in the documents reviewed about client satisfaction. There is evidence of strong demand, in particular for the youth internship and local economic development initiatives elements.

Most CFDCs interviewees indicated that counselling, advice, training and workshops were provided by their CFDC via their other programming, not via the EODF. Nevertheless, where comments were made on this issue, they generally believed that clients were satisfied. Some viewed this in the context of the training provided by the funded organizations through the skills development component. In such cases, interviewees noted that the people who had received training were satisfied.

The secondary recipients who were surveyed were specifically asked about their satisfaction with counselling, advice, training and workshops. While most indicated that training and workshops were not applicable, those providing responses were very satisfied with all these elements, as shown in Table 8.

Table 8: Satisfaction by Program Component (Mean rating out of 10)
Element Business Development Access to Capital Skills Development Attraction and Retention of Youth Total
Counselling 8.8 9.3 8.8 9.2 8.9
Advice 8.9 9.1 8.8 9.2 8.9
Training 9.1 9 9.6 8.4 9.2
Workshops 9.1 10 8.9 8.3 8.9
Note 1:
Some means are based on a small number of cases, particularly for training and workshops.
Note 2:
Separate results are not presented for Technological Enhancements because only one interview was completed; this interview is, however, included in the totals.

3.2.6 Is FedNor playing an appropriate role in support of the projects?

Documents show that CFDCs have local control over most program elements, but that FedNor approval is required for individual projects over $10,000. Minutes of an EODF meeting with representatives from the CFDCs and FedNor show that this level of control has general approval.

FedNor managers noted that they thought staff had done an amazing job at rolling out the program as quickly as they did and providing the required support to CFDCs. It was noted that the role of staff was important, in particular with due diligence, because of the lack of experience of CFDCs in program delivery of this nature.

FedNor staff noted that they thought the support they provided was important. However, some noted that they would have liked to have more direct involvement in the projects (and with the secondary recipients), rather than just support for those projects that required FedNor approval (and with only the CFDCs).

The CFDC representatives indicated that the role of FedNor in support of the projects included:

  • support related to the announcements;
  • documentation for press releases;
  • administrative / communication support;
  • ensure due diligence is being met;
  • excellent advice / perspective / expertise;
  • contractual;
  • support related to meeting the guidelines; and,
  • physical support (attendance at meetings, there for press conferences, etc.).

In general, the CFDCs were very satisfied with the support provided by FedNor. They believed that the support provided was appropriate and sufficient. Several commented on the outstanding job that FedNor had done, particularly given the time frame and its limited human resources. Nevertheless, a few would have liked more independence in administering the program and would have preferred it if FedNor had not been involved quite as much.

3.2.7 Conclusions – Design and Delivery

  1. The service delivery model for the EODF is appropriate given the program objectives. While the CFDCs need to ensure that they involve other important economic development groups, the network of CFDCs is an excellent delivery mechanism. This is due to the fact that CFDCs have an existing relationship with FedNor, are recognized players in their communities at the economic development table, and provide complementary services through their existing programming. Additionally, CFDCs are unique in their regional rather than local perspectives.
  2. Even though the program was implemented within a very short time frame and is in its infancy, the CFDCs used a wide range of effective means to promote it to ensure that it is well known. While there is still an ongoing need to continue to promote the initiative and make more organizations aware of its availability, the CFDCs received more applications than they could approve within their budgets.
  3. The project selection process for small local initiatives is effective. While the exact process differs from one CFDC to the next, each was able to receive and review a large number of applications within a very short time frame and approve projects that were successful.
  4. The project selection process for larger local initiatives where FedNor needed to be involved in the approval process is also effective. The initial review and recommendations of the CFDCs was effective in that few recommended projects were not approved by FedNor. Additionally, FedNor was very effective in reviewing and approving these applications.
  5. The project selection process for regional projects is not as effective as it could be. The definition of regional projects is unclear. However, the process involving all CFDCs in the review and recommendation of projects is appropriate, as are the role of FedNor in approving the applications and the consultations with other regional economic development stakeholders.
  6. The project selection criteria are clear, appropriate and applied. The relevance of the projects selected (as demonstrated by the appropriateness of the early results) are indicative of the fact that the criteria are applied and appropriate given the program objectives.
  7. The role of FedNor in support of the projects is important given its experience with other programming and with similar types of projects in Northern Ontario. This support is appropriate and sufficient.

3.3 Issues – Success

Some of the findings related to the success of the program are more general in nature. Overall, the FedNor interviewees believed that the EODF has been successful in achieving its intended outcomes.

CFDC representatives also believed that the program has been successful, even in its early days. It is expected that, over time, the impacts will be even more significant. It was noted that, while the impacts or results have been significant, there are still major gaps and an ongoing need for EODF funding in the Eastern Ontario communities.

The survey of secondary recipients revealed that the respondents believed their projects to have been very successful (overall mean of 9.0 out of 10). This was consistent across program components (i.e., 9.2 for Access to Capital and Retention and Attraction of Youth; 9.0 for Skills Development; and 8.8 for Business Development).

3.3.1 To what degree has the EODF contributed to the development and retention of new and existing businesses? To what degree had the EODF improved competitiveness of rural Eastern Ontario firms?

It is too early to truly report on this issue, but there is early indication that the program will make an impact in this regard. For example, from the case studies the following potential results were identified:

  • for the Northumberland CFDC Skills Development Initiative, several organizations noted that the skills gained will help their organization be more competitive in the future;
  • for the Stormont, Dundas and Glengarry CFDC's Local Initiatives and Business Planning, some organizations are already reporting an increase in business whereas others indicate that their project results will help attract new business and maintain existing ones;
  • for the South Lake CFDC's broadband project, it is expected that the region will be able to attract new businesses; in fact, four different businesses have already indicated that they wanted to move to the area but that they needed broadband access; this could result in 200 jobs in the region.

The survey of secondary recipients asked respondents a series of questions regarding this. The results are provided in Table 9.

Table 9: Project Results
Result Has occurred Will occur Will / may not occur / did not expect
Development of a new business 13% 12% 75%
Retention of an existing business 43% 8% 49%
Improved competitiveness 55% 20% 25%

3.3.2 To what degree has the EODF helped attract, retain and develop human capital, including special client groups? (E.g., women, Francophones, First Nations and other Aboriginals, youth)

One of the mechanisms for the attraction, retention and development of human capital has been the Youth Internship element. During the first year of operation, over 100 interns were hired, including disabled persons. In addition, one project of the Valley Heartland CFDC involved identifying those individuals most at risk from the upcoming closing of the Rideau Regional Centre in Smiths Falls and their need for retraining. In this way, the study contributed to a plan for the redevelopment of human capital at risk.

All case studies noted the contribution of the projects to this outcome, as explained below:

  • from the 1000 Islands Youth Internship Initiative, several interns have already found full-time employment either in the funded organization or in another organization in the region;
  • from the Northumberland Skills Development Initiative, human capital was developed in all projects through the training;
  • for the Stormont, Dundas and Glengarry Local Initiatives and Business Planning, some organizations reported that they were able to attract people to their organizations whereas others reported that they had developed business or trade skills;
  • for the South Lake CFDC's broadband project, if the region can attract new businesses, this could result in 200 jobs in the region; and,
  • for the Frontenac Electronic Waste Recovery Facility Pilot Project, students have been trained and it is expected that the project could result in reduced youth out-migration and increased employment in the region.

The survey of secondary recipients revealed the results outlined in Table 10.

Table 10: Project Results
Result Has occurred Will occur Will / may not occur / did not expect
Attraction of people to your organization or region 52% 19% 29%
Retention of people within your organization or region 58% 11% 31%
Development of people within your organization 65% 9% 26%

Of those who indicated that the project had resulted in the attraction, retention or development of people, 9% indicated that the project targeted women, 4% indicated that it targeted Francophones, 1% said it targeted First Nations and other Aboriginals, and 35% noted that it targeted youth.

3.3.3 To what degree has EODF contributed to the economic development of rural Eastern Ontario communities?

Reports show that several EODF projects under the Community Capacity Building element have helped develop community infrastructure, and strategic plans to support economic development. For example, one project supported development of the Rideau Canal Corridor as a tourism initiative. Specific elements included improving the dockage on the Rideau Canal at Smiths Falls, providing a business and economic development plan for Perth and a feasibility study for a World Heritage Interpretive Centre in Merrickville. Another project involved the development of options for the future use of the Rideau Regional Centre, a provincially operated centre for the mentally disabled slated for closure in 2008–2009.

All case studies involved actual or potential results which contribute to economic development. For example:

  • the Frontenac e-waste facility will eventually be in a position to be self-sustaining through the revenues made by sending materials to recyclers; as it becomes self-sustaining, it can increase its employment and thus contribute to the economic development of the community;
  • some of the local initiatives in Stormont, Dundas and Glengarry will contribute to economic development through improved planning, increased tourism, increase in business, and others.

In the survey, 44% of secondary recipients indicated that the project has results in economic development, whereas another 27% indicated that it will in the future.

3.3.4 To what degree has EODF helped develop business and trade skills?

There is little information in the documents reviewed about the number of people receiving training through this program. However, one CFDC reported that in the first year, there were 10 projects receiving EODF funding involving advice and training to over 250 participants. According to the report, at least 215 of those trained have continuing employment.

Examples from the case studies include:

  • for the Stormont, Dundas and Glengarry Local Initiatives and Business Planning, some organizations reported that they were able to attract people to their organizations whereas others reported that they had developed business or trade skills;
  • for the Frontenac e-waste facility, students have gained important information technology skills;
  • for the Northumberland skills development case, all projects involved the development of manufacturing skills, software-related skills and / or internet-related skills;
  • project management skills were gained in the South Lake broadband project; and,
  • the interns in the 1000 Island youth internship case, all gained business and / or trade skills, many of which they will be able to apply to other employment.

The secondary recipients who were surveyed also indicated that their projects had resulted in business or trade skills, as per Table 11.

Table 11: Project Results
Result Has occurred Will occur Will / may not occur / did not expect
Development of business skills 58% 8% 34%
Development of trade skills 42% 6% 52%

3.3.5 To what degree has EODF helped develop external markets for rural Eastern Ontario firms?

There is limited information available on this issue. From the case studies, of the ten business and trade skills related projects funded by the Northumberland CFDC, five involved training that contributed to the development of external markets for the firms involved.

3.3.6 To what degree has EODF increased the use of technology? Has contributed to the development of innovation?

A Valley Heartland project involved the design, development and testing of e-learning multi-media programs to assist Autism Spectrum Disorder impaired children, as an alternative to live tutors. Immediate outcomes include employment and development of expertise in autism in individuals from the local arts and technical communities, and development of partnerships with provincial and other research groups, as well as public education and awareness of autism. This project has laid the foundation for further development of the program and establishment of the region as a centre of technical expertise and innovation in this field.

Of those firms participating in the Northumberland CFDC funded skills development initiatives, six used the training as a strategy to help the firm adopt new technology to improve their competitive position.

The South Shore broadband case is also directly related to technology, although it is too early to report on actual impacts.

The survey of secondary recipients shows that the EODF has contributed to these outcomes as per Table 12.

Table 12: Project Results
Result Has occurred Will occur Will / may not occur / did not expect
Increased use of technology 44% 5% 51%
Development of an innovation 32% 10% 58%

3.3.7 What is the incremental impact of EODF projects?

The survey of secondary recipients shows that the EODF has a very high incremental impact on the projects. When asked what impact the absence of EODF funding would have had on the project, 76% reported that it would have had a major negative impact on the project, and another 21% indicated that it would have had a minor negative impact on the project. Respondents were then asked what, specifically would have happened. Responses included:

  • would not have gone ahead with the project (37%);
  • it would have taken more time to do the project or the project would have been delayed (19%);
  • it would have had an impact on the training (13%); and / or
  • the scope or quality of the project would have changed (10%).

Based on these responses, an incrementality assessment was made. Full incrementality included those who indicated that they would not have gone ahead with the project and that this would have been major; high incrementality consists of all others who indicated a major negative impact but who would still have proceeded with the project; minor incrementality consists of those who said that the absence of funding would have had a minor negative impact on the project; and no incrementality consists of those who said that the absence of EODF funding would not have had any impact on the project. Incrementality is depicted in Figure 3.

Figure 3: Incrementality
Figure 3: Incrementality
[Description of Figure 3]

Figure 3 shows that incrementality is fairly high. However, full incrementality is significantly lower for those who received funding under the skills development elements. Major incrementality is lower for those who received funding under the access to capital element.

The case studies also show that the EODF has a high incremental impact on the project. In all case studies, recipients either noted that they would not have been able to proceed with the project or that the projects would have been negatively affected.

An additional incremental impact of the EODF has been on the CFDCs themselves. As reported in the interviews with both FedNor and CFDCs, the program has provided the CFDCs with needed money to become a more active player at the economic development table in their communities. Before the EODF, they contributed to Community Economic Development plans, but could not provide the money to help the plans be implemented. Now they can. As a result, the CFDCs have benefitted from the program in some of the following ways:

  • developed new relationships or partnerships;
  • enhanced role in the community; and,
  • positive impact on Board members (even with the added workload).

However, the program has also had a negative impact on the CFDCs in that this had added significantly to their workload. While some CFDCs added a part-time or full-time staff to assist, others worked with their existing staff complement.

3.3.8 Conclusions – Success

  1. The EODF has been successful, even in its infancy, in achieving its intended immediate and intermediate outcomes. Even though many projects were still ongoing at the time of the evaluation, all projects have resulted in at least one of the program's intended outcomes.
  2. The EODF is helping organizations undertake projects that they could not otherwise undertake or would have to complete differently (lower quality, smaller scope, delayed, etc.). The program is therefore highly incremental. As such, the success of the projects is highly attributable to the program.

3.4 Issues – Monitoring and Accountability

3.4.1 Is reporting timely, reliable and complete? Are reports providing FedNor useful information for management purposes?

Reports are provided by the CFDCs to FedNor only for individual projects with over $10,000 in funding. For other smaller projects, the CFDCs provide a roll-up summary report for each of their contribution agreements. At the time of this evaluation, a limited number of final reports were available. The final reports examined all provided useful information about the project outputs and outcomes.

A database consolidating the information from the progress reports provided by the CFDCs was also provided. A review of the information in this database found that the information was not reliable and was incomplete. Examples of problems with the data include:

  • In some cases, information that should have been completed was not included in the database. It is unclear if this is because the response should have been zero (in the case of numeric requirements) or if the information was not provided. For example, 16 of the reports indicated that there were agreements with secondary recipients; however, only 15 of the reports provided a value for the agreements with secondary recipients. There are many other examples of this nature.
  • In some cases, the numbers provided are possibly wrong. This perception is due to the fact that some numbers are either extremely large or extremely small. For example, a $238 agreement appears very small for this type of program, particularly when most of the others are over one thousand dollars.

FedNor interviewees noted that reporting requirements from the CFDCs were fairly timely and useful. It was noted that when requests for interim or final reports were made, the CFDCs were very responsive. One interviewee mentioned that it will be critical for CFDCs to track the success of projects over the long term in order to document longer term impacts; since some of the projects are developmental in nature, results will not be apparent for some time.

Not surprisingly, all interviewed CFDCs stated that they were reporting progress in a timely fashion. Some of the good practices uncovered during the interviews include:

  • one CFDC has imposed reporting deadlines on its secondary recipients to ensure that it has the required information to put together its own summary report;
  • one CFDC invested in creating its own template to be able to easily roll up the secondary recipient progress reports into the report required by FedNor.

Several CFDC interviewees noted that it would be useful to get The Exceptional Assistant (TEA) grant module. This would greatly speed up the reporting process as well as make reporting more consistent with other CFDC reporting requirements to FedNor. In fact, one CFDC reported that it will order it, once it is available in French.

Several CFDC interviewees also commented on the inconsistency of the reporting structure with the program structure. The report is structured according to five sections:

  • technology infrastructure and innovation;
  • community capacity and response;
  • business and trade skills;
  • human capital and special client groups;
  • access to capital and leveraged capital.

However, the program is also comprised of five elements or priorities:

  • business development;
  • access to capital;
  • skills development;
  • retention and attraction of youth; and,
  • technological enhancement.

However, the report structure does not necessarily match with the program structure. The challenges outlined in Table 13 are based on the possible yes answers to the screening question in each section of the report outline.

Table 13: Program Structure Compared to Program Reporting Structure
Program Structure Reporting Structure
Technology Infrastructure and Innovation Community Capacity and Response Business and Trade Skills Human Capital and Special Client Groups Access to Capital and Leveraged Capital
Business Development
Checkmark
Checkmark
   
Checkmark
Access to Capital  
Checkmark
   
Checkmark
Skills Development
Checkmark
Checkmark
Checkmark
Checkmark
Checkmark
Retention and Attraction of Youth
Checkmark
Checkmark
Checkmark
Checkmark
Checkmark
Technological Enhancement
Checkmark
Checkmark
   
Checkmark

Even with these challenges, most CFDCs indicated that they had the tools needed to complete reliable and accurate reports, even though it was time consuming.

3.4.2 Is the reporting system allowing FedNor to meet accountability requirements?

The RMAF identifies a series of indicators which are key for monitoring purposes. Table 14 provides an assessment of the ability of the Project Interim Reports to address the indicators.

Table 14: Key RMAF Indicators Versus Reporting Requirements
Key Indicators Reporting Requirements
Number of businesses adopting new technologies This information is requested in Section 1 – Technology Infrastructure & Innovation and Section 5 – Access to Capital and Leverage Capital; the problem lies with the fact that the same project could be reported in both sections and there is no way to identify this.
Number and type of innovations produced This is asked directly in S.1; ok
Number of training and workshop participants This is asked directly in several sections; again, the same project could be reported several times.
Number of clients receiving counselling and advice Asked in S.2 and S.3; could be duplicate counting
Number of projects contributing to the development of business and trade skills Screening question to S.3; ok
Percentage of clients satisfied with counselling and training Not reported
Percentage of training participants expecting to use counselling/training content Not reported
Number and range of CFDC business loans S.5 collects the # and value but not the range
Number of businesses created Asked in all sections; could be duplicate counting
Number of projects contributing to the development of existing businesses Not reported
Number of projects contributing to the development of external markets Asked in all sections; could be duplicate counting
Number (and trends) of jobs created and maintained Asked in all sections; could be duplicate counting
Number of community economic development projects S.2; not clear if directly aligned
Number of projects where the community received a recognizable socioeconomic benefit from the project Not reported

The key problem appears to be due to the fact that CFDCs are completing the forms to summarize all projects funded to secondary recipients, whereas the report form is designed for one project (which is how the contract is initiated between FedNor and the CFDCs).

FedNor interviewees noted that it was still fairly early to assess if the Results-based Management and Accountability Framework (RMAF) and the Risk Based Audit Framework (RBAF) accountability commitments were met. The audit requirements are to be addressed in the Fall and there were too few projects with final project reports to assess their compliance with performance reporting requirements.

It was noted that the problems associated with the departmental contribution database (Contribution Management Information System – CMIS) which have been identified regarding all other FedNor initiatives are applicable to the EODF. A useful and adequate performance measurement system is still a work in progress for FedNor overall.

3.4.3 Conclusions – Monitoring and Accountability

  1. FedNor has made good progress in implementing the performance measurement strategy outlined in its Results-based Management and Accountability Framework (RMAF). In addition, the CFDCs have been very diligent in respecting their commitments regarding progress reporting. As such, they have taken the steps required to ensure that they have the information they need to meet these commitments. However, some of the reporting requirements are not clear. Additionally, it is not evident that all RMAF commitments are met through the existing reporting structure. Performance measurement is still a work-in-progress.

3.5 Issues – Alternatives and Cost-Effectiveness

3.5.1 Are there alternative service delivery approaches? Are there alternatives to increase the cost-effectiveness of EODF programming?

Based on evidence provided throughout this report and additional comments made by interviewees, the study findings regarding alternatives indicate that there are three options for service delivery of a program with the EODF objectives. These options are:

  • Option 1 – full control and accountability given to the network of Eastern Ontario CFDCs: Some CFDC interviewees noted that they would like to be able to make decisions on all EODF local initiatives, regardless of the size of the project and that the need to go through FedNor for larger local initiatives as well as regional ones only added steps to the process and could result in a longer project approval process. However, several CFDC interviewees recognized the value added by FedNor staff and management and did not consider this as an optimal option.
  • Option 2 – continued blend between CFDC and FedNor decision-making and accountability: The majority of CFDC and FedNor interviewees indicated that the blend of CFDC decision-making for small local initiatives and FedNor approval for larger local initiatives and for regional projects was probably the best option. CFDCs recognized the added value of FedNor's involvement; everyone recognized the importance of local decision-making to best address local priorities. However, the majority of interviewees believed that the overall service delivery approach for regional projects needs improvement.
  • Option 3 – FedNor has full control and accountability for administering the program: Some FedNor interviewees indicated that an approach which is similar to the Northern Ontario Development Fund where FedNor deals with a wide range of primary recipients (as opposed to just the CFDCs). However, other FedNor interviewees, all CFDC interviewees and the stakeholders were of the opinion that such an option would be less than optimal for several reasons. Interviewees noted that the presence of CFDCs in their communities was an important delivery component which would be lost under this option; that FedNor could not deliver a grassroots program of this nature in Eastern Ontario; that the experience so far showed that CFDCs could successfully deliver a program of this nature, so why change it; etc.

In terms of cost-effectiveness, there are a few key points to highlight:

  • The FedNor operational component of this initiative was approximately $400,000 for the first round of projects and $250,000 for the second round. FedNor representatives who were interviewed indicated that this was very little. On the other hand, some of the CFDC interviewees believed that, in the context of their operational costs and tasks, it was too much. The FedNor component represents less than 4% of the $18.2 million total for this initiative.
  • All but one CFDC indicated that there was no way their costs could have been reduced. In fact, some stated that they did not necessarily charge all administrative expenses associated with the EODF so that they could maximize the dollars that were made available for projects in the community. The CFDCs' budget for administrative expenses represent approximately 6% or 7% of the total EODF $18.2 million allocation.
  • A lot of the work done at the CFDC level is done by the volunteer Boards of Directors. Some of the Boards added one or two days of meetings to make EODF decisions. This could represent as much as 180 days of work for each application deadline. There is no cost to the program for this work.
  • If the program is continued, some of the CFDCs will need to increase their operational costs. Several CFDCs indicated that their staff had been pushed to the limit for this program and this was unlikely to be able to continue on an ongoing basis.

Other interesting findings related to cost-effectiveness are highlighted in Table 15 which can be summarized as follows:

  • The program funding encouraged the financial contributions of other partners on the project. As of March 31st, 2005, for close to $9 million in approved EODF funding, an additional $28.6 million was leveraged.1 The program therefore funded less than 25% of the total project costs, yet was highly incremental to the organizations' ability to undertake the projects.
  • As of March 31st, 2005, 643 projects were approved for the close to $9 million in approved funds. This represents average EODF projects costs of less than $14,000. As reported previously, these projects are already reporting significant economic development outcomes and impacts (see Success summary).

3.5.2 Conclusions – Alternatives and Cost-Effectiveness

  1. There are no alternative service delivery approaches for the EODF that would produce the same results at a lower cost. Currently, the administrative portion of this initiative is reasonable, the CFDCs rely on the large contribution of its volunteer Boards of Directors, a significant proportion of the projects costs are leveraged through other sources, and the projects are contributing to the achievement of the intended outcomes. The program is therefore cost-effective.
Table 15: EODF Summary Report (as of March 31, 2005)
Priority # of Projects Approved EODF Funding Other Contributions Total $ Average Total Project $ Average EODF Project $
$ % $ %
Business Development  
Business Planning 50 $264,258 29% $634,644 71% $897,902 $17,958 $5,285
Local Initiatives 161 $1,518,665 30% $3,542,287 70% $5,060,952 $31,434 $9,433
Community Capacity 47 $1,701,863 19% $7,180,788 81% $8,882,651 $188,993 $36,210
Access to Capital 95 $2,177,960 14% $12,993,173 86% $15,171,133 $159,696 $22,926
Skills Development 175 $1,899,923 34% $3,728,225 66% $5,628,148 $32,161 $10,857
Retention and Attraction of Youth (Youth Internship) 115 $1,407,679 71% $566,373 29% $1,974,052 $17,166 $12,241
Total 643 $8,970,348 24% $28,645,490 76% $37,614,838 $58,499 $13,951

1 While the additional funds are not fully attributable to the program (in light of the fact that some recipients may still have proceeded with the project, albeit differently), they are highly attributable to the EODF because of the high incrementality as well as the fact that many CFDCs did not fund 100% of the project costs, and therefore required leveraged funds before they would approve projects. (Return to reference 1)

3.6 Issues – Lessons Learned

3.6.1 What lessons have been learned from EODF programming for the future?

Most of the reported lessons learned (through the interviews, survey and case studies) were more suggestions for improvement.

Comments made by FedNor interviewees included:

  • need to link up the performance measurement to coincide with the description of categories so people will understand at their community level;
  • good up front communications is key – as much as FedNor tried, there were misunderstandings during the process;
  • it is important to time the program announcements – these were the cause of many glitches, particularly in terms of the very short time frame allocated to implement and deliver the program;
  • one contract with each CFDC rather than five would provide more local flexibility to better meet the needs of the communities; this could still be done with minimum requirements; it would also reduce the number of contracts to administer;
  • local stakeholders need to drive the process; the grassroots approach is critical to the success of the program; and,
  • FedNor's role is important and needed; there is a need for CFDCs to recognize that there is a cost for this.

CFDCs made the following comments:

  • the guidelines are set up for larger projects and do not provide much flexibility for very small project (one CFDC reported that it had gone through the same process for a $67 contribution to assist a small business representative get needed training);
  • flexibility within and across programming elements would allow communities to adapt or expand without stepping outside parameters;
  • good communication throughout the process is critical (FedNor and CFDCs);
  • faster payment of claims is important; some of the secondary recipients are not-for-profit organizations that do not have the cash flow to wait five or six weeks for payment;
  • time frames should be more reasonable;
  • keep as many decisions local as possible;

Stakeholders reiterated comments related to the short time frame and communication issues.

The secondary recipients identified a wide range of lessons learned. The key ones were:

  • they gained an increased awareness of how government programming works (20%)
  • they learned how to use or access resources or money (14%);
  • they learned how to fill out applications and other aspects related to paperwork (13%); and,
  • they learned that they wanted to continue to keep in touch with the CFDC, to use them again in the future (10%).

The case studies revealed some lessons learned including:

  • community support is critical and there needs to be proper time allocated to consult with the community in some cases (e.g., South Lake broadband project);
  • the possibilities of delays due to technical difficulties and lack of supplier readiness are significant in certain types of projects (e.g., broadband). There is therefore a need to be able to come up with innovative solutions to technical problems and to ensure ongoing communications with suppliers; and,
  • for some projects, it is difficult to attribute the actual and potential impacts to EODF funding, particularly when the project is one component of several others (e.g. Frontenac e-waste facility).

3.6.2 What factors have facilitated and / or impeded implementation of EODF?

Table 16 highlights some of the key factors that facilitated implementation of EODF and Table 17 highlights some of the key factors that impeded implementation, as identified by the various groups consulted.

Table 16: Factors that Facilitated EODF Implementation
FedNor interviewees
  • energy and enthusiasm of management and staff
  • existing relationship with CFDCs as a network
  • expertise of FedNor staff
  • the 15 CFDCs were fairly cohesive as a group
  • grassroots aspect of program design and implementation
  • the regional coordinator (Dan Borowec)
  • ongoing consultations / communications
  • FedNor had a small team working on this
  • strong political support from local MP and Minister's Office
  • willingness of FedNor to provide support beyond Northern Ontario boundaries
  • the collective experience of FedNor and the CFDCs
CFDC interviewees
  • the 15 CFDCs knew what was needed / cohesiveness / cooperation
  • same amount allocated to each CFDC (no competing for the funds)
  • existing relationship with FedNor
  • hiring of coordinators in some CFDCs
  • extensive work / engagement of Boards (volunteers)
  • FedNor flexibility / flexibility within our community to shape to our needs
  • senior FedNor representatives added credibility and made a huge difference
  • continuous intake until funding was fully subscribed in order to avoid tying up the funds too early (one CFDC)
Other stakeholders
  • CFDC staff
  • CFDCs and FedNor work well as a team
  • 15 CFDCs worked well together / collaboration and cooperation
Secondary recipients (project related)
  • availability of much need money in the community (42%)
  • CFDC support (20%)
  • support of local businesses and of the community (19%)
  • good training / good quality trainers (14%)
  • good intern / able to attract good staff (12%)
Table 17: Factors that Impeded EODF Implementation
FedNor interviewees
  • delays in getting approvals and very short time frame to implement
  • not enough flexibility for some CFDCs in terms of the allocation of funds
  • guidelines for regional projects not clear
  • undue expectations of CFDCs in terms of what they could and could not do with the money
  • slow payment process
  • need to reprofile the fund
CFDC interviewees
  • delays in announcements
  • had to turn things around really quickly
  • slow payment process
  • administrative burden
  • definitions and approval process for regional projects
  • building awareness of program, particularly in short time
  • reluctance of private sector to participate
  • political climate (minority government)
  • time to get translations
Other stakeholders
  • short time frame
  • bureaucracy
  • lack of CFDC staff
Secondary recipients (project related)
  • short time frame (17%)
  • not enough money (10%)

3.6.3 Conclusions – Lessons Learned

  1. The relationships among all players involved in this initiative was key to the success of the EODF. These included the relationship: 1) across the 15 CFDCs who worked extremely well in reaching consensus on the design and delivery of this initiative; 2) between the CFDCs and FedNor where the pre-existing relationship helped to contribute to the smooth implementation of the initiative and the commitment of both parties to the initiative helped make it a success; 3) within the CFDCs among staff and with the Boards where staff demonstrated a high commitment to make this initiative work and the Board members worked with staff to ensure that the project applications were reviewed and approved in a timely fashion; and 4) between the CFDCs and the community where the fact that the CFDC was a recognized credible economic development organization which knew the community and vice versa made it easier to roll out the program within a very short time frame.
  2. Flexibility was also key to the success of this initiative, yet ongoing and increased flexibility may help ensure the continued success of the initiative. The program was set up by the CFDCs to provide each CFDC the same amount for each of the program elements. However, enough flexibility was incorporated into the guidelines so that the CFDCs were able to redistribute some of their budgets across components in order to better meet the needs of their communities. While some CFDCs would have liked more flexibility right from the start, the combination of guidelines and flexibility within those guidelines helped ensure that the intended program results were achieved while ensuring that community needs were considered. Nevertheless, increased future flexibility, within set parameters, may contribute to a more effective decision-making process and ensure that the needs of the communities are better met. Additionally, more flexibility regarding bureaucratic requirements for very small projects could help make the program more cost-effective.
  3. In order to ensure good ongoing performance reports, it is key to make sure that the performance measurement system is reflective of the program and that performance reporting requirements are clearly defined. The RMAF for the EODF was developed by FedNor. Because of extremely tight deadlines, CFDCs were not directly involved in the development of this RMAF. While FedNor did attempt to consult with some CFDC representatives, these consultations were limited and most CFDCs were not familiar with the document. This presented challenges in that CFDCs did not fully understand (or agree with) the program logic model and performance indicators. As such, they did not understand some of the reporting requirements. Additionally, some of the reporting requirements were not clear and were therefore interpreted differently from one CFDC to the next. Finally, the key indicators identified in the RMAF were not fully integrated into the performance reporting requirements.

4.0 Conclusions and Recommendations

The conclusions presented throughout this report are reiterated in Table 18 with their ensuing recommendations.

Table 18: Conclusions and Recommendations
Conclusions Recommendations
4.1 Relevance

Conclusion 1

The EODF programming is relevant to the economic development needs of the communities in Eastern Ontario. To a large extent, this is due to the fact that the program was designed by the 15 Eastern Ontario CFDCs based on their familiarity with the socioeconomic challenges faced by the communities they serve.

Recommendation 1

The EODF should be renewed.

Note: the balance of the recommendations are based on the assumption that the EODF will be renewed.

Conclusion 2

The five program priorities (business development, access to capital, skills development, retention and attraction of youth, and technological enhancements) are relevant, again due to the fact that the program was designed by the 15 CFDCs. While the specific need for funds across these priorities may have differed across CFDCs, these are legitimate and critical priorities to help Eastern Ontario address its economic development gaps. Additionally, while none of the projects were classified under the technological enhancement priority this is because of the way some projects were classified and not because it is not an important priority.

Recommendation 2

The five program priorities should continue to form the basis of EODF programming. These encompass current priority needs and are broad enough to address evolving economic development needs.

Conclusion 3

The EODF does not duplicate or overlap with other government programming. The EODF is unique in Eastern Ontario.

See Recommendation 1
top of page
Table 18: Conclusions and Recommendations (continued)
Conclusions Recommendations
4.2 Design and Delivery

Conclusion 4

The service delivery model for the EODF is appropriate given the program objectives. While the CFDCs need to ensure that they involve other important economic development groups, the network of CFDCs is an excellent delivery mechanism. This is due to the fact that CFDCs have an existing relationship with FedNor, are recognized players in their communities at the economic development table, and provide complementary services through their existing programming. Additionally, CFDCs are unique in their regional rather than local perspectives.

Recommendation 3

The EODF should continue to use the existing delivery model which involves FedNor contribution agreements with each of the 15 CFDCs who, in turn, provide funding to secondary recipients.

Conclusion 5

Even though the program was implemented within a very short time frame and is in its infancy, the CFDCs used a wide range of effective means to promote it to ensure that it is well known. While there is still an ongoing need to continue to promote the initiative and make more organizations aware of its availability, the CFDCs received more applications than they could approve within their budgets.

Recommendation 4

The CFDCs should continue to use a multitude of promotional tools to ensure that their communities are appropriately aware of the EODF. However, promotional strategies should consider the limited availability of EODF funds.

Recommendation 5

While the promotional efforts of the CFDCs have been fruitful, a process should be implemented to ensure that promotional materials are shared and to thus avoid duplication of efforts and build on best practices.

Conclusion 6

The project selection process for small local initiatives is effective. While the exact process differs from one CFDC to the next, each was able to receive and review a large number of applications within a very short time frame and approve projects that were successful.

Recommendation 6

The current project selection process for small local initiatives does not need to change. However, since CFDCs have implemented different processes for the review and selection of projects, the different processes should be shared among CFDCs to help in the sharing of best practices and lessons learned.

Conclusion 7

The project selection process for larger local initiatives where FedNor needed to be involved in the approval process is also effective. The initial review and recommendations of the CFDCs was effective in that few recommended projects were not approved by FedNor. Additionally, FedNor was very effective in reviewing and approving these applications.

See Recommendation 6, but for larger local initiatives.

Conclusion 8

The project selection process for regional projects is not as effective as it could be. The definition of regional projects is unclear. However, the process involving all CFDCs in the review and recommendation of projects is appropriate, as are the role of FedNor in approving the applications and the consultations with other regional economic development stakeholders.

Recommendation 7

The project selection process for regional projects needs to be clearly defined and agreed upon by FedNor and the CFDCs. This should include a clear definition of the scope of regional projects. It could also include a series of outcomes that are regional in nature. The improved project selection process should, however, continue to involve consultations with other regional economic development stakeholders with respect to needs for regional EODF projects. In refining the project selection process, FedNor and the CFDCs will need to clarify the following:

  • definition and scope of regional projects.
  • how regional projects should be approved.
  • how project leads should be identified (one CFDC, several CFDCs, others).
  • how regional projects should be monitored;
  • etc.

In refining the project selection process, care should be taken to ensure that the factors that have made this initiative successful (such as its grassroots aspect) are preserved or built upon, and that other recommendations are taken into account.

Conclusion 9

The project selection criteria are clear, appropriate and applied. The relevance of the projects selected (as demonstrated by the appropriateness of the early results) are indicative of the fact that the criteria are applied and appropriate given the program objectives.

Recommendation 8

The current project selection criteria should not change.

Conclusion 10

The role of FedNor in support of the projects is important given its experience with other programming and with similar types of projects in Northern Ontario. This support is appropriate and sufficient.

Recommendation 9

Notwithstanding other recommendations, FedNor should continue to provide the same level and type of support to CFDCs and to ensure that adequate resources are allocated to provide this support.

top of page
Table 18: Conclusions and Recommendations (continued)
Conclusions Recommendations
4.3 Success

Conclusion 11

The EODF has been successful, even in its infancy, in achieving its intended immediate and intermediate outcomes. Even though many projects were still ongoing at the time of the evaluation, all projects have resulted in at least one of the program’s intended outcomes.

See Recommendations 1 to 9

Conclusion 12

The EODF is helping organizations undertake projects that they could not otherwise undertake or would have to complete differently (lower quality, smaller scope, delayed, etc.). The program is therefore highly incremental. As such, the success of the projects is highly attributable to the program.

See Recommendations 1 to 9

top of page
Table 18: Conclusions and Recommendations (continued)
Conclusions Recommendations
4.4 Monitoring and Accountability

Conclusion 13

FedNor has made good progress in implementing the performance measurement strategy outlined in its Results-based Management and Accountability Framework (RMAF). In addition, the CFDCs have been very diligent in respecting their commitments regarding progress reporting. As such, they have taken the steps required to ensure that they have the information they need to meet these commitments. However, some of the reporting requirements are not clear. Additionally, it is not evident that all RMAF commitments are met through the existing reporting structure. Performance measurement is still a work-in-progress.

Recommendation 10

FedNor should continue to improve its performance measurement system with respect to the EODF. This should include updating the RMAF in consultation with the 15 CFDCs to ensure that they are in agreement with the commitments outlined in this document. It should also include revising the reporting structure to ensure that it is consistent with the program’s structure as well as the RMAF indicators. The revised reporting requirements should include clear definitions and instructions.

Recommendation 11

Once the performance reporting requirements are finalized, FedNor should give due consideration to providing CFDCs with more effective tools for reporting. This could include the grants module of The Exceptional Assistant (TEA).

top of page
Table 18: Conclusions and Recommendations (continued)
Conclusions Recommendations
4.5 Alternatives and Cost-Effectiveness

Conclusion 14

There are no alternative service delivery approaches for the EODF that would produce the same results at a lower cost. Currently, the administrative portion of this initiative is reasonable, the CFDCs rely on the large contribution of its volunteer Boards of Directors, a significant proportion of the projects costs are leveraged through other sources, and the projects are contributing to the achievement of the intended outcomes. The program is therefore cost-effective.

See Recommendations 1 and 3 and:

Recommendation 12

FedNor and the CFDCs should monitor the appropriateness of the operational budgets for the EODF. It is important to ensure that, with the continuation of the program, the burden on FedNor and CFDC staff is not unreasonable. It is also important to ensure that the program requirements do not conflict with other FedNor and CFDC staff priorities.

Recommendation 13

CFDCs should continue to require that, where possible, additional sources of funding be sought by the secondary recipients.

top of page
Table 18: Conclusions and Recommendations (continued)
Conclusions Recommendations
4.6 Lessons Learned

Conclusion 15

The relationships among all players involved in this initiative was key to the success of the EODF. These included the relationship: 1) across the 15 CFDCs who worked extremely well in reaching consensus on the design and delivery of this initiative; 2) between the CFDCs and FedNor where the pre-existing relationship helped to contribute to the smooth implementation of the initiative and the commitment of both parties to the initiative helped make it a success; 3) within the CFDCs among staff and with the Boards where staff demonstrated a high commitment to make this initiative work and the Board members worked with staff to ensure that the project applications were reviewed and approved in a timely fashion; and 4) between the CFDCs and the community where the fact that the CFDC was a recognized credible economic development organization which knew the community and vice versa made it easier to roll out the program within a very short time frame.

See all previous recommendations

Conclusion 16

Flexibility was also key to the success of this initiative, yet ongoing and increased flexibility may help ensure the continued success of the initiative. The program was set up by the CFDCs to provide each CFDC the same amount for each of the program elements. However, enough flexibility was incorporated into the guidelines so that the CFDCs were able to redistribute some of their budgets across components in order to better meet the needs of their communities. While some CFDCs would have liked more flexibility right from the start, the combination of guidelines and flexibility within those guidelines helped ensure that the intended program results were achieved while ensuring that community needs were considered. Nevertheless, increased future flexibility, within set parameters, may contribute to a more effective decision-making process and ensure that the needs of the communities are better met. Additionally, more flexibility regarding bureaucratic requirements for very small projects could help make the program more cost-effective.

Recommendation 14

FedNor should consider the feasibility of putting in place a process to have only one contribution agreement with each of the 15 CFDCs. This agreement would include all elements of the program for local initiatives (i.e., excluding regional projects and community capacity building, but including business planning, access to capital, skills development, retention and attraction of youth, and technological enhancements) with guidelines as to the minimum and maximum proportion of funds to be allocated to each of these elements. This would provide the flexibility required at the local level while ensuring that all priorities are addressed.

Recommendation 15

FedNor should consider implementing a risk-based approach for its monitoring and reporting strategy. This should be integrated in the considerations for Recommendation 10.

Conclusion 17

In order to ensure good ongoing performance reports, it is key to make sure that the performance measurement system is reflective of the program and that performance reporting requirements are clearly defined. The RMAF for the EODF was developed by FedNor. Because of extremely tight deadlines, CFDCs were not directly involved in the development of this RMAF. While FedNor did attempt to consult with some CFDC representatives, these consultations were limited and most CFDCs were not familiar with the document. This presented challenges in that CFDCs did not fully understand (or agree with) the program logic model and performance indicators. As such, they did not understand some of the reporting requirements. Additionally, some of the reporting requirements were not clear and were therefore interpreted differently from one CFDC to the next. Finally, the key indicators identified in the RMAF were not fully integrated into the performance reporting requirements.

See Recommendation 10

Management Response to Conclusions and Recommended Areas for Follow-up

Recommendation #1: The EODF should be renewed.

Management Response: Agreed.

If the Government agrees to renew EODF, FedNor will ensure that it continues to be delivered as an effective economic stimulus for the rural areas of Eastern Ontario.

Departmental Audit and Evaluation Committee (DAEC) Response:

Recommendation not supported.

Recommendation No. 1 was not supported by DAEC members since evaluation findings were deemed to be insufficient to support this recommendation.

Recommendation #2: The five program priorities should continue to form the basis of EODF programming. These encompass current priority needs and are broad enough to address evolving economic development needs.

Management Response: Agreed.

The five program priorities should be maintained and any adjustments should be considered in the context of change in the economic development needs of the region.

Recommendation #3: The EODF should continue to use the existing delivery model which involves FedNor contribution agreements with each of the 15 CFDCs who, in turn, provide funding to secondary recipients.

Management Response: Agreed.

However, consideration should be made to also allow direct contributions to other economic development organizations in addition to CFDCs, provided that there is support from Eastern Ontario CFDCs, although this might initially encounter resistance from the CFDCs, over time the flexibility to extend eligibility to a broader range of recipients could result in efficiencies and a broader program impact.

Recommendation #4: The CFDCs should continue to use a multitude of promotional tools to ensure that their communities are appropriately aware of the EODF. However, promotional strategies should consider the limited availability of EODF funds.

Management Response: Agreed.

Much has been learned from the initial launch of EODF with respect to the community response. With proper lead time, the CFDCs could be more strategic in the marketing and promotion of EODF in their local and regional communities and as a result increase the effectiveness of the limited EODF funds.

Recommendation #5: While the promotional efforts of the CFDCs have been fruitful, a process should be implemented to ensure that promotional materials are shared and to thus avoid duplication of efforts and build on best practices.

Management Response: Agreed.

Mechanisms, such as a secured website, are in place to share best practices. The CFDCs also meet as a group on a regular basis offering an opportunity to share approaches. These mechanisms will be used to share promotional material and communication strategies.

Recommendation #6: The current project selection process for small local initiatives does not need to change. However, since CFDCs have implemented different processes for the review and selection of projects, the different processes should be shared among CFDCs to help in the sharing of best practices and lessons learned.

Management Response: Agreed.

The same mechanism to be used in response to Recommendation #5 and will also be used to share best practices in project selection.

Recommendation #7: The project selection process for regional projects needs to be clearly defined and agreed upon by FedNor and the CFDCs. This should include a clear definition of the scope of regional projects. It could also include a series of outcomes that are regional in nature. The improved project selection process should, however, continue to involve consultations with other regional economic development stakeholders with respect to needs for regional EODF projects. In refining the project selection process, FedNor and the CFDCs will need to clarify the following:

• definition and scope of regional projects
• how regional projects should be approved
• how project leads should be identified (one CFDC, several CFDCs, others)
• how regional projects should be monitored

In refining the project selection process, care should be taken to ensure that the factors that have made this initiative successful (such as its grassroots aspect) are preserved or built upon, and that other recommendations are taken into account.

Management Response: Agreed.

The process used to select regional projects can be improved. There needs to be a better understanding of the definition of regional projects and their expected outcomes. A selection process, in which CFDCs vote after a brief presentation, needs a higher level of due diligence. The role in regional project selection for the newly formed CFDC/CAO regional committee should be considered. Collectively the 15 CFDCs have agreed that the selection process can be improved and there is a willingness to address the problem with FedNor's guidance.

Recommendation #8: The current project selection criteria should not change.

Management Response: Agreed.

The selection criteria are still appropriate. However, much can be gained by encouraging CFDCs to share selection processes (see Recommendation #5), although, there will continue to be unique features to each of the 15 organizations. CFDCs will be encouraged to sustain their current high level of due diligence.

Recommendation #9: Notwithstanding other recommendations, FedNor should continue to provide the same level and type of support to CFDCs and to ensure that adequate resources are allocated to provide this support.

Management Response: Agreed.

If EODF is renewed at the current level, FedNor will need to enhance and/or refocus its administrative support, particularly if proponents, other than CFDCs become eligible recipients. As the program matures, the level of support may change, perhaps with less FedNor involvement on local projects, and a higher level of engagement regarding regional projects.

Recommendation #10: FedNor should continue to improve its performance measurement system with respect to the EODF. This should include updating the RMAF in consultation with the 15 CFDCs to ensure that they are in agreement with the commitments outlined in this document. It should also include revising the reporting structure to ensure that it is consistent with the program's structure as well as the RMAF indicators. The revised reporting requirements should include clear definitions and instructions.

Management Response: Agreed.

FedNor, as a result of this formative evaluation, is consulting with CFDCs and a process is already underway to review the reporting tools currently in use. In this regard, the RMAF and the "Project Final Report" will be revised in order to better align with the EODF guidelines. This performance measurement system will also be reviewed to ensure that data for all RMAF indicators is being collected.

Recommendation #11: Once the performance reporting requirements are finalized, FedNor should give due consideration to providing CFDCs with more effective tools for reporting. This could include the grants module of The Exceptional Assistant (TEA).

Management Response: Agreed.

If EODF is renewed, FedNor will undertake a cost/benefit analysis of the grant module in The Exceptional Assistant (TEA) software to determine whether or not it would be an appropriate tool to assist CFDCs in reporting EODF outcomes. A quote for the site licences, installation and training pertaining to this module has been provided by CommonGoals, the service provider, to the CFDCs .

Recommendation #12: FedNor and the CFDCs should monitor the appropriateness of the operational budgets for the EODF. It is important to ensure that, with the continuation of the program, the burden on FedNor and CFDC staff is not unreasonable. It is also important to ensure that the program requirements do not conflict with other FedNor and CFDC staff priorities.

Management Response: Agreed.

FedNor staff, CF staff and volunteer directors have extended their resources to accommodate the additional demands associated with the delivery of EODF. If the program is renewed, resource levels will be reviewed to ensure all parties can sustain the higher level of activity. To date, conflicts between program requirements and other priorities have, for the most part, been managed.

Recommendation #13: CFDCs should continue to require that, where possible, additional sources of funding be sought by the secondary recipients.

Management Response: Agreed.

There was substantial leverage of additional funding in the initial round of EODF, particularly given the very short lead time provided prior to implementation. In future funding cycles, CFDCs will be better prepared to negotiate with secondary recipients to include financial involvement of partners.

Recommendation #14: FedNor should consider the feasibility of putting in place a process to have only one contribution agreement with each of the 15 CFDCs. This agreement would include all elements of the program for local initiatives (i.e., excluding regional projects and community capacity building, but including business planning, access to capital, skills development, retention and attraction of youth, and technological enhancements) with guidelines as to the minimum and maximum proportion of funds to be allocated to each of these elements. This would provide the flexibility required at the local level while ensuring that all priorities are addressed.

Management Response: Disagree.

In some respects, one agreement for the core program elements (not including community capacity building and regional projects) may offer administrative efficiencies for FedNor and the CFDCs. On the other hand, possible limitations on reporting outcomes for individual elements within a "master" agreement will need to be considered. Another major consideration is the delegation of authority within Industry Canada for various levels of funding. Larger agreements requiring approval at higher levels may result in an increased administrative burden and longer time frames for approval.

Recommendation #15: FedNor should consider implementing a risk-based approach for its monitoring and reporting strategy. This should be integrated in the considerations for Recommendation 10.

Management Response: Agreed.

Much has been learned from the initial round of EODF. Program management and staff will continue to determine the appropriate level of reporting based on their assessment of risk. Wherever possible, refinements to reporting will be made to create efficiencies. This is particularly true of smaller projects that are carried out by secondary recipients.


Figure 1

Eastern Ontario Development Fund Logic Model

[Description of logic model]

Return to document


Descriptions of Figures

EODF Logic Model

The EODP logic model presented is driven by three activities: facilitating and coordinating partnerships and networks; providing non-financial support to CFDCs and communities and others; and providing funding to CFDCs and their network. Direct outputs resulting from the stated activities are: MOUs, networks and partnerships; advice, information, workshops, referrals and training; contribution agreements and funding as "primary recipients"; and contribution agreements and funding between CFDCs and/or their network and "secondary recipients". EODP has four immediate outcomes. These outcomes are short-term and stem from the stated activities and outputs. Each outcome corresponds to a priority area of the program. They are: enhanced technology infrastructure and innovation; improved community capacity and response to economic development opportunities and challenges; improved business and trade skills; improved SME access to capital and leveraged capital. Intermediate outcomes result from the immediate outcomes, and can be considered as medium-term outcomes. These outcomes are not typically under the complete control of the program, but could be considered to be in the realm of influence, in terms of program effort. EODF has the following five intermediate outcomes: improved business practices and increased entrepreneurship; attraction, retention and development of human capital, including special client groups; improved competitiveness through retention and expansion of existing businesses and new business startups; increased use of technology (including information and communications technologies (ICTs), commercialization of research and adaptation to the knowledge-based economy); and development of external markets (out of region, province, and country). Ultimate outcomes are longer-term and subject to influences beyond the program itself. They are considered to be at a more strategic level. The ultimate outcomes of EODF are: diversification and competitiveness of rural Eastern Ontario economy; economic stability, growth and job creation; and sustainable, self-reliant communities.

Return to document


Figure 1: Importance of EODF Priorities

Figure 1 shows the importance of the five priorities of the EODF to support community economic development, on a scale of 0 to 10. The average ratings results were as follows: business development (9.4); access to capital (9.3); skills development (9.3); retention and attraction of youth (8.9); and technological enhancements (8.8).

Return to document


Figure 2: Project Selection Process

The EODF project selection process is depicted in this figure.

For agreements with secondary recipients, applications are first received by CFDC. These applications are reviewed by CFDC staff, followed by the CFDC Board (or sub-committee). The applicant is then advised if the project is approved or not. If approved, a contribution agreement is signed. The recipient is then responsible for preparing project progress reports until the project is completed.

For community capacity, applications are received by CFDC. These applications are reviewed by CFDC staff, followed by the CFDC Board (or sub-committee). Following that, applications are reviewed by FedNor. The CFDC is advised whether the project is approved, who in turn advises the applicant. If approved, a contribution agreement is signed. The recipient is then responsible for preparing project progress reports until the project is completed.

For regional initiatives, CFDC brings potential projects to the group of 15 CFDCs to review and vote on their acceptance. If the CFDC group does not support, the lead CFDC advises the applicant. If the group votes in favour of the project, the project is reviewed by FedNor. If approved, the lead CFDC is advised, who in turn advises the applicant. A contribution agreement is signed, and the recipient is then responsible for project progress reports until the project is completed.

Return to document


Figure 3: Incrementality

Based on the responses from the survey of secondary recipients on the incremental impact of EODF projects, an incrementality assessment was done. This assessment is depicted in figure 3 for total projects and for four program components: business development, access to capital, skills development and retention and attraction of youth.

Specifically, figures are provided for full, major, minor and no incrementality. Full incrementality included those who indicated that they would not have gone ahead with the project and that this would have been major; high incrementality consists of all others who indicated a major negative impact but who would still have proceeded with the project; minor incrementality consists of those who said that the absence of funding would have had a minor negative impact on the project; and no incrementality consists of those who said that the absence of EODF funding would not have had any impact on the project.

For full incrementality total projects was 34, business development (48), access to capital (43), skills development (17) and retention and attraction of youth (40).

For high incrementality, total projects was 41, business development (39), access to capital (29), skills development (44) and retention and attraction of youth (44).

For minor incrementality, total projects was 21, business development (13), access to capital (21), skills development (31) and retention and attraction of youth (16).

For no incrementality, total projects was 4, business development (0), access to capital (7), skills development (8) and retention and attraction of youth (0).

Return to document