Evaluation of ISED Support to Extending Broadband Access to Rural and Remote Communities

From: Innovation, Science and Economic Development Canada

February 2020

Evaluation of ISED Support to Extending Broadband Access to Rural and Remote Communities [PDF - 1.26 MB]

Table of Contents

1. Program Context
2. Evaluation Scope and Context
3. Evaluation Methodology
4. Limitations and Mitigation Strategies
5. Program Alignment with Government Priorities
6. Program Alignment with Federal Role and Responsibilities
7. Design and Delivery - Program Improvements
8. Design and Delivery - Application Review and Contribution Negotiations
9. Design and Delivery - Mapping Data
10. Program Outcomes
11. Indirect Program Benefits
12. Efficiency of Program Delivery
13. Recommendations

1. Program Context

This is an evaluation of Innovation, Science and Economic Development Canada's (ISED) support to expand broadband access to rural and remote communities in Canada. This study includes the Connecting Canadians Program (CCP) and the Connect to Innovate Program (CTI), performed as a cluster evaluation, and represents the first time these programs have been evaluated.

Broadband internet is an essential service in Canada. Adequate internet services are of paramount importance to the economic and social well-being of Canadians. Some communities however, especially in rural and remote areas, continue to struggle to access these services.

A gap in broadband access exists between rural and urban communities. This gap has resulted from the increased cost of rural broadband infrastructure and reduced potential customer base per mile of service, that has limited Internet Service Providers (ISPs) investments in rural and remote communities. These market forces have largely focused private ISP investment in urban centers. As a result, rural and remote areas remain underserved with private investments alone.

The future of our economy, our prosperity and our society — indeed, the future of every citizen — requires us to set ambitious goals, and to get on with connecting all Canadians for the 21st century,

There have been multiple players working to expand broadband service to underserved rural and remote areas in Canada:

ISED

Policy setting, national strategy, administering radio spectrum, assessment and mapping of broadband service availability, broadband infrastructure investment funding, and coordination.

Infrastructure Canada

Broadband infrastructure investments in rural and northern communities. 

Canadian Radio-television and Telecommunications Commission

Assessment and mapping of broadband service availability, setting service targets, and arm's length quasi judicial telecommunications regulator.

Indigenous Services Canada

Guidance and funding for broadband infrastructure and connectivity for Indigenous projects and communities, and funding for community and economic development.

Provinces/Territories

Supplement federal programs/policies with their own initiatives, broadband investment funding, and facilitate access to passive infrastructure.

Municipalities

Investments in broadband infrastructure to undertake their own projects or partner with ISPs, and provide access to existing infrastructure.

ISED Broadband Initiatives

Since 2002, several ISED initiatives have worked to expand broadband service to underserved rural and remote areas where low population density and difficult terrain make private sector investments uneconomic:

• 2002: Broadband for Rural and Northern Development
• 2009:  Broadband Canada Program
• 2014:  Connecting Canadians Program (CCP)
• 2016:  Connect to Innovate Program (CTI)

The CCP was established to help bridge the gap through projects that would provide households with access to 5/1 Mbps service where no business case existed for private sector ISPs, while the CTI provided support for high-capacity backbone (infrastructure that interconnects various pieces of network) and last-mile infrastructure (final leg of networks that delivers service to customers) in rural and remote underserved communities.

CCP Targets

• Extend and/or enhance robust, affordable broadband services to 280,000 households in rural and remote communities and 13,000 households in northern communities in the Nunavut and the Nunavik regions.
• Target speed of 5Mbps download and 1Mbps upload ("5/1") in rural and remote communities, and 3-5/1Mbps in northern communities (Nunavut and Nunavik).
• CCP was announced in 2014, with most projects completed, and is now in its final stages.
• Projects in rural regions were eligible for ISED contributions up to 50% of project costs.
• Projects in remote/northern communities and/or Indigenous communities were eligible for ISED contributions up to 75% of project costs.
• For projects in remote/northern communities and/or that contribute to achieving the objectives of the program for the benefit of Indigenous communities, funding from all federal sources was up to 100%. Other projects, up to 100% total funding from all government sources (federal, Provinces/Territories, municipal) was also possible.

CTI Targets

• Extend or upgrade high-capacity broadband backbone infrastructure (essentially, the very high speed core services) in 300 underserved communities.
• Provide anchor institutions (e.g., schools, health clinics) within underserved communities with access to high-capacity broadband infrastructure.
• Extend broadband services to households with speeds of less than 5/1 through "last-mile" access (e.g., from backbone to home) but not the focus of the program.
• CTI was announced as a five-year program in 2016, and is currently in the project implementation phase.
• Projects proposing to upgrade existing backbone infrastructure or network resiliency were eligible for ISED contributions up to 50% of project costs.
• Projects proposing new backbone and new last-mile access were eligible for ISED contributions up to 75% of project costs.
• Projects proposing new backbone in remote communities, were eligible for ISED contributions up to 90% of project costs. New backbone and last-mile were eligible for up to 100% funding from all federal sources. Other projects, up to 100% total funding from all government sources (federal, Provinces/Territories, municipal) was also possible.

Since the launch of CCP and CTI, there have been a number of policy changes and evolving speed targets that continue to challenge bridging the gap between rural and urban access to high speed service. In December 2016, the CRTC defined broadband as a basic service and announced new universal service level targets of 50/10 Mbps by 2031 (up from 5/1 under CCP and CTI), redefining the definition of "high quality" and "high speed" internet, and therefore the service gap between rural and urban centers. These new targets reflect service levels required for Canadian households and businesses to take full advantage of the internet. 

In 2017, 37% of households in rural areas and 97% of households in large urban areas had access to 50/10 Mbps.

Federal Initiatives

Recent federal initiatives further support this evolving landscape:

• The 2019 Federal Budget announced up to $6 billion for new broadband investments over 10 years, including a new ISED Universal Broadband Fund, top-up funding for CTI, and a CRTC Broadband Fund.
• The new Connectivity Strategy, launched in June 2019, outlined the federal government's plan to coordinate broadband investments based on three key pillars:
  1. Working with partners to achieve universal 50/10 Mbps service levels and improving access to mobile wireless services.
  2. Targeting investments to deliver universal connectivity to areas where they are needed most, reducing cost and barriers to investment.
  3. Partnering with all orders of government, the private sector, ISPs, and other stakeholders to close the broadband gap.

Broadband Programs

These changes in strategy and investment have increased the scope of broadband programs since CCP and CTI were launched and emphasize the importance of future collaboration between funders to ensure access to timely non-overlapping funding in the future.

2. Evaluation Scope and Context

Evaluation Objectives:

1. Meeting planned Departmental evaluation coverage and Treasury Board requirements through a clustered approach to cover CCP and CTI over the evaluation period (April 2014 to March 2019).
2. Addressing the commitment to conduct an evaluation as a result of the Office of the Auditor General of Canada's 2018 Audit on Connectivity in Rural and Remote Areas.
   • The OAG's Report recommendation (reference number 54) states - For future broadband Internet funding programs, Innovation, Science and Economic Development Canada should adopt a program design that:
     • Maximizes the outcomes of public spending; and
     • Minimizes negative commercial effects on existing Internet service providers.
   • As part of the response to Recommendation 54, ISED's Detailed Action Plan committed to: ". . . conduct a program evaluation of the CCP and CTI programs that will assess the extent to which these programs achieved their expected outcomes. The evaluation will also assess program implementation and delivery, including work with partners. The findings will help to inform future program design."

1. To what extent does ISED's broadband infrastructure programming align with the Canadian government's priorities to have all Canadians able to access higher quality Internet services?
2. To what extent is ISED's broadband infrastructure programming aligned with federal roles and responsibilities?
3. To what extent has ISED's broadband infrastructure programming been implemented as planned – especially impacts of design and delivery, and lessons learned from CCP to CTI?
4. What progress toward achievement of expected outcomes has occurred?
5. Has the program been delivered efficiently?

3. Evaluation Methodology

1. Literature review

Utilized external third-party research and reporting on the relevance and need of broadband programming, the role and need for public institution involvement in the telecommunications sector, as well as documented benefits of internet access and other international indicators. Sources include: CRTC, The Economist, OCED, Statistics Canada, etc.

2. Document review

Provided background evidence on the relevance, design, and achievements of the programs. This included Memorandums to Cabinet, Treasury Board Submissions, federal budgets, Speeches from the Throne, CCP and CTI mandate letters, the 2018 report of the Office of the Auditor General of Canada on ISED's recent broadband programming, and application and progress reports for individual projects.

3. Broadband mapping data

ISED and CRTC mapping data was used to compare to the programs' targets of expanding 5/1 Mbps coverage to an additional 280,000 households, and 3-5/1 Mbps for northern regions for CCP; and expanding community connectivity to an additional 300 communities for CTI.

4. Key information interviews

Gathered in-depth information, including views, explanations and factual information on the evaluation questions. The interviews were semi-structured in nature with representatives of key stakeholders – ISED, CCP, and CTI staff; other government departments; and broadband units within Provincial and Territorial governments.

• 18 individuals representing every province and territory
• 11 Program Staff (4 director level, 3 program officers,  and 4 engineers)
• 5 Representatives from  Infrastructure Canada,  Indigenous Services and  Northern Affairs Canada

5. Financial data

Supported evaluation questions related to efficiency of program delivery. For example, operating efficiency was explored by comparing administrative costs as a percentage of total program expenditures. Data on actual vs. planned expenditures also provided indirect  information on efficiencies, identifying how projects were rolled out and potential delays in implementation.

6. Online surveys

Used to obtain information regarding the experience of funded and non-funded CCP and CTI applicants with the programs, as well as a limited number of questions related to performance. In addition, a survey of non-applicants was conducted, with limited application for the evaluation.

A total of 113 responses was received from the surveys from the funded and non-funded surveys. The funded survey had a 71% response rate and the non-funded survey had a 32% response rate.  The breakdown of survey responses and response rates follows:

7. Case studies

Encompassed both CCP and CTI programs. The case studies were used as a major data source in the assessment of outcomes and lessons learned from projects. Each case study involved interviews with a sample of representatives from each project (e.g., ISP project leads and stakeholders from communities and/or regional or Provincial/Territorial representatives) and document review (e.g., project progress reports, and ISED's notes on the projects' fit to program criteria) where available. CTI projects are still underway and case study participants reported on their experience to date.

Case Studies: Location of CCP and CTI Projects

4. Limitations and Mitigation Strategies

CTI projects are still ongoing and progress reporting to date is limited.

Generally due to the stage of project completion with respect to the CTI program, certain performance information on ongoing projects was unavailable at the time of the evaluation. For many of the ongoing CTI projects, the extent to which projects will deliver on their planned outcomes is a moderate limitation for the study. This has been mitigated through review of the project plan, the current project stage of completion, the analysis of previous CCP and CTI completed projects, and other lines of evidence and triangulation of data.

Indirect social and economic impacts derived from broadband access for households and communities is not part of project reporting.

The programs and ISPs are responsible for the investments in broadband infrastructure and do not have mandates to collect and report indirect social and economic benefits stemming from broadband infrastructure. In addition, these indirect impacts may be affected by a range of other factors and take many years to observe:

• The program and ISPs do not collect non project information on the indirect impacts for beneficiaries.
• It was not possible to identify and contact beneficiaries (e.g., through surveys), for reasons of privacy and confidentiality.

To mitigate, case studies were used as a key source of information on beneficiary impacts for completed projects. In addition, the nature of these impacts were corroborated by the case studies and literature review, which identified similar beneficiary benefits. As a result the CCP and CTI projects are expected to benefit households and communities similarly.

5. Program Alignment with Government Priorities

The CCP and CTI programs are aligned with the government's key broadband priority of access to quality broadband for all Canadians.  The programs were aligned through their focus on backbone broadband and last-mile infrastructure to further enable community and household access to affordable, quality broadband.

In addition, CCP and CTI programs continue to be aligned to other key federal government priorities:

• Supporting economic development

  Recipients identified that increased access to affordable, quality broadband derived other significant economic benefits for businesses, mainly through increased access to customers and suppliers.

• Supporting Northern and Indigenous development

  190 Indigenous communities will have access to broadband infrastructure through CTI projects.  In addition, higher project funding levels (CCP – 75%, and CTI – 90%) were available for remote and Indigenous projects in hard to reach areas that are more costly to connect.

• Having Canada benefit from the digital economy

  Recipients identified that the increased broadband access derived from these programs, allowed more households to access the digital economy and derive benefits in areas related to: safety, education, health, economic, and social services.

Internet Coverage Requirements

While CCP and CTI programs were aligned with government priorities and now most of Canada (96.9%) has access to the 5/1 Mbps internet service standard established at the time; internet coverage requirements continue to change to new standards, most recently to 50/10 Mbps. As a result, the rural gap is now assessed differently.  Currently, 37% of rural Canada has access to 50/10 Mbps.

The Federal budget announced an extension of the CTI program and additional funding, allowing ISED to proceed with CTI projects beyond 2021 to 2022-23. These additional projects were part of the original application assessment process and remain aligned to government priorities.  They also represent foundational projects required to enable communities to eventually meet evolving internet standards.

6. Program Alignment with Federal Role and Responsibilities

The federal government has a clear role to play in supporting broadband internet deployment to underserved areas. The CCP and CTI provide funding to ISPs in areas where market failure exists (i.e., too costly to build broadband infrastructure, and too few customers to recover costs) helping support a viable business case for the ISPs.  According to survey respondents, 71% of unfunded CCP projects did not proceed due to a lack of funds.  The federal government also co-funds projects with provinces to help ensure service availability throughout Canada.

This aligns with what ISP survey respondents believe to be the role of the federal government in supporting broadband deployment to underserved areas (Figure 1).  While most of these roles are funding related, the graph illustrates the importance of mapping data (location of broadband internet services and backbone infrastructure in Canada) to provide broadband connectivity gap analysis.

Figure 1

Interview and case study respondents identified that the federal role differs from other stakeholders as it has a key coordination and collaboration role to support efficient national broadband deployment. 

Federal

Development of a national strategy, coordination of stakeholders, provision of technical advice, funding to overcome market failure, and acting as a catalyst.

Provinces/Territories

Identifying and communicating regional priorities, outreach to ISPs to inform them about the programs, outreach to communities to inform them of the benefits of broadband, helping ISPs and other partners develop applications and implement projects, provide project funding, and project monitoring.

Private (ISP)

Building and operating high bandwidth and reliable broadband services, offering service packages with affordable pricing, and (in cases where they had significant broadband expertise) identifying appropriate technical solutions.

Budget 2019 noted CCP and CTI type programs will continue to be foundational to support the connectivity path moving forward. New funding announced will further emphasize the need for a federal coordination role to support timely, effective investments, while minimizing overlap and duplication.

7. Design and Delivery – Program Improvements

CTI was more complex than CCP due to a number of factors including: larger size and remote location of projects, nature of infrastructure requirements, multiple funders, and issues related to ownership of networks and use of third parties to help new organizations, such as, municipalities, small ISPs, and Indigenous groups roll out their projects. Due to the complexity of the CTI program and as a normal course for continuous improvement, ISED reviewed its design and delivery processes for CTI program improvement opportunities. The following outlines some of the considerations and changes made after the implementation of CCP and before the implementation of CTI:

1. Focus on Highest Needs

CTI funded infrastructure projects were typically more expensive, harder to build, and focused on highest needs rural, remote and satellite dependent communities as opposed to CCP's emphasis on cost per household. This contributed to the selection of projects in areas less profitable to the private sector that would not otherwise proceed without government funding.

2. Forms and Mapping Data

The CTI program simplified and updated online application intake forms to accommodate CTI's increased project complexity.  In addition, new mapping data was created to locate Points of Presence (PoP) and cross-referenced with community data to create an additional report/layer to the existing coverage data used for last-mile. [A PoP is a site in a network where a backbone network ends and connects to a local last-mile infrastructure].

3. Extensive consultations with stakeholders

Extensive consultations with PTs, ISPs, associations, parliamentarians, municipalities and Indigenous groups supported changes to CTI program design, including contribution ratios and enhanced flexibility in the type of projects approved for funding.

• Contribution funding ratios were adjusted for CTI based on stakeholder feedback to allow projects proposing to bring new infrastructure to more challenging communities that are satellite-dependent or in very remote areas to be eligible for ISED funding up to 90% (from 75%).
• The CTI program was based on the objective to bring backbone access to 300 communities, but was adjusted to consider redundancy needs and last-mile connections as without last-mile many underserved areas would still not have access to improved higher broadband capacity.
• The CTI program introduced project categorization for different types of last-mile, backbone, and redundancy projects to help in the analysis and comparison of similar projects.  This also allowed program staff to customize reporting templates and streamline recipient interactions to obtain additional application information.

4. Program Delivery Consideration

Program staff considered alternative ways to deliver the program to enhance value-for-money (e.g., efficiency - reduce administrative cost and increase timeliness; and effectiveness - increase outcomes and reduce funding overlap). Alternative approaches considered included changes to: transfer payments (e.g., direct transfers to Provinces/Territories), application intake (e.g., one-time, continuous, etc.), and tax incentives.  Certain program delivery changes were not implemented at the time as these options were not clearly identified as an improvement for all stakeholders.

8. Design and Delivery – Application Review and Contribution Negotiations

To access CTI funding, applicants were required to meet specific criteria, including project eligibility (e.g., backbone and last-mile).  In addition, online applications needed to be completed in full by the application deadline (April 1, 2017).

Overall, 69% of CTI recipients were satisfied with CTI's application process. However, evaluation respondents (interviews, surveys and case studies) consistently identified the application review, due diligence and resulting contract negotiation processes as burdensome, contributing to project delays.

Application Review, Due Diligence and Contribution Agreement

To maximize cost-effective program outcomes, CTI programming was designed and implemented to focus on projects where no business case existed for sole private sector investment. Accordingly, ISED conducted extensive application review and due diligence, relied on by Provinces/Territories and other funders, to help ensure projects meet coverage and speed requirements and to minimize overlap amongst proposals and overbuild of existing services. This due diligence involved ensuring the applicant has the financial resources needed, undertaking an initial environmental review (e.g., adjusting project route), determining whether there is a duty to consult Indigenous groups, and verifying project specifics (e.g., equipment, sites, routes, timelines, ownership and maintenance network.  This work required time to complete and staff and applicants noted delays in approval notification and signing of contribution agreements which resulted in project delays. Provinces/Territories observed that these delays may have an impact on future projects.

Interviews, case studies, and surveys suggest that the technical details required during the application review, due diligence and resulting contract negotiation processes were burdensome. Respondents also expressed concern these requirements may prevent smaller ISPs and other organizations operating in remote northern communities from applying for funding.  In addition, it was noted these organizations may need further time to collaborate and enter into the contractual agreements required to meet the applicant and project eligibility requirements; and other flexibilities to enhance the range of applicant project types.

Examples of potential improvements

The following were identified by respondents of interviews and case studies conducted:

• Continue allowing last-mile projects to be eligible to further enable access to broadband.
• Consider enhancing flexibility for very small remote communities and ISPs:
  • removing requirement for anchor institutions that do not exist in many communities;
  • project type (e.g., last-mile, backbone, back-up infrastructure); and
  • allowable cost types (e.g., operational costs, training, indirect travel and equipment);
• Consider enhancements to the application process to support further diversity of applicants:
  • publishing application guidelines further in advance;
  • longer or multiple application intake windows;
  • direct non-ISPs to seek technical expertise to help with proposals and project implementation; and,
  • reconsideration of level of detail required in the application.
• Consider further changes to contribution ratios and funding criteria to minimize burden.

9. Design and Delivery – Mapping Data

ISED and the CRTC maintain comprehensive and precise mapping data that describes connectivity in Canada and is a critical tool to inform project proposals and to support other decision-making. Mapping data compares the specific location of network infrastructure, such as the location of PoP in relation to demographic information, such as population/households, communities, and roads. This overlaying coverage data in relation to demographic information allows ISED to assess which areas have access and where gaps exist to further support the assessment of projects during the application review and due diligence processes to prevent duplication, and minimize displacing existing ISP investment.

Public Availability of Precise Mapping Data

Due to confidentiality restrictions, applicants were provided with aggregated mapping data identifying areas without existing service, but not showing the precise edge of competing service providers' networks. Applicants that chose to submit projects in areas close to existing services were required to demonstrate market research which showed the lack of existing services where their project was proposed. 

Further, all projects in areas of existing services were adjusted during the due diligence (e.g., negotiation) phase to minimize any proposed overbuilding of existing services based on evaluation against the precise mapping of current services (only available to ISED and CRTC). Many respondents identified the negative impact on timing, including approval/start up delays caused by the lack of access to the most up to date mapping data required to support applications.

CTI Eligibility Map Illustration (New backbone and last-mile)

Examples of potential improvements

The following were identified by respondents of interviews and case studies conducted:

• Continue to enhance the resolution of mapping data that is available publicly.
• Consider increasing the reliance placed on community knowledge, and unique needs gained through further collaboration with communities, particularly for last-mile service.

10. Program Outcomes

1. CCP:

   With the substantial majority of CCP projects completed at the time of this report, according to mapping data, an estimated 350,000 additional households received (or will receive) access to planned coverage as a result of the CCP program.   This will exceed its target of 280,000 by 70,000 households, or 25%.  Coverage across Canada has increased from 92.2% in 2014 to 96.9%. In addition, over the same period, an additional estimated 380,000 households received access to 5/1 Mbps or more, as a result of private ISPs.

2. CTI: 

   The CTI program is on track to surpass its goal of 300 additional connected communities with an estimated 900 communities impacted by new PoP and other upgrade projects, including 190 Indigenous communities. Higher capacity PoPs can deliver higher network speeds to end users.

Figure 2 identifies, based on CCP and CTI funded survey respondents, the service improvements resulting from ISED funded projects.

Figure 2

11. Indirect Program Benefits

1. Improved public safety

Improved coordination of emergency services - fire and flood emergency alerts, evacuation notices, current emergency status, ability to return home after evacuations, ability to call 911 when in very remote areas for business or recreation.

2. Improved access to public services

Easy access by the public to government information and forms, and the ability to complete forms online, e.g., building permits, notices of board meetings, general communication on government affairs.

3. Improved access and quality of education

In some rural areas they are now offering "Grade 12 at home" from online sources where it's not locally available, including remote access to high school and colleges.

4. Improved access and quality of Healthcare

Increased access to quality broadband makes it easier for households in rural and remote communities to access digital health services.  This includes the ability to contact your doctor, for physicians to share and access patient information, to implement tele-pharmacy and tele-ophthalmology, or to obtain test results.  These services are even more critical because the geographic location of the communities makes in-person access to medical expertise very challenging.

5. Social good

Broadband allows members of the public who have left small communities for urban centers (or abroad) to return home (e.g., when they retire, or finish schooling), or to allow them not to leave at all, because better broadband services allow communications to be conducted remotely and members of the community feel more connected to urban centers through social media.

6. Business Development

Small businesses can reach potential customers outside the community, providing more opportunities for growth. Big businesses can work more efficiently with ready access to supply, at low cost; preventing lengthy and costly shut-downs.

12. Efficiency of Program Delivery

For the purposes of this evaluation, the efficiency of the programs has been measured using the following financial metrics:

• Funds leveraged;
• Budget vs. actual spending:
  • Contribution;
  • Total operating; and
  • Administrative costs as a percentage of total program expenditures.

Funds Leveraged - Funding leveraged though ISED programming was higher than expected, further maximizing the outcomes of public spending and further demonstrating value for money.

• ISED's CCP funding eligibility ranged from 50% for rural regions, up to 75% for remote/north and/or Indigenous communities. CTI funding eligibility ranged from 50% for existing backbone and last-mile upgrades, to up to 90% for new backbone in remote communities.  As a result, in both CCP and CTI programs, depending on project mix, ISED funding proportions are expected to be within these ranges.
• Figure 3 identifies that CCP program funding represented 36% of total funding ($1.76 additional program dollars for every $1 of ISED funding), and CTI program funding represented 49% of total funding ($1.06 additional program dollars for every $1 of ISED funding).  In each case, ISED funding leverage was higher than anticipated.

Figure 3a

CCP:

Figure 3b

CTI:

*Other represents a diverse population of funders including: Provincial/Territorial, Municipal, ISPs, and not-for-profit orgs.

Budget variances were identified in CTI's annual contribution funding, based on the completion timeframes of CTI projects.  The observed contribution spending gaps in 2017-18 and 2018-19 funding, as per Figure 4, reflect timing delays in CTI projects over that period.

Figure 4

Case study, interview and survey consultations identified a number of internal and external factors contributing to CTI project delays, including:

1. Higher than anticipated applicant volume (CTI nearly double CCP)
2. Project complexities and locations
3. Extensive due diligence and application review
4. Partner consultation and readiness

CTI projects were often complex, involving multiple partners and covering difficult terrain in the most remote regions in Canada. The program implemented a number of steps to address potential challenges, including extensive application review and due diligence to help ensure projects meet coverage and speed requirements and to minimize displacement of private sector investment. In addition, staff provided guidance to recipients and pre-populated some data to help applicants complete their statement of work requirements for a contribution agreement. Even so, project delays were noted by the variance between budgeted and actual program spending.  Variances in program spending are mainly driven by: delays in due diligence review and negotiation of contribution agreements; and delays experienced by recipients during project construction (e.g., due to weather, changes to equipment or routes, time to obtain permits and licenses, procurement processes, etc.).

Figure 5 illustrates that actual total CCP and CTI operating cost (direct salary and O&M) were below budget by $868,000 over the evaluation period.

Figure 5

When O&M costs are calculated as a percentage of total costs (including contribution spending), the actual proportion of O&M cost is higher than planned; mainly due to the lower CTI contribution spending to date.

Budgeted administrative costs (direct salary and O&M) as a percentage of total cost was anticipated to represent approximately 3.3% of total cost (including contribution spending).  To date (2018-19), actual administrative costs for CCP and CTI as a percentage of total cost is 6.3%.  As indicated above, while total actual administrative costs were below budget, CTI contribution spending was substantially below budget for 2017-18 and 2018-19, increasing the actual proportion of administrative costs over total costs. 

The Programs have used a greater percentage of direct salary and O&M than planned due to delays in implementation in the CTI program and declining budgeted operating funds available in later years based on the original funding profile.

Despite the spending variances, with significant CTI contributions forecast to flow in 2019-20, it is expected that the substantial majority of the spending gap will be eliminated in 2019-20 and any excess O&M costs will be absorbed by ISED.

13. Recommendations

The findings of the evaluation led to the following recommendations:

1. Process Improvements– ISED should examine its processes for application intake and review, project selection and approval, and negotiations of contributions to identify opportunities to:
   • Further streamline processes and improve the timeliness of project approval and implementation; and
   • Allow more flexibility for small ISPs and non-ISPs to demonstrate eligibility and meet technical requirements.
2. Enhance Coordination– ISED, in consultation with key federal partners, should clarify and communicate to stakeholders the federal roles and responsibilities related to broadband programming and examine the feasibility of an enhanced coordination role, including the establishment of a "single window" for stakeholder inquiries and interactions regarding projects and funding.
3. Mapping Data Improvements– ISED should continue to work with the CRTC and ISPs to enhance the level of precision of publicly available mapping data (while considering confidentiality of data).