March 2011
Recommended for approval to the Deputy Minister by Departmental Audit Committee on
Approved by the Deputy Minister on
Table of contents
- 1.0 Executive summary
- 2.0 About the Audit
- 3.0 Findings and recommendations
- 4.0 Overall Conclusion
- Appendix A: Audit Criteria
- Appendix B: Management Action Plan
1.0 Executive Summary
1.1 Introduction
As part of Canada's Economic Action Plan, the Government of Canada provided $225 million to Industry Canada to develop a broadband strategy over three years, of which $190 million was made available to the Broadband Canada: Connecting Rural Canadians program (the "Program") to extend broadband availability to as many un-served and under-served households as possible.
Of this $225 million, $190 million was made available to the program Broadband Canada: Connecting Rural Canadians. The program is administered by the Electronic Commerce (ECOM) Branch within the Spectrum, Information Technologies and Telecommunications Sector. Through a competitive application process, it provides federal funding to organizations that build or operate broadband infrastructure in remote and rural areas.
As of February 17, 2011, the program had 91 active project files: 43 with signed contribution agreements and 48 for which contribution agreements were being prepared for signature.
The deadline for signing the remaining contribution agreements is March 31, 2011. Projects must be completed no later than December 31, 2011; there is considerable pressure on the program to finalize the contribution agreements so that approved projects can be completed on time.
At the time of this audit, the program was in the initial phases of recipient monitoring, with the large majority of funded projects less than 50% complete.
The objective of this audit was to assess the program's management control framework with specific focus on internal controls, governance and risk management surrounding the program lifecycle. The scope of the audit included the following phases of the program lifecycle:
- Application Assessment Process – the process to select applications for funding;
- Contribution Agreement Development – the process to develop and sign contribution agreements;
- Recipient Monitoring and Reporting – the process of monitoring and reporting on recipient activities; and
- Program Oversight – the overall program monitoring and oversight process to ensure the program is meeting its intended objectives and potential issues are identified and followed up on.
1.2 Overall Conclusion
Although some exceptions were identified, this audit indicated that the Broadband Canada program was generally meeting the pre-established audit criteria associated with adequate and effective internal controls, governance and risk management as of February 2011. Please refer to Appendix A for more details.
Recommendations address opportunities to improve documentation, monitoring and intra-departmental collaboration.
1.3 Main Findings
Application Assessment Process
Eligibility and assessment criteria have been clearly defined, properly approved and incorporated into the program's tools, guidelines and checklists.
A risk-based monitoring strategy developed by the program facilitates the use of risk information to support funding decisions and assess the level of monitoring required.
Contribution Agreement Development
Contribution agreements reflected the Treasury Board Policy on Transfer Payments, were signed by the appropriate parties, and included reporting requirements that were consistent with the recipient monitoring strategy.
Recipient Monitoring and Reporting
Claims from recipients were processed in accordance with the Financial Administration Act section 34 and contribution agreement requirements.
Program Oversight
Program management has made investments in training, resources, tools and information to support the discharge of the Project Officer's responsibilities.
Program management was appropriately using the Industry Canada Delegation of Authority matrix; furthermore, responsibilities and accountabilities for program management were defined in job descriptions and/or performance agreements.
Senior management and oversight bodies received timely information on key aspects of the program, including risks and status of implementation.
1.4 Recommendations
Application Assessment Process
Recipient Monitoring and Reporting
Program Oversight
1.5 Statement of Assurance
In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the audit opinion provided and contained in this report. The audit opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed to with management. The audit opinion is applicable only to the entities examined and within the scope described herein. This audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada.
Susan Hart
Chief Audit Executive, Industry Canada
Date
1.6 Audit Opinion
In my opinion, the administration of the Broadband Canada program has no material weaknesses related to the risk management and governance processes. There are moderate risk exposures related to internal control processes where improvements are required.
Susan Hart
Chief Audit Executive, Industry Canada
Date
2.0 About the Audit
2.1 Program Background
As part of Canada's Economic Action Plan, the Government of Canada provided $225 million to Industry Canada to develop a broadband strategy over three years, of which $190 million was made available to the Broadband Canada: Connecting Rural Canadians program (the "Program") to extend broadband availability to as many un-served and under-served households as possible.
Of this $225 million, $190 million was made available to the program Broadband Canada: Connecting Rural Canadians. The program is administered by the Electronic Commerce (ECOM) Branch within the Spectrum, Information Technologies and Telecommunications Sector. It aims to provide essential broadband connectivity infrastructure to Canadians in remote and rural areas. The availability of broadband will allow participation in the Internet economy by facilitating access to information, services and opportunities that would otherwise be out of reach.
Through a competitive application process, the program provides federal funding to private sector companies or consortia, provincial/territorial or publicly owned entities, and/or not-for-profit organizations that build or operate broadband infrastructure.
To understand the extent to which Canadians remained un-served or under-served, the program conducted a mapping exercise in consultation with the provinces and territories, the private sector, the Canadian Radio-Television Communications Commission and other federal departments. Based on the mapping data, Geographic Service Areas (GSAs) were defined.
Between September 1 and October 23, 2009 the program opened for applications to fund projects in the GSAs. The financial assistance provided by the program is limited to a maximum of 50% of total eligible project costs. The stacking of federal funding is also limited to 50%, with the exception of projects proposing to serve First Nations communities, where federal stacking may be allowed to reach 100% of eligible costs.
In response to the competitive call for applications, the program received 570 applications for funding. The program team established guidelines to assess recipients' proposals using a two-phased approach.
- In the first phase, the program team reviewed the documentation for completeness and assessed the proponent's proposal against the eligibility criteria.
- If the proposal's documentation was complete and the proposal met all eligibility criteria, it was evaluated further in the second phase against a set of assessment criteria. If the proposal met all the assessment criteria, the project was shortlisted into a pool of applications to be considered for funding.
Applying this approach, the program team came to an initial shortlist of 226 projects that passed the eligibility and assessment criteria. The projects were then ranked based on the lowest cost per household within each GSA to determine which projects would be considered for funding. As of February 17, 2011, there were:
- 91 active project files for which funding had been provided or would be provided; and
- 135 projects that will not be funded for various reasons.
As of February 17, 2011, 43 contribution agreements have been signed out of the 91 active project files. The deadline for signing the remaining contribution agreements is March 31, 2011.
As demonstrated below, the majority of projects that have signed contributions are less than 50% complete:
- 10% complete – 24 projects (total IC contribution value: $34,142k)
- 25% complete – 9 projects ($4,245k)
- 50% complete – 4 projects ($411k)
- 75% complete – 4 projects ($322k)
- 100% complete – 2 projects ($63k)
Total – 43 projects with signed contribution agreements
At the time of the audit, the program was in the initial phases of recipient monitoring – 76% of the projects with signed contribution agreements were less than 50% complete and 48 active project files did not have signed contribution agreements.
The program had experienced some delays, particularly during the assessment phase of the application review. There were 570 applications to review totalling $1 billion of requested funding for a program with $175 million available to contribute. Consequently, the assessment phase took longer than expected.
In addition, there was a considerable degree of overlap among the 226 applications considered eligible for funding, making negotiations progressively more difficult: as initial projects were identified for funding, subsequent ones had to be re-worked to minimize duplication. This created a domino effect, delaying the signing of contribution agreements (CAs), which led to the re-profiling of funding to subsequent fiscal years.
Some of the mitigating strategies in response to the delays include monitoring of proponents and working closely with other stakeholders (e.g. provincial governments) to identify any issues that may arise. The decision to re-profile $70.5 million has been approved. Furthermore, costs for projects using satellite technology to deliver Internet services can now be paid over five years. As a result, the program can now focus on getting the remaining 48 CAs signed, which is expected to be completed by March 31, 2011.
2.2 Objective and Scope
The objective of this audit was to assess the management control framework of the Broadband Canada program, with specific focus on internal controls, governance and risk management over the program lifecycle. The scope of the audit included the following phases of the program lifecycle:
- Application Assessment Process – selecting recipients for funding. The audit assessed the process to ensure that only eligible proponents were funded based on a formal assessment process using established criteria; that the funding decisions were documented and justifiable; and that the funding provided to recipients was approved, taking into consideration the associated risks.
- Contribution Agreement Development – the process to develop and sign contribution agreements. The audit examined a sample of funding agreements to ensure they were developed using an approved template containing complete and appropriate terms and conditions that adhere to Treasury Board's Policy on Transfer Payments and Directives.
- Recipient Monitoring and Reporting – the process of monitoring and reporting on recipient activities. The audit reviewed a sample of claims to ensure that the recipient has met the appropriate reporting requirements; that program management has evaluated the actual project results against the planned results; and that appropriate approvals were obtained prior to disbursement of funds.
- Program Oversight – the overall program monitoring and oversight process to ensure the program is meeting its intended objectives and potential issues are identified and followed up on. The audit assessed processes and infrastructure (e.g. resources, tools, etc.) established to support the delivery and management of the program.
2.3 Audit Methodology
This internal audit was conducted in accordance with Treasury Board's Policy on Internal Audit and its Internal Auditing Standards for the Government of Canada. The audit approach consisted of the following:
- Planning phase – We conducted a risk assessment of the program's management control framework with respect to governance, internal control and risk management processes. We gathered evidence through document review and interviews with key stakeholders and synthesized the evidence to identify the areas of higher risk in the program. The results of the risk assessment contributed to the development of the audit criteria and the audit program used during the conduct phase.
- Document Review – We reviewed over 50 key documents to gain an understanding of the processes throughout the program lifecycle, including the tools/templates used by program officers and examples of reports prepared by management. Also, the documentation provided corroborating evidence to support conclusions reached and assertions made by the program management.
- Interviews – We conducted 12 interviews with key individuals involved with the program, including the Associate Assistant Deputy Minister, the Director General of ECOM, the Director of the program, four Program Officers, the Special Advisor and other staff.
- Application File Review – We examined 43 applicant files as follows:
- 10 files that did not pass the completeness screening phase
- 5 files that did not pass the eligibility criteria screening phase
- 14 files that did not pass the assessment criteria screening phase
- 14 files approved for funding.
The information gathered through the above procedures was analyzed against the audit criteria contained in Appendix A. The audit criteria selected for this audit were based on the Treasury Board Core Management Controls, in combination with 'risk-based' controls that were specific to grants and contributions Programs. The audit criteria were designed to enable an assessment of key practices, procedures and controls in place within the program.
The planning phase of this engagement was from October 2010 to January 2011. Audit fieldwork was conducted from January to February 2011. We debriefed management on February 25, 2011 to validate the accuracy of findings contained in this report.
3.0 Findings and Recommendations
3.1 Introduction
This section presents detailed findings from the Broadband Canada program audit. Findings are based on the evidence and analysis from both the initial risk assessment and the detailed audit.
In addition to the findings presented below, observations of conditions that were non-systemic and of low materiality and risk have been communicated verbally to management for its consideration.
3.2 Application Assessment Process
During our review of project files, we noted that eligibility and assessment criteria have been clearly defined, properly approved and incorporated into the program's tools, guidelines and checklists to enable a consistent approach to assessing applications. Templates were consistently used to clearly demonstrate the results of the assessment of proponents against the pre-established criteria (i.e. pass/fail decision).
During our review of project files, we noted that each file contained a completed risk complexity assessment form. The program assessed each proponent's financial, project and applicant risk and provided an overall level of risk (i.e. high, medium, low). This risk information was used to determine whether to provide funding to a particular proponent, and formed part of the project summary form used to document project approval.
The program also developed a Risk Monitoring Strategy, which provides guidelines on the use of risk mitigation activities based on level of risk. For example, the Risk Monitoring Strategy indicates that if a project's risk is assessed as high, the recipient will not have access to cash advances and a holdback of 10% will be applied to each payment.
Finding 1.0: File Documentation
The project file is the primary record of decision for the program and as such should include all the necessary documentation to support the funding decision. The assessment scoring was clearly noted in the project files we examined; however, the approach to compiling the documentation was not consistent (i.e. the documentation was housed in various locations and in a variety of forms). As a result, project files did not always contain sufficient documentation to support the scoring against the pre-established assessment criteria.
Without a clear and organized project file containing support for all assessment decisions/scoring, there is a risk that the program may have difficulty demonstrating the consistency of decisions made.
Recommendation 1.0
We recommend that the Director General (DG), ECOM develop a structured approach to records management to ensure there is adequate documentation and transparency within project files to support the scoring assessment against the pre-established assessment criteria.
3.3 Contribution Agreement Development
Program management, in collaboration with Legal Services and Comptrollership and Administration Sector (CAS), developed a contribution agreement template that captures all the requirements from the Treasury Board directive on Transfer Payments. Legal Services was also engaged to provide advice and feedback on any proposed CA amendments.
The template was presented to the Programs and Services Board (PSB), which as part of its mandate reviews proposals against the requirements of the TB Policy and Directive on Transfer Payments (TPP/D) and against departmental policies and directives. Specifically, the PSB considers the conformity of the contribution agreement template with the requirements of the TPP/D.
3.4 Recipient Monitoring and Development
Based on our review of project files, all recipient claims contained the information required under the contribution agreement (e.g. claim for eligible expenses only). Proponents completed all the required forms supporting the claim. There was sufficient evidence that the appropriate delegated authority had reviewed and approved the claims as per the Industry Canada Delegation of Authority (DOA) matrix.
Finding 2.0: Recipient Tracking of Quarterly Reports
As a project progresses, the recipient is required to submit a quarterly progress report to enable program officers to monitor and oversee the project.
During the conduct phase of the audit, the program was just beginning to receive quarterly reports from recipients, and had not yet implemented a standard tool/template to track the receipt of these reports. We observed that some recipient reports were arriving late, and that program officers were using a variety of tools/templates to track the receipt of reports.
To ensure there is a consistent approach to monitoring the receipt of quarterly reports and immediately identify where follow-up is required for missing information, the program should develop and use a standard tool/template.
Without a standard tool/template to track the receipt of recipient reports, there is a risk that the approach to monitoring and follow-up may be inconsistent across program officers, leading to potential delays in the receipt of recipient reports.
Recommendation 2.0
As the program is in the initial stages of monitoring, we recommend that the DG, ECOM implement a consistent approach and mechanism for tracking the receipt of recipient reports to ensure timely monitoring of progress.
3.5 Program Oversight
The program has developed assessment guidelines—a series of checklists and rating grids—that have helped ensure consistency in assessing proponents. Furthermore, the program has created a number of templates (e.g. risk assessment template) that have aided in standardizing the file management process.
The employees involved in the assessment phase were provided training on scoring applicants against the eligibility and assessment criteria. The program also changed the composition of the program team as the requirements changed (i.e. a different skill set is required for developing a program vs. implementing a program).
During our review of project files, we noted that project approvals were made in accordance with the DOA matrix, which includes FAA requirements. The work descriptions developed for the program provide a clear overview of the responsibilities and accountabilities of each of the employees involved.
The program was diligent in providing timely information to senior management and oversight bodies. The deputy minister was informed of the status of the program weekly. Another tool the program used to assist with communication was a program dashboard, which provided a budget overview, claim status and overall project snapshot with weekly updates.
Finding 3.0: Collaboration for Transfer Payment Programs
During our audit, we noted that the program received support from various groups within Industry Canada including the Industrial Technologies Office, CAS, Legal Services, PSB, and the Spectrum Operations Branch. The program sought out these groups to obtain support; however, there is no mechanism in Industry Canada to efficiently identify collaborative opportunities relating to transfer payment programs.
Treasury Board's Policy on Transfer Payments specifies that collaboration and coordination should exist within and among departments to harmonize transfer payment programs and standardize their administration, when appropriate. Although collaboration exists in the department, a departmental information-sharing mechanism could provide a more efficient approach to identifying collaborative opportunities.
Without a formal departmental information-sharing mechanism, program managers may miss opportunities to fully leverage the knowledge, tools and resources within the department, and inefficiencies and delays may be experienced in the design and implementation of short-term funding initiatives.
Recommendation 3.0
We recommend that the Chief Financial Officer of Industry Canada consider opportunities to enhance collaboration within the department relating to transfer payment administration. For example, the mandate of the Programs and Services Board could be expanded to include a periodic discussion on collaboration such as sharing of best practices.
4.0 Overall Conclusion
Although some exceptions were identified, this audit indicated that the Broadband Canada program was generally meeting pre-established audit criteria associated with adequate and effective internal controls, governance and risk management as of February 2011.
Recommendations address opportunities to improve documentation, monitoring and intra-departmental collaboration.
Appendix A: Audit Criteria
| Audit Criteria | Internal Controls | Risk Management | Governance | Criteria Met or Not Met |
|---|---|---|---|---|
| Application Assessment Process | ||||
| 1. Eligibility and assessment criteria/requirements that align to and reflect the program's objectives and the terms and conditions are established and approved. | X | Met | ||
| 2. Applications are consistently assessed against approved eligibility and assessment criteria while respecting authorities and key controls. | X | Met with exceptions | ||
| 3. Risk information is used to support funding decisions. | X | Met | ||
4. The funding commitment is approved by an individual with sufficient authority prior to the signing of the contribution agreement and is supported with appropriate and adequate documentation ( i.e. FAA section 32). | X | Met | ||
| Contribution Agreement Development | ||||
5. There is an appropriate and timely process for drafting, revising, approving and signing all contribution agreements. | X | Met | ||
6. Agreements are developed using an approved departmental template that includes all requirements from the Treasury Board Policy on Transfer Payments. | X | Met | ||
7. Reporting requirements included in contribution agreements are clear, relevant and measurable and aligned with the Recipient Monitoring Strategy to enable performance and financial monitoring of the recipient funding. | X | Met | ||
8. All contribution agreements, including amendments (as applicable), are signed by the appropriate delegated authority before eligible expenditures are reimbursed. | X | Met | ||
| Recipient Monitoring and Reporting | ||||
9. Claims from recipients are processed in accordance with FAA requirements (i.e. section 34), are accurately made, and reflect funding agreements. | X | Met | ||
10. Financial and non-financial information is received and adequately reviewed on a timely basis to ensure that:
| X | Met with exceptions | ||
| Program Oversight | ||||
11. Authority, responsibility and accountability for program management are clearly defined and understood. | X | Met | ||
12. The organization provides employees with the necessary training, tools, resources and information to support the discharge of their responsibilities. | X | Met | ||
13. Collaboration and coordination exist to harmonize transfer payment programs and standardize their administration, when appropriate. | X | Met with exceptions | ||
14. Management identifies, assesses and mitigates risks that would preclude the achievement of program objectives. | X | Met | ||
15. Recipient monitoring activities reflect the level of risk for the recipient. | X | X | Met | |
16. Financial and non-financial information is collected and analyzed and results are consolidated to demonstrate overall program performance and effectiveness. | X | Met | ||
17. Management monitors actual results against budget and planned performance and adjusts course as needed. | X | Met | ||
18. Oversight bodies and senior management receive sufficient, complete, timely and accurate information. | X | Met | ||
Appendix B: Management Action Plan
| Recommendation | Planned Action or Justification for No Action on the Recommendation | Responsible Official | Target Completion Date | Current Status |
|---|---|---|---|---|
Recommendation 1.0: | A) The program, working with Records Management (RM), has developed a comprehensive filing structure for the program. | Senior Policy Advisor with the Manager, Service Centre & Records Management | Complete | |
B) RM is in the process of creating file folders for the program. | Senior Policy Advisor with the Manager, Service Centre & Records Management | In progress | ||
C) In order to consolidate records, the program is clearing out the Shared drive, filing paper copies and migrating files to the Electronic Document Records Management System (EDRMS) where appropriate. | Manager, Policy and Liaison | In progress | ||
D) The program has documented file management procedures that will ensure that all pertinent documentation is contained within the application and project files. | Manager, Program Operations | In progress - draft document exists | ||
E) Application files considered "marginally failed" – where documentation on file may not be sufficient to justify failure – will be reviewed to ensure adequate documentation exists on file. | Manager, Policy and Liaison | In planning stages | ||
F) All documentation that exists in a consolidated fashion (i.e. that includes assessments on multiple files) will be contained in a supplementary application assessment file and a note referring to that file will be included in all application files. | Manager, Policy and Liaison with the Manager, Service Centre & Records Management | In planning stages | ||
G) Revise the management practice of geospatial data (connectivity and project information contained within a database and maps). This involves the creation of a new file structure and archival strategy for the non-EDRMS compatible file types associated with this data. | Manager, Engineering and Data Analysis | Initial analysis complete, a draft file structure and working group has been established. | ||
Recommendation 2.0: | A) The program will finalize a comprehensive risk-based monitoring strategy that includes mechanisms for tracking reports and for reporting on project achievements. | Manager, Program Operations | Draft complete | |
B) Document tracking sheet will be added to the "active files" spreadsheet, which program officers will update weekly, and lock a version each Wednesday to maintain thorough records. | Manager, Program Operations | In development | ||
Recommendation 3.0: | Agreed that the CFO will promote collaboration within IC relating to transfer payment administration.This will be done by sharing best practices through several venues and forums including: increased information on the CAS website, grants and contributions (G&C) training sessions given by the Programs and Services Directorate (PSD), governance structures (PSB, CAS Advisory Committee, and Management Committee), and increased proactive communications with G&C sectors. This increased activity will be coordinated by PSD. PSB, as the senior departmental committee overseeing grants and contributions, will receive a new annual summary report on improvements and best practices shared/implemented in the past year. For all new programs launched, PSD will proactively offer training sessions and share tools that may be of assistance. Online information will be augmented with tools and information to assist program managers. | Director, Programs and Services Directorate | By recent changes to the Transfer Payment policy will be posted on the CAS website, along with implications for sectors. | Material developed. Now at approval stage before sending to translation. |
Risk assessments will also be posted for at least 2 major programs. | Risk assessment for Broadband approved and tabled at PSB last Fall. SADI risk assessment at approval stage. NODP risk assessment underway. | |||
PSB will receive a summary report of improvement activity by . | Information gathered throughout the year. Report to be developed. | |||
Sharing and increased promotion of activities such as training will be ongoing. | PSD offers 5 to 10 sessions annually. |