Final Evaluation of Industry Canada's Involvement in the International Telecommunication Union

Final Report—April 29, 2009

Prepared for: Industry Canada/Audits and Evaluation Branch
Tabled and approved at DEC on June 26, 2009.

Table of Contents

• Executive Summary
  • 1.0 Industry Canada's Involvement in the International Telecommunication Union
    • 1.1 Introduction
    • 1.2 Background and Context
    • 1.3 Program Description
    • 1.4 Logic Model
  • 2.0 Evaluation Methodology
    • 2.1 Evaluation Issues
    • 2.2 Methodology
    • 2.3 Limitations of Methodology
  • 3.0 Evaluation Findings
    • 3.1 To What Extent Is Industry Canada's Involvement in the ITU Consistent with Industry Canada's Mandate and Government Directions?
    • 3.2 Is There a Continued Need for Industry Canada's Involvement in the ITU?
    • 3.3 To What Extent Does Industry Canada Achieve Its Intended Results?
    • 3.4 What Leadership Role Is Canada Playing At The ITU?
    • 3.5 What Value Does Industry Canada's Involvement in ITU Provide?
    • 3.6 Does Industry Canada Have an Adequate Succession Plan to Ensure Continued Influence at ITU?
    • 3.7 Other Findings
    • 3.8 Summary of Findings/Conclusions
  • 4.0 Recommendations
  • Management Response

Executive Summary

Background

The International Telecommunication Union (ITU), created in 1865, is the United Nations Specialized Agency for telecommunications. The ITU is an inter-governmental organization, based in Geneva, Switzerland, with 191 Member States party to its treaty instruments. As the global focal point for governments and the private sector, ITU's role in helping the world communicate spans three core sectors: radio communication, standardization and development. The key functions of the organization are its treaty-binding regulations and the facilitation of world-wide standardization of telecommunications.

As a state party to the Constitution and Convention of the ITU, and its complementary Administrative Regulations (i.e. the Radio Regulations and the International Telecommunication Regulations), Canada has been a member in its own right since 1932, following an act of Parliament to join the organization. Membership in the ITU conforms to Section 6 (e) of the Department of Industry Act whereby the Minister shall "take any action that may be necessary to secure, by international regulation or otherwise, the rights of Canada in communications matters." Canada's membership in the ITU and Industry Canada's leading role contribute to the Department's strategic outcome of a fair, efficient and competitive market place. and is listed in the Main Estimates as part of Industry Canada's Program Activity Architecture, under the Program Activity concerning the development of regulations, policies, procedures and standards governing Canada's spectrum and telecommunications industries and the digital economy, which is entitled "Spectrum, Information Technologies and Telecommunications Sector—Marketplace".

Industry Canada's ITU activities are carried out mainly by two sectors:

• the Strategic Policy Sector and
• the Spectrum, Information Technologies and Telecommunications Sector (SITT).

The two branches under SITT are directly involved in the ITU technical and regulatory issues. The Telecommunications Policy Branch of the Strategic Policy Sector is responsible for coordination of Canadian participation in ITU activities, as well as a broad range of organizational and governance issues related to the strategic policy direction and management of the Union.

An Evaluability Assessment¹ (EA) was conducted in 2007–2008 to determine whether Industry Canada's involvement in the International Telecommunication Union (ITU) was ready for an evaluation, what might be the barriers to undertaking an evaluation and how best to implement an evaluation. The EA concluded that Industry Canada's involvement in the ITU was ready to be evaluated and that Industry Canada managers believed there was value in doing so. The EA also identified evaluation issues and potential evaluation methodologies. In the EA the following issues and questions were identified:

Methodology

The Evaluability Assessment² identified that Industry Canada's involvement in the ITU is a low risk activity. As such, the Evaluability Assessment suggested that a moderate evaluation effort would be appropriate. Accordingly, the following evaluation methodologies were employed in this evaluation:

1. Document Review
2. Review of Post-Conference Reports
3. Program Manager Interviews
4. Survey of Stakeholders
5. Follow Up Interviews
6. Interviews of Other Government Departments
7. Interviews of ITU Officials and Representatives of other ITU Member Countries
8. Succession Planning Analysis

These methodologies rely heavily on the subjective opinions of stakeholders, representatives of other government departments and retired ITU officials and representatives of other ITU member countries. However, it was expected that these groups would be unbiased in their opinions and would be quite willing to identify any shortcomings in Industry Canada's performance in relation to the ITU.

Evaluation Findings

The following conclusions were reached in relation to each issue as a result of the evidence collected through the evaluation:

• Industry Canada's involvement in the ITU is consistent with its mandate and Government directions.
  • The mandate is derived from the Department of Industry Act and the Radio Communication Act. The ITU activity is also an identified element of Industry Canada's Program Activity Architecture and contributes to all three of its Strategic Outcomes. Finally, the activity is consistent with the directions set out by the Government in the 2008 Speech from the Throne.
• There is a continued need for Industry Canada's involvement in the ITU.
  • Most stakeholders reported that there would be moderate to large impacts on their organization; Canadian telecommunication carriers, suppliers and manufacturers; and, Canadian users of spectrum and satellite, if Industry Canada no longer developed and submitted Canadian positions to the ITU. The implications are that it would be much more difficult, or impossible, for Canada to protect and access spectrum, satellite orbits and frequency assignments and to influence ITU regulations and standards.
• Industry Canada has been largely successful in achieving most of its intended outcomes in relation to its ITU activities.
  • The evidence collected revealed that Industry Canada has been largely successful in contributing to most of the different levels of intended outcomes as depicted in the logic model for the activity (see section 1.3) While no direct evidence was collected that the activities contribute to the Industry Canada Strategic Outcomes, the logic described in section 1.3 would lead the evaluation team to conclude that the ITU activities are contributing to the Strategic Outcomes as well.
• Canada, as represented by Industry Canada, is a recognized leader at the ITU.
  • Stakeholders, including ITU participants who have viewed Canada's performance at ITU forums, believe that Canada is a strong leader at the ITU. Stakeholders believe this is based on the merit of Canada's proposals and its high quality and well-known representatives. Evidence of Canada's leadership is portrayed in the success Industry Canada achieves at the ITU and the high number of ITU leadership positions Canada has been successful in obtaining.
• Industry Canada's involvement in the ITU provides good value to Canada.
  • When informed of the full cost of Industry Canada's participation in the ITU, almost all stakeholders suggested that this participation provides very good to excellent, value to their organization; the Canadian telecommunication carriers, suppliers and manufacturers; and Canadian users of spectrum and satellite.
• There is an opportunity for Industry Canada to improve its succession planning in relation to its ITU activities.
  • In the conduct of the Evaluability Assessment, stakeholders suggested that succession planning may be an issue in relation to ITU activities. The analysis revealed that there are some demographics that suggest that succession planning will become an issue. However, a formal gap analysis has not been undertaken to determine the potential severity of this issue. While commitments have been made to develop and implement a formal SITT succession plan, this has not yet commenced. On the other hand, the evaluation found evidence that succession planning type activities, such as sending junior engineers to ITU conferences, occur.

There are two key findings from the evaluation that are not directly related to the evaluation issues:

• The most commonly cited Industry Canada-ITU weakness by stakeholders was that Canada does not send enough representatives to the ITU forums; and,
  • Sending an insufficient number of representatives to ITU forums was the most commonly cited Industry Canada ITU weakness by stakeholders. However, this weakness was only cited by 25% of stakeholders. In addition, the evidence collected suggests that this may be simply a perception. Furthermore, the evaluation evidence suggests that Industry Canada has been successful in its ITU activities, suggesting that there is not a strong requirement for additional resources to be sent to ITU meetings
• Post-Conference reports do not provide a good indication of the level of Industry Canada's success at the ITU.
  • The review of post-conference reports revealed a lack of consistency in format and content. While recent reporting has improved, there is still an opportunity to improve the reporting to demonstrate Industry Canada's performance at the ITU.

Evaluation Recommendations

The following are the two evaluation recommendations relating to Industry Canada's involvement in the ITU:

• Develop an improved methodology for measuring and reporting the success of Industry Canada activities at the ITU.
• Follow through on the SITT HR Plan commitment to develop and implement a succession plan for SITT.

¹Evaluability Assessment of IC's Involvement in the ITU, Industry Canada, 2007. (Return to text.)

²Ibid (Return to text.)

Industry Canada's Involvement in the International Telecommunication Union

1.1 Introduction

An Evaluability Assessment³ (EA) was conducted in 2007–2008 to determine whether Industry Canada's involvement in the International Telecommunication Union (ITU) was ready for an evaluation, what might be the barriers to undertaking an evaluation, and how best to implement an evaluation. The EA concluded that Industry Canada's involvement in the ITU was ready to be evaluated and that Industry Canada managers believed there was value in doing so. The EA also set out the issues to be investigated in an evaluation and the methodologies to be employed.

This report presents the results of the evaluation of Industry Canada's involvement in the International Telecommunication Union in accordance with the EA. This evaluation was undertaken in consultation with a Steering Committee established for the purposes of the evaluation, and managed by the Audit and Evaluation Branch of Industry Canada. This report is organized into four sections:

1. Section 1 provides the general background and description of the activities;
2. Section 2 presents the methodology followed in conducting the evaluation;
3. Section 3 presents our findings, organized in relation to the evaluation issues; and
4. Section 4 presents our recommendations.

1.2 Background and Context

Background

International activities are becoming more important to Canadians and the importance of Canadian participation and input into international forums that develop the technical basis for regulations and standards is increasing. International standards set common requirements for the global marketplace and federal government policy directs regulators to use international standards as the basis of national regulations and standards to the maximum extent possible. As a result, many Canadian telecommunications standards and regulations are adopted directly from those agreed internationally, such as within ITU. Particularly in radio communications, internationally agreed, treaty-binding regulations are incorporated into domestic regulations.

Context

The International Telecommunication Union (ITU), created in 1865, is the United Nations Specialized Agency for telecommunications. The ITU is an inter-governmental organization, based in Geneva, Switzerland, with 191 Member States party to its treaty instruments. Moreover, the ITU also engages more than 700 Sector Members and Associates,⁴ representing a cross-section of the telecommunications and information technology industry and other related organizations. As the global focal point for governments and the private sector, the ITU's role in helping the world communicate spans three core sectors: radio communication, standardization and development. The key functions of the organization are its treaty-binding regulations and the facilitation of world-wide standardization of telecommunications. The ITU also organizes TELECOM events and is the lead organizing agency for the World Summit on the Information Society.

Founded on the principle of international cooperation between the public and the private sectors, the ITU represents a global forum through which government and industry can work towards consensus on a wide range of issues affecting the future direction of this increasingly vital industry.⁵ The specific functions of the Union are as follows:

1. To effect through treaty level regulatory instruments, the allocation of bands of the radio-frequency spectrum, the allotment of radio frequencies and the registration of radio-frequency assignments. And, for space services, the ITU assigns any associated orbital position in the geostationary-satellite orbit or of any associated characteristics of satellites in other orbits, in order to avoid harmful interference between radio stations of different countries;
2. To coordinate efforts to eliminate harmful interference between radio stations of different countries and to improve the use made of the radio-frequency spectrum for radio communication services and of the geostationary-satellite and other satellite orbits;
3. To provide a unique world-wide venue for government and industry to work together to develop inter-operable, nondiscriminatory and demand driven international standards for telecommunications equipment and services which, in turn, may form the basis of many national standards and proposed regulations and statutes.

1.3 Program Description

Mandate

As a state party to the Constitution and Convention of the ITU, and its complementary Administrative Regulations (i.e. the Radio Regulations and the International Telecommunication Regulations), Canada has been a Member State of the ITU in its own right since 1932, following an act of Parliament to join the organization. Membership in the ITU conforms to Section 6 (e) of the Department of Industry Act whereby the Minister shall "take any action that may be necessary to secure, by international regulation or otherwise, the rights of Canada in communications matters." Canada's membership in the ITU and Industry Canada's leading role contribute to the Department's strategic outcome of a fair, efficient and competitive market place. Moreover, this role is listed in the Main Estimates as part of Industry Canada's Program Activity Architecture under the Program Activity concerning the development of regulations, policies, procedures and standards governing Canada's spectrum and telecommunications industries and the digital economy, which is entitled "Spectrum, Information Technologies and Telecommunications Sector—Marketplace".

Objectives

Active Canadian participation in ITU activities supports both government and industry objectives. The key objectives in contributing to the ITU are as follows:

• To secure Canada's interests in the international regulation of the radio frequency spectrum and international telecommunication regulation to protect Canadian interests in the access to the spectrum and satellite orbit resource as a means, for example, of facilitating communications across Canada and of protecting Canadian sovereignty in remote areas through modern digital technologies;
• To provide leadership and focus in promoting the competitiveness of the Canadian telecommunications industry's interests, i.e.. by working closely with Canadian stakeholders to obtain spectrum allocations at the global conferences, and development of global standards for telecommunications through the work of the ITU;
• To work with other countries to harmonize policy and regulatory frameworks, promote interconnection and interoperability of global telecommunication networks and services, and deal strategically with them to facilitate access to key markets;
• To build effective consultation and information dissemination processes with Canada's key trading partners; and
• To promote Canadian expertise, products and services with the ITU Member countries and over 700 Sector Members and Associates.

Resources

There are three primary cost elements of Industry Canada's participation in the ITU: Canada's financial contribution to the ITU, Industry Canada's staff costs and Industry Canada's travel costs.

Industry Canada receives authorization to provide a financial contribution to the ITU from Treasury Board. The current authorization level is $6,808,000 per year. This allows for currency fluctuations as the payment is in Swiss Francs and Industry Canada is also authorized by Treasury Board to use any surplus funds for other-related ITU activities. The use of surplus funds can be seen in Table 1 where they are described as "Other Uses of Grant". It should be noted that Industry Canada has not fully used the authorized funds in each year. For example, Industry Canada lapsed between $9,000 and $709,000 spanning the years 2003–2004 through 2006–2007.

The amount of the Canadian contribution to the ITU is commensurate with its international standing and commitment to the UN system and its specialized agencies, as well as, for example, a member of the G8.⁶ The authorized cost is associated with Canada's contribution level of eighteen units⁷ under the Minister of Industry's discretion; and other ITU related costs of benefit to Canada.

As shown in the organization chart of Industry Canada's involvement in the ITU, there are six directorates that are directly involved in the ITU activities. ITU staff estimate that on average they employ 25 FTEs on ITU activities across the six directorates which represents approximately $2.28 million dollars in salary per year.

Table 1 shows that the average cost of Industry Canada participation in the ITU, including contributions, was approximately $9.5 M per year over the past five years.

Accountabilities

Industry Canada's ITU activities are carried out mainly by two sectors: the Strategic Policy Sector and the Spectrum, Information Technologies and Telecommunications Sector (SITT). The two branches under SITT, Spectrum Engineering, and Radiocommunications and Broadcasting Regulatory, are directly involved in the ITU technical and regulatory issues. The Telecommunications Policy Branch of the Strategic Policy Sector is responsible for coordination of Canadian participation in ITU activities, as well as a broad range of organizational and governance issues related to the strategic policy direction and management of the Union. There are principally six directorates from three branches that are directly involved in ITU activities:

• Strategic Policy Sector:
• Telecommunications Policy Branch:
  • International Telecommunications Policy and Coordination (DIT)
  • Spectrum and Radio Policy (DSRS)
• Spectrum, Information Technologies and Telecommunications Sector:
• Spectrum Engineering Branch:
  • Spectrum Planning and Engineering (DDGSE)
  • Broadcast, Multimedia Planning and Technical Policy (DBCP)
  • Telecom Engineering and Certification (DSI)
• Radio Communications and Broadcasting Regulatory Branch:
  • Space and International Regulatory Activities (DSIR)

The organization chart in the Figure 1 shows Industry Canada's involvement in the ITU.

Figure 1: Organizational Structure for Industry Canada's ITU Activities

Description Link

Stakeholders and Beneficiaries

Industry Canada's contribution to the ITU enables the Government of Canada, working with stakeholders, including Canadian telecommunication carriers, service providers and manufacturers, to reflect Canadian interests and requirements in the formulation of treaty-binding regulations, in the development of global standards and in enabling Canadian industry to use ITU processes, products and services to enhance Canada's competitiveness in telecommunications worldwide. It should be noted that many Canadian companies and organizations also participate directly in the ITU as Sector Members or Associates.

In addition, more than forty companies in Canada participate in Industry Canada's national ITU consultation process along with other government departments and agencies, such as:

• Department of Foreign Affairs and International Trade (DFAIT)
• The Canadian Space Agency (CSA)
• Department of National Defence (DND)
• Canadian International Development Agency (CIDA)
• Indian and Northern Affairs Canada (INAC)
• Canadian Radio-television and Telecommunications Commission (CRTC)
• Transport Canada (TC)
• Environment Canada (EC)
• National Research Council (NRC)
• Health Canada, (HC), and
• The International Development Research Centre (IDRC).

General Description

Consistent with Industry Canada's statutory responsibilities under the Department of Industry Act and the Radiocommunications Act, active Canadian participation in ITU activities supports both government and industry objectives. The most important objectives are:

• securing access to the radio-frequency spectrum and satellite orbit slots;
• avoiding harmful interference to radio communication services;
• ensuring management and functioning of ITU meets Canadian requirements and is consistent with best practices for governance in the United Nations system; and
• reducing costs for equipment manufacturers to bring products to market for the ultimate benefit of the Canadian consumer.

The processing of satellite and frequency registrations is primarily an operational activity that is undertaken by the Space and International Regulatory Activities directorate. The work of ensuring ITU regulations, frequency allocations and technical standards conform to Canadian interests and requirements involves all six Industry Canada directorates.

1.4 Logic Model

Expected Results

Logic Model

In this section, we provide a logic model of Industry Canada's involvement in the ITU. A logic model describes the relationship amongst activities, outputs and outcomes. The following section provides more details on the activities and outputs. There is a graphic of the model presented at the end of the section.

Activities and Outputs

Consult with Stakeholders and Partners
Industry Canada consults with stakeholders and partners in relation to the ITU using a number of different methods. Industry Canada conducts a national ITU consultation process with more than forty companies in Canada along with other government departments and agencies, such as Department of Foreign Affairs and International Trade, Environment Canada, Health Canada, the Canadian Space Agency, the Department of National Defence, the Canadian International Development Agency, Indian and Northern Affairs Canada , the Canadian Radio-television and Telecommunications Commission, Transport Canada, the National Research Council, the International Development Research Centre. Consultation is done partly through the establishment of a number of Canadian National Organizations (CNOs) that parallel ITU activities. The objective of the CNOs is to coordinate Canadian participation in the activities of the ITU, with the ultimate objective of promoting and protecting Canadian public and private sector interests.

Through this consultative process, Industry Canada is able to ascertain the requirements and interests of Canadian companies and other government departments. In developing a Canadian position, Industry Canada must balance the interests of all stakeholders including other government departments to formulate the optimal position for Canada. While Industry Canada considers the inputs of all consultative participants, it has the ultimate authority in finalizing the Canadian proposals and positions.

It should also be noted that Canadian companies can, and do, represent themselves at the ITU as Sector Members or Associates. There are twenty-nine Canadian Sector Members and eleven Associates, which participate in the activities of the three ITU Sectors, namely, the Radiocommunication Sector, the Telecommunication Standardization Sector and the Telecommunication Development Sector.

Consulting with stakeholders and partners is an activity that facilitates the activity, Prepare Submissions, Participate in Conferences, Meetings, Study Groups, Assemblies, and Chair Committees.

Consult with other Countries and Regional Bodies
Industry Canada consults with other countries and regional bodies to garner support for Canadian positions and to influence the positions put forward by regional blocks.

The regional block that Canada is a part of is the Inter-American Telecommunication Commission (CITEL). As part of the Organization of American States, it is the main forum in the hemisphere in which the governments and the private sector meet to coordinate regional efforts in telecommunications. Canada is currently one of the elected eleven Permanent Executive Committee (COM/CITEL) members of CITEL.

Depending on Canadian policy objectives and/or commercial needs and concerns, Canadian positions and proposals may be aligned with either the United States, countries of the Americas through CITEL common proposals, Europe on matters of mutual interest, the Asia Pacific Telecommunity countries under the auspices of CITEL or ISACC, or others.

Industry Canada also seeks support for positions and proposals through:

• bilateral negotiations in select countries;
• multilateral negotiations; and
• the development of common objectives within the Commonwealth ITU Group.

The output of this activity is Regional and/or Country Consensus. This consensus can be evidenced through the establishment of CITEL Inter-American Proposals and positions to be presented at the ITU. In other cases, Canada can be assured of the support of other regional member countries for its positions.

Process Satellite Applications and Terrestrial Frequency Assignments
Industry Canada reviews satellite network licensing applications submitted by industry; submits coordination and notification information to ITU; and completes ITU coordination and notification procedures to secure international rights for the use of spectrum and orbit.

For terrestrial networks, Industry Canada submits notification information and completes the relevant ITU procedure to secure international rights for the use of spectrum.

The outputs of this activity are Satellite Filings and Spectrum and Terrestrial Notifications to ITU. Essentially, Industry Canada notifies the ITU of the planned usage of satellite orbits and spectrum usages according to Canadian interests.

Prepare Submissions, Participate in Conferences, Meetings, Study Groups, Assemblies, and Chair Committees
Industry Canada, in representing Canada, participates in the following ITU activities:

• Radiocommunication Sector (ITU-R):
  • World Radiocommunication Conferences, Radio Assemblies, Radiocommunication Advisory Group and Study Group meetings and related activities
• Telecommunication Standardization Sector (ITU-T):
  • World Telecom Standards Assemblies, Telecommunication Standardization Advisory Group and Study Group meetings and related activities
• Telecommunication Development Sector (ITU-D):
  • World and Regional Telecom Development Conferences, Telecommunication Development Advisory Group and Study Group meetings, and related activities

While Industry Canada does not participate in all ITU activities, it strategically selects those that are most important to Canadian interests, as determined through the CNO consultative process. Industry Canada prepares proposals and positions on regulatory and procedural issues and participates in ITU conferences and meetings to secure and facilitate Canada's access to orbital and spectrum resources and to promote Canada's interests in global standards development and interoperability.

In some cases, Industry Canada advocates a position that will be beneficial to Canadian businesses or protects Canadian interests. In other cases, Industry Canada attempts to anticipate future requirements and stake out territory for Canada.

The outputs associated with this activity are submissions, positions, contributions and proposals. In different formats, this is the Canadian position on particular issues that are presented to the ITU.

Develop Canadian Standards, Regulations and Licensing Procedures
Industry Canada develops Canadian standards, regulations and licensing procedures to harmonize with the ITU, when it is required to do so under treaty with the ITU or when there are benefits for Canadian businesses or spectrum users. Canada as a signatory to the ITU is required to implement its treaties domestically.

Some examples of how international regulations affect Canadian standards, regulations and licensing procedures include:

• The ITU Radio Regulations form the basis for spectrum policy and technical standards in Canada (e.g. cellular and PCS services);
• Coordination of satellite services is governed exclusively by the ITU Radio Regulations; and
• Interference criteria developed in the ITU-R are the basis of Industry Canada coordination processes.

The outputs associated with this activity are Canadian Standards, Regulations and Licensing Procedures.

Immediate Outcomes (ITU)

The first level of outcomes, associated with Industry Canada's involvement in the ITU, are actions taken by the ITU:

• The ITU Adopts Binding Regulations;
• The ITU Allocation of Radio Spectrum;
• The ITU Develops Global Technical Standards; and
• The ITU Registration of Satellite Orbits and Frequency Assignments.

In the first three areas, adoption, allocation, and development, Industry Canada is looking for outcomes that are consistent with Canadian interests. The last area, registration, assures the protection of Canadian spectrum uses.

Immediate Outcomes (Canada)

The actions of the ITU have direct consequences for Canadian businesses and spectrum users. The following are the immediate outcomes (Canada) that are expected to be realized:

Canadian Companies Exploit Existing/Develop New Services and Technologies
There are two aspects to this outcome. The first aspect is the exploitation of existing technologies. Industry Canada will often promote positions at the ITU that will allow Canadian companies to exploit their existing technologies. On the other end, once ITU regulations or standards are developed, Canadian firms can target their R&D to technologies that will be consistent with the ITU standards and regulations.

Protection and Access of Spectrum Consistent with Canadian Interests
When the ITU allots spectrum, this protects the use of that spectrum for existing Canadian users or it may provide access for new uses. When allocation for new uses occurs, Canadian companies may be able to take advantage of their existing technologies to create new applications and services.

Economies of Scale for Canadian Companies
When ITU standards and regulations are developed, it allows Canadian businesses to develop and manufacture products consistent with those standards and regulations. Because these standards and regulations are recognized around the world, it allows Canadian companies to develop a product that can be sold worldwide, resulting in economies of scale and increased market access.

Connectivity and Interoperability of Global Communications Networks and Services
Connectivity refers to the ability of telecommunication devices and networks to connect with devices and networks in other countries. For example, Canadians can use their landlines, cellular or Internet phones, to contact parties in other countries. They can also use their computers to access web sites in other countries. Interoperability refers to the ability to use telecommunication devices in other countries. For example, Canadians can use their cell phones in Europe and RIM can sell their Blackberries in many countries around the world. Interoperability also ensures that the networks in different countries can interconnect and interwork with each other. Binding regulations, allocation of radio spectrum, and global technical standards all facilitate connectivity and interoperability.

Protection and Access of Satellite Orbits and Frequency Assignments Consistent with Canadian Interests
Canadian registration of satellite orbits and frequency assignments connotes the right of those registered to use those satellite orbits and frequency assignments. This then protects Canadian users from others who may wish to use these orbits and frequency assignments in a way that would interfere with the Canadian use.

Intermediate Outcomes

The immediate outcomes for Canada lead to a series of intermediate outcomes:

Increased Sales for Canadian Businesses to International Markets
When Canadian companies are able to exploit their technologies for application in the global market or target their R&D to this market, it is then possible to expand their market potential and increase their sales to international markets. Furthermore, economies of scale allow them to be competitive in a global market, further leading to increased international sales.

Countries around the World Serve the Canadian Market
Connectivity and interoperability means that many of the products of foreign telecommunication companies will function in Canada and as a result, many of these companies will sell their products directly or indirectly in Canada, thus ensuring the widest possible choice of communications technologies and products for Canadian consumers.

Interference Managed and Communication Facilitated
The Protection and Access of Spectrum Consistent with Canadian Interests ensures that other uses of spectrum do not interfere with the Canadian uses and thus communication is facilitated.

Connectivity and interoperability also serve to facilitate communication. Finally, the protection and access of satellite orbits and frequency assignments also prevents interference with Canadian uses, further facilitating communication.

Ultimate Outcomes

There are four broad outcomes from Industry Canada's involvement in the ITU:

Globally Competitive Telecommunications Carriers, Suppliers and Manufacturers
A larger market, global interoperability and resultant increased sales are expected to lead to Globally Competitive Telecommunications Carriers, Suppliers and Manufacturers.

Lower Costs and More Technologies Made Available to Canadians
With manufacturers around the world being able to serve the Canadian telecommunications market, Canadians benefit from lower costs due to competition and economies of scale. Consumers also benefit from access to new technologies developed around the globe.

Other Government Department Outcomes That Benefit Canadians
There are a myriad of benefits that accrue to Canadians through other government departments' use of spectrum. These include national defence, public safety, law enforcement, weather prediction, civil aviation, space program, maritime safety, etc.

Communication, Broadcasting Benefits to Canadians
With protected spectrum and a lack of detrimental interference, Canadians benefit from the unfettered use of their telecommunication devices, systems and networks and the enjoyment of broadcasting services.

Industry Canada's Strategic Objectives

Under Industry Canada's Program Activity Architecture, ITU activities are expected to contribute to the Industry Canada Strategic Outcome of a Fair, Efficient and Competitive Marketplace. This is expected to be realized through both globally competitive telecommunications carriers, suppliers and manufacturers, as well as lower costs and more technologies made available to Canadians. However, an argument can made that ITU activity also contributes to the Industry Canada Strategic Outcome of an Innovative Economy as a result of the ultimate outcome of Globally Competitive Telecommunications Carriers, Suppliers and Manufacturers. Furthermore, other government department Outcomes That Benefit Canadians and the Communication, Broadcasting Benefits to Canadians can be seen to logically lead to the sustainable communities' aspect of the Strategic Outcome, Competitive Industry and Sustainable Communities.

The graphical logic model is portrayed in the Figure 2 below.

Figure 2: Graphical Logic Model

Description Link

ITU Accountability and Performance Measurement Activities

International Organizations (IO) have internal accountability and performance measurement functions on which member countries can rely. However, Canadian departments contributing to those IOs must assure themselves that these accountability and performance measurement functions satisfy their own requirements. Otherwise, they must be supplemented by additional Canadian accountability and performance measurement activities.

In the case of the ITU, Industry Canada functions involved in the ITU report that the ITU has stringent internal controls and internal mechanisms which include audit, evaluation, inspection and investigation services that ensure its visibility and transparency. For example, the ITU Financial Operating Report, which has to be approved by the ITU Council.

The ITU has annual audits conducted by an independent External Auditor. The external Auditor is the Auditor General of the host nation, the Swiss Confederation. Through analysis of the ITU's financial transactions, evaluation of its activities and functions, the External Auditor offers comments and recommendations for improvement of financial and operational aspects, control mechanisms, and aspects of economy, efficiency and effectiveness of activities and operations of the ITU.

The External Auditor issues an annual report on the audit of the ITU, which is submitted to the ITU Council for follow-up action. There is also an internal control and audit function governed by the relevant provisions of the Financial Regulations.

The ITU is also subject to oversight by the Geneva Group, a policy and governance oversight mechanism. The Geneva Group is comprised of 14 countries, representing the largest "assessed" contributors to the UN system including Australia, Canada, France, Germany, Italy, Japan, Switzerland, the United Kingdom, and the United States.

In reviewing recent audit reports, the evaluators found that the primary focus has been on financial accountability. There does not appear to have been significant evaluation activity except perhaps on processing and procedural efficiencies. For example, the Kyoto Plenipotentiary Conference in 1994 adopted the first-ever Strategic Plan for ITU, which advocated a more client-oriented approach and a program of activities centered around the changing roles, needs and functions of ITU members.

In terms of performance monitoring, Canada has been instrumental in pushing results-based management concepts at the ITU. For example a Canadian proposal to the 2006 ITU Plenipotentiary Conference for the ITU to implement results-based management at the earliest opportunity, was adopted.

The ITU has set strategic goals and key expected results and is now developing key performance indicators. However, the goals and performance indicators focus primarily on the ITU functions of participation and coordination and are mostly irrelevant for evaluating Canada's participation in the ITU.

In conclusion, the ITU appears to have solid financial accountability. However, the performance measurement and evaluation functions of the ITU appear limited at this time and do not seem to be sufficient to meet Industry Canada's requirement to measure the value of its participation in the ITU.

³Evaluability Assessment of IC's Involvement in the ITU, Industry Canada, 2007 (Return to text.)

⁴International Telecommunication Union (Return to text.)

⁵International Telecommunication Union: Membership overview (Return to text.)

⁶Deck—ITU Overview, Industry Canada-SITT, March 2007. (Return to text.)

⁷Note: under the ITU, system, Member States are required to commit themselves to a level of contribution which is based on a unit system, and not to a specific annual contribution amount. (Return to text.)

Evaluation Methodology

2.1 Evaluation Issues

The Evaluability Assessment⁸ (EA) which was conducted in 2007–2008 identified evaluation issues and potential evaluation methodologies. The Federal Accountability Act requires that contribution programs be evaluated both in regards to relevance and effectiveness. Evaluation issues were derived from three sources:

1. Industry Canada program staff;
2. stakeholders involved in the ITU in Canada; and,
3. standard evaluation issues identified in the Treasury Board Evaluation Policy.

The final list of evaluation issues was confirmed with the Director General of the Audit and Evaluation Branch. The following are the evaluation issues and questions for this evaluation study:

2.2 Methodology

This section provides an overview of the methodologies that were employed in conducting this evaluation.

2.2.1 Document Review

The objective of the document review was to provide a line of evidence from available secondary data and to seek independent, objective information, where possible. Over fifty documents were reviewed under the following broad categories:

• Program documentation providing overviews of Industry Canada's ITU activities
• Narrative and statistical reports on Industry Canada's ITU activities
• Central agency documents related to Industry Canada's ITU activities
• General documents such as the Departmental Performance Report and Report on Plans and Priorities
• General Government of Canada documents such as the Speech from the Throne
• External documents from ITU Secretary-General and OECD
• Industry Canada ITU post-conference reports (see a fuller description in section 1.2.2.)

2.2.2 Review of Post-Conference Reports

When Industry Canada presents Canadian positions to the ITU, the intended immediate outcome is that Canadian positions will be adopted. (Please refer to section 1.3 for a complete description of intended Industry Canada ITU outcomes.) In fact, the Evaluability Assessment⁹ proposed that an appropriate performance indicator related to this outcome would be the percentage of Canadian positions that are adopted. In the development of the Evaluability Assessment, program staff had suggested that this information could be gleaned from Industry Canada ITU post-conference reports that are completed after each ITU conference. Based on this advice, the evaluation team reviewed the following Canadian Delegation Post-Conference reports:

• World Radio Communication Conference–2000, Istanbul, Turkey (2000)
• ITU World Telecommunication Development Conference (WTDC–02) Istanbul, Turkey (2002)
• ITU Plenipotentiary Conference–2002, Marrakech, Morocco (2002)
• World Radio Communication Conference (WRC–2003), Geneva (June 9—July 4, 2003)
• World Telecommunication Standardization Assembly (WTSA–04), Florianopolis, Brazil (2004)
• ITU World Telecommunication Development Conference (WTDC–06), Doha, Qatar (2006)
• ITU Plenipotentiary Conference (PP–06), Antalya, Turkey (2006)
• ITU World Radio Communication Conference (WRC–07),Geneva (2007)
• ITU World Radio Communication Conference Executive Summary (WRC–07), Geneva (2007)

The delegation reports were not all written in the same format so it was necessary to design two methodologies to review them.

The post conference report for the 2007 World Radio Communication Conference provided an assessment of Canada's success in relation to each agenda item. For each item, Canada's success was documented as "completely satisfied", "partially satisfied", "not satisfied", "not applicable", or "no change". Therefore, the evaluation team was able to calculate a percentage of the Canadian agenda items that were completely and partially satisfied. Additionally, a pre-conference briefing was obtained which outlined the responsibilities and functions of the delegations and provided nine issues of priority interest to Canada. This pre-conference memo was compared to the Executive Summary of the WRC–07 to ascertain whether the objectives in the pre-briefing were met.

The other post conference reports reviewed did not provide a comprehensive comparison of pre-conference Canadian positions to post-conference results. Rather, they provided information on the nature of the outcomes and some anecdotal reports on the success related to Canadian positions. As a result, it was not possible to determine the percentage of Canadian positions adopted. However, the evaluation team was able to document the number and nature of successes that were documented in each report.

2.2.3 Program Manager Interviews

The objective of interviewing program managers was to obtain their perspectives on the evaluation issues and to collect all available objective data on outcomes achieved. While program managers may have a positive bias in their views regarding Industry Canada success in ITU activities, it was important to solicit their opinions and to collect evidence to document Industry Canada successes.

Section 1.2 outlines how ITU activities are undertaken with staff from six directorates in three branches in two sectors of Industry Canada.

2.2.4 Survey of Stakeholders

The purpose of the survey of stakeholders was to determine the degree to which Industry Canada's ITU activities impacted targeted stakeholder groups. Examples of targeted stakeholders are telecommunication manufacturers, telecommunication service providers, broadcasters, impacted associations and other affected stakeholders.

A sampling frame was developed based on all the individuals that the program areas had involved in their ITU consultations over the past five years. One major benefit of this sample was that the evaluation team could be confident that these individuals and their organizations were aware of the ITU and Industry Canada's activities in relation to the ITU. Moreover, since there were no benefits associated with their responses, stakeholders were expected to provide unbiased views of their experience with Industry Canada. To reduce the potential for bias, potential interviewees were informed that no remarks would be attributed to them directly in the report. The downside of this sample was that the individuals who had been involved in Industry Canada's consultation processes related to the ITU and their organizations may have been more positively impacted by Industry Canada's ITU activities than stakeholders not involved in Industry Canada's ITU consultations.

The program area provided a list of 117 individuals that had been involved in Industry Canada's ITU consultations. EKOS Research Associates were engaged to conduct the interviews. They attempted interviews with all 117 of these potential contacts. Of these 117 potential contacts, 13 were eliminated because of invalid contact information, duplicates, etc. This left 104 valid contacts. Of the 104 valid contacts, 51 resulted in completed interviews. (31 could not be completed after 10 attempts, 4 were unavailable, 17 refused and 1 was broken off.) The overall response rate to the valid contacts was 49%.

Interviewees from the stakeholder survey were

• telecommunication manufacturers (37%),
• telecommunication service providers (22%),
• knowledge and information support providers (22%),
• associations (10%),
• aircraft manufacturers (4%) and
• broadcasters (2%).

It should be noted that, in this report, all percentages cited in relation to the survey include only those who could answer the question, e.g., if 10% could not answer a specific question and 45% said "yes" and 45% said "no", then the report would indicate that 50% (who could answer the question) said "yes".

2.2.5 Follow Up Interviews

Follow up interviews were attempted using the same sample frame as the survey of stakeholders described in section 2.2.4. The purpose of these interviews was to obtain in-depth explanations of the nature of the impacts of Industry Canada's ITU activities on the respondents' organizations and clients. The survey of stakeholders was conducted separately because: there was a concern that combining the two questionnaires would create a lengthy interview process that would result in a higher drop off rate; and, by separating the two interviews, a senior interviewer could be assigned to the follow up interviews which required a greater level of probing and recording detailed responses.

As with the original survey, attempts were made to contact all 104 valid contact names. From these, 32 follow up interviews were concluded, including four respondents who did not participate in the original survey. The overall response rate was 30%. A lower response rate was expected in the follow up survey because of respondent fatigue from participating in the original survey. This was not a major concern because the objective was to collect some illustrative examples of how Industry Canada's ITU activities had impacted the respondents and their organizations and clients.

The follow up interviews were also conducted by EKOS Research Associates.

2.2.6 Interviews of Other Government Departments

The purpose of the other government department interviews was to determine the impact that Industry Canada's ITU activities have on other departments' mandates and clients. Industry Canada program management identified other departments that may be impacted or involved with Industry Canada ITU activities and the evaluation team attempted to conduct interviews with all of these individuals.

The evaluation team was able to conduct interviews with representatives of the following seven departments:

• Transport Canada (2)
• Environment Canada
• Department of Fisheries and Oceans (2)
• Canadian International Development Agency
• National Research Council
• Canadian Space Agency
• Department of National Defense

In total, nine interviews were conducted with interviewees from other government departments.

2.2.7 Interviews of ITU Officials and Representatives of other ITU Member Countries

In section 2.1, one of the evaluation issues identified was, "What leadership role is Canada playing at the ITU?" Interviews of ITU officials and representatives of other ITU member countries were identified in the Evaluability Assessment¹⁰ as a key methodology for addressing this question: ITU officials and representatives of other member countries are front line observers of Canada's performance in ITU forums and are best placed to provide opinions on Canada's performance and impacts.

The Evaluability Assessment¹¹ proposed that interviews would be completed to provide representation from ITU officials and representatives of all continents and including developed and undeveloped/under-developed countries. Program officials suggested that current ITU officials would likely be unable to participate in the exercise because this could be construed as a conflict of interest with their official duties at the ITU. As an alternative, it was suggested that attempts be made to interview retired ITU officials.

Program management provided a list of three retired ITU officials representing different ITU sectors and representatives of eighteen ITU member countries. Telephone interviews were completed with all three retired ITU officials and representatives of Armenia, Australia, Japan, Korea, Mexico, Senegal, Sweden and the United States.

2.2.8 Succession Planning Analysis

Succession planning and management involves an integrated, systematic approach aimed at identifying, developing, and retaining talent for key positions and areas in line with current and projected business objectives.

The evaluation team assessed the current state of Industry Canada's succession planning in relation to ITU activities using the Canada Public Service Agency (CPSA) Succession Planning and Management Guide.

The evaluation team used the following CPSA success criteria for evaluating the Succession Planning of Industry Canada–ITU activities:

• Senior executives communicate the importance of succession planning and management as an organizational priority and are actively involved in the process.
• Succession planning is aligned with business plans and to the broader HR planning process and activities, including performance, development, learning, and recruitment.
• A fair, accessible, and transparent process is used.
• Planning extends to all levels of the organization rather than remaining limited to senior executive positions. There is collaboration among key players and buy-in from stakeholders.
• Employees are assessed through multiple sources of data, and critical development opportunities are identified early.
• Experiential learning is encouraged and is supported by coaching and evaluation of progress.
• A range of developmental activities are employed. These activities are individually tailored to address gaps in skills and competencies.
• Competencies and skills for key positions are reinforced in various HR systems such as recruitment, learning, development, and performance management.
• Mechanisms are in place to ensure the full realization of employment equity and diversity goals.
• A good communications plan is in place.
• Managers and employees participate in workshops and information sessions to learn about the succession planning process and their role in it.
• Tools that support the process are easy to use and accessible.
• The process is ongoing, monitored, evaluated, and refined based on feedback from stakeholders, leading research, and new developments in technology.

To conduct an assessment against these criteria, the evaluation team conducted interviews of program managers and program staff involved in ITU activities and reviewed various HR documents and statistical profiles.

Following the CPSA guide, the evaluation team interviewed five program managers. Managers were interviewed to determine what succession planning activities had been implemented and staff were interviewed to determine the impacts of any succession planning activities. Table 3 on the following page outlines the distribution of staff interviewed by classification/level:

The evaluation team also conducted a document review specific to succession planning. The following types of documentation were reviewed:

• Industry Canada HR documentation to assess whether succession planning linked with business and to the broader HR planning process and activities, including performance, development, learning and recruitment.
• Employee demographics to identify whether there is a succession planning problem
• Conference participation to assess whether junior staff were being given opportunities for training and to acquire ITU related skills

2.3 Limitations of Methodology

As previously discussed, one of the limitations of the survey of stakeholders was that it may not be representative of all ITU stakeholders. The stakeholders that were interviewed had been involved in Industry Canada ITU consultations over the past five years and while they would have been knowledgeable of the ITU and Industry Canada's involvement, they and their organizations may have been more impacted by the ITU and Industry Canada's involvement than other stakeholders.

There was also a lack of objective data on the impacts of Industry Canada's ITU activities. As much as possible, the evaluation team relied on objective data but it was limited. In some cases, the impacts would be difficult to attribute to Industry Canada's ITU activities or stakeholders would be reticent to provide them, e.g., impacts on profitability.

These limitations are tempered by the fact that the Evaluability Assessment¹² identified that this is a low risk program in that stakeholders did not identify any critical issues facing it. As such, the EA suggested that a moderate evaluation effort would be appropriate. Furthermore, while the evaluation relied heavily on the subjective opinions of stakeholders, representatives of other government departments and retired ITU officials and representatives of other ITU member countries, it was expected that these groups would be unbiased in their opinions and would be quite willing to identify any shortcomings in Industry Canada's performance in relation to the ITU.

⁸Evaluability Assessment of IC's Involvement in the ITU, Industry Canada, 2007 (Return to text.)

⁹Ibid (Return to text.)

¹⁰Ibid (Return to text.)

¹¹Ibid (Return to text.)

¹²Ibid (Return to text.)

3.0 Evaluation Findings

3.1 To What Extent Is Industry Canada's Involvement in the ITU Consistent with Industry Canada's Mandate and Government Directions?

The evaluation team examined a variety of different documentation to ascertain whether Industry Canada's involvement with the ITU is consistent with Industry Canada's mandate and government directions. These documents included Treasury Board submissions, the Department of Industry Act, the Speech from the Throne and Industry Canada's Program Activity Architecture.

The Department of Industry Act states in section 6(e) whereby the minister shall: Take any action that may be necessary to secure by international regulation or otherwise, the rights of Canada in communication matters. The Minister has similar power described in section 5(1)(k) of the Radio Communication Act. Through participation in the ITU, Canada is able to take actions (by submitting and presenting Canadian positions) to influence international regulations and standards. These actions would obviously protect the rights and interests of Canada and Canadian businesses in communication matters.

Participation in the ITU is a lower level activity element in Industry Canada's Program Activity Architecture. It falls under the Sub-Activity of Spectrum/Telecom Management, the Program Activity of Spectrum, Information Technologies and Telecommunications Sector—Marketplace and the Strategic Outcome of a fair, efficient and competitive marketplace. Therefore, Industry Canada's ITU activities are consistent with its approved Program Activity Architecture.

In section 1.3 of this report, the description of the logic model outlines how Industry Canada's involvement with the ITU is consistent with all three of Industry Canada's Strategic Outcomes. The official MRRS strategic outcome related to the ITU activity is to achieve a fair, efficient and competitive marketplace. As described in section 1.3, this is expected to be realized through globally competitive telecommunications carriers, suppliers and manufacturers; as well as lower costs and more technologies made available to Canadians.

Industry Canada's involvement with the ITU also contributes to Industry Canada's Strategic Outcome of an Innovative Economy. This is as a result of the ultimate outcome described in section 1.3 of Globally Competitive Telecommunications Carriers, Suppliers and Manufacturers.

Finally, section 1.3 describes how Industry Canada's involvement in the ITU logically leads to communication and broadcasting benefits to Canadians and other government department outcomes that benefit Canadians which in turn lead to the sustainable communities' aspect of the Industry Canada's Strategic Outcome, Competitive Industry and Sustainable Communities.

Industry Canada's involvement with the ITU is also consistent with current Government directions as set out in the November 19th, 2008 Speech from the Throne. The following are highlights from the Speech which are consistent with Canada's participation in the ITU:

Expanding Investment and Trade

• Our Government will expand the opportunities for Canadian Firms to benefit from foreign investment and knowledge, while taking steps to safeguard consumers and our national security.
• Our Government is committed to seeking out new opportunities for Canadians and to promoting global prosperity through free trade.
• Our Government understands that advances in Science and Technology are essential to strengthen the competitiveness of Canada's economy. Our government will work with industry to apply the best Canadian scientific and technological know-how to create innovative business solutions.

Industry Canada "expand(s) the opportunities for Canadian Firms to benefit from foreign investment and knowledge, while taking steps to safeguard consumers and our national security ", through its participation in the ITU. IC is able to promote Canadian technology and develop contacts for Canadian businesses with foreign firms. Both actions increase the potential of Canadian firms to benefit from foreign investment. In addition, by participating in the development of international standards at the ITU and translating them into Canadian standards, Canadian firms can better develop products and technologies tailored to international markets. This also expands the opportunities for Canadian telecommunication firms to benefit from foreign investment. Finally, the Canadian telecommunications industry has been able to gain product and market intelligence through Canada's participation in the ITU allowing Canadian businesses to benefit from foreign knowledge.

Furthermore, the ITU is an international body that develops and promotes international standards. This facilitates free trade by ensuring interconnectivity and interoperability of global telecommunication networks and services. Standardization, interconnectivity and interoperability also lead to accelerated science and technology advances because of global suppliers and markets. Industry Canada's involvement in the ITU ensures that Canada can influence these standards so that Canadian firms can best exploit their existing technologies and products and so that they can also exploit new markets that arise from the development of new standards.

3.2 Is There a Continued Need for Industry Canada's Involvement in the ITU?

Findings related to this evaluation issue were based on evidence collected in the survey of stakeholders, the follow-up interviews, interviews of other government departments and interviews of program staff.

In the survey of stakeholders, a large majority of stakeholders suggested there would be moderate to large impacts on ITU stakeholders if Industry Canada no longer developed and submitted Canadian positions to the ITU:

• 86% of stakeholders believed that there would be a moderate or large impact to their organization if Industry Canada no longer developed and submitted Canadian positions to the ITU (26% moderate, 60% large).
• 98% of stakeholders believed that there would be a moderate or large impact to Canadian telecommunication carriers, suppliers and manufacturers if Industry Canada no longer developed and submitted Canadian positions to the ITU (26% moderate, 73% large).
• 98% of stakeholders believed that there would be a moderate or large impact to Canadian users of spectrum and satellite if Industry Canada no longer developed and submitted Canadian positions to the ITU (23% moderate, 76% large).

When asked about the importance of ITU activities, standards and regulations, 73% of surveyed stakeholders indicated that these activities were extremely important to their organizations. Further, 78% of the stakeholders pointed to Industry Canada's involvement with the ITU as being extremely important to their organizations.

Listed below are some comments from stakeholders, highlighting, from their viewpoints, why there is a continued need for ITU participation in the ITU:

• Without Industry Canada's involvement in ITU, Canada would be in the dark on many issues. This would eventually come back to haunt us. Industry Canada involvement allows industry to keep abreast of all the spectrum allocations locally and that is the biggest issue when it comes to licensing new services in Canada. If Canada just waited until the U.S. did what they wanted to do, and then followed their path, it would not likely be in their best interest.
• The industry benefits 100%. If Industry Canada were not involved, industry leaders would simply have no way of doing it themselves. The industry would not be able to function without their support.

Representatives of other government departments also saw a continued need for Industry Canada's continued involvement in the ITU:

• Nine out of nine interviewed believed that there would be an impact on their organization if Industry Canada no longer developed and submitted Canadian positions to the ITU. (Two believed the impact would be moderate and seven suggested the impact would be large.)
• All whom could answer the question (seven out of nine) believed that there would be an impact to Canadian users of spectrum and satellite if Industry Canada no longer developed and submitted Canadian positions to the ITU ( three believed that the impact would be moderate and 4 believed the impact would be large.)

Other government departments surveyed indicated that ITU activities, standards and regulations were important to their organizations, with more than half suggesting that it was extremely important to their organizations. Government departments interviewed suggested, for the most part, that Industry Canada's involvement with the ITU is extremely important, with only a few saying that it was just somewhat important. Most government departments interviewed believed that Industry Canada adequately represents their interests at the ITU, and most suggested that they benefit extremely from Industry Canada's involvement with the ITU. Government departments indicated that their clients benefit moderately from Industry Canada's involvement with the ITU, with a few indicating that they benefit significantly from this involvement.

Many other government departments commented that the benefit of Industry Canada's involvement derives from the knowledge and expertise that Industry Canada brings when negotiating on behalf of the other departments. Many departments are aware that Industry Canada represents more than one government department and know that if Industry Canada were not to represent them they would need to go to the ITU. Many departments mentioned that they would only be able to represent their own interests and not those of other government departments. To this end, departments believed that Industry Canada was able to represent diverse interests to a large extent. Many of the interests that government departments suggested were important were:

• protection of spectrum assignments/management;
• satellite collection of valuable data;
• guarantee of data quality (interference can degrade data quality);
• protection from private sector or industry interference on needed frequencies;
• creation of contacts and knowledge of the ITU by Industry Canada; and
• data collection experiments by Industry Canada that supported the claim that wireless internet interferes with weather radar (the experiments looked at ways to minimize the impact of interference on weather radar).

Industry Canada program managers suggested that Industry Canada's involvement with the ITU was in line with the mandate for a fair and efficient marketplace and the regulatory mandate under the Radio Communication Act. They suggested that if Canada did not participate in ITU activities then Canada would become an acceptor of standards rather than contributor which would weaken Canadian competitiveness. Industry Canada managers suggested that participation at the ITU is essential, as many issues must be negotiated through an international forum and could not be negotiated bilaterally.

3.3 To What Extent Does Industry Canada Achieve Its Intended Results?

To address this question it is necessary to refer to the logic model depicted in section 1.3. This section is organized along the outcomes described in section 1.3 along with some other results that would contribute to these outcomes. The findings in this section are based on the document review; survey of stakeholders; follow interviews; interviews of other government departments; and, interviews of retired ITU officials and representatives of other ITU member countries.

Consult with Stakeholders and Partners

As depicted on the logic model, Industry Canada must consult with stakeholders and partners because it cannot contribute to the outcome of Canadian companies exploiting existing/developing new services and technologies unless it successfully consults with Canadian companies to become knowledgeable of their existing technologies and those being developed.

As described in section 1.3, Industry Canada conducts a national ITU consultation process with more than forty companies in Canada along with other government departments and agencies, such as the Department of Foreign Affairs and International Trade, Environment Canada, Health Canada, the Canadian Space Agency, the Department of National Defence, the Canadian International Development Agency, Indian and Northern Affairs Canada, the Canadian Radio-television and Telecommunications Commission, Transport Canada, the National Research Council and the International Development Research Centre.

In developing a sampling frame for the survey of stakeholders, a list of 104 unique stakeholders was developed. The survey of stakeholders revealed that 98% of stakeholders believed that Industry Canada's ITU consultations were adequate or more than adequate.

The survey of stakeholders further revealed that:

• 83% of stakeholders believe that Industry Canada had represented their interests to an adequate or large extent (45% adequate, 38% large).
• 98% of stakeholders believe that Industry Canada had represented the diverse interests of Canadian stakeholders to an adequate or large extent (41% adequate, 57% large).
• 96% of stakeholders believe that Industry Canada had been moderately or very successful (26% moderately successful, 70% very successful).
• 91% of stakeholders believe that Industry Canada had had moderate or great success in representing the interests of their organization at the ITU (46% moderate success, 45% great success).
• 100% of stakeholders believe that Industry Canada had had moderate or great success in representing the diverse interests of Canadian stakeholders at the ITU (34% moderate success, 66% great success).

Some comments from the follow up interviews of stakeholders illustrate how Industry Canada consults with stakeholders:

• Industry Canada is involved with industry in the planning process (e.g., through consultations). They take the outcomes of discussions with industry leaders and represent them at the ITU.
• Industry Canada certainly makes an effort to share their findings with their organization. Industry Canada will gather information on their behalf and keep them informed of upcoming and past meetings. There is a real openness with Industry Canada and a sense that they have the company's interests in mind.
• Industry Canada works very closely with industry in general. The positions they have developed have been very supportive of the perspectives that industry brings to the table.
• Industry Canada does the best they can to balance the objectives of the various Canadian corporations.

In the case of other government department representatives, all those interviewed, except one, believed that Industry Canada had conducted adequate consultation with stakeholders. They also, for the most part, suggested that Industry Canada was able to represent the diverse interests of Canadian stakeholders at the ITU. Finally, they believed that, for the most part, Industry Canada was successful in representing the interests of their organization.

In summary, the evidence collected indicates that Industry Canada has been successful in consulting and representing the interests of stakeholders at the ITU.

Immediate Outcomes (ITU)

In section 1.3, the following four ITU intended immediate outcomes are identified:

• ITU Adopts Binding Regulations;
• ITU Allocation of Radio Spectrum;
• ITU Develops Global Technical Standards; and
• ITU Registration of Satellite Orbits and Frequency Assignments.

The first three outcomes are related to the success Industry Canada is able to achieve in getting its positions adopted at the ITU. One of the proposed performance indicators to measure these outcomes is the percentage of Canadian positions adopted. In the review of post-conference reports, the evaluation team found that in most post conference reports there is no matching of Canadian positions to outcomes; rather the reports provide anecdotal evidence of Canadian successes achieved. Therefore, it was not possible to calculate the percentage of Canadian positions adopted from these reports.

The post conference report for the ITU World Radio Communication Conference in 2007, however, did report on the degree of success that Industry Canada achieved in relation to each agenda item. While there could be multiple Canadian positions related to each agenda item, the report allowed an assessment of success in relation to agenda items. The WRC–07 Delegation report shows that 74% of Canada's agenda item positions were completely satisfied by the ITU outcomes, 19% were partially satisfied, 3% were not applicable, and 3% were not satisfied. In addition, a comparison of the Industry Canada's WRC–07 pre conference memorandum objectives to the post conference executive summary results revealed that all nine Industry Canada pre-conference priority objectives were satisfied at the conference.

Retired ITU officials and representatives of other member countries suggested that Canada has been good to excellent in advancing its positions at the ITU. Other government departments interviewed believe that Industry Canada has had moderate to great success in the influence of ITU regulations, the ITU allocation of Radio Spectrum and ITU global Technical standards.

In terms of ITU registration of satellite orbits and frequency assignments, Industry Canada analysis of ITU–R publication BR–IFIC reveals that over the past five years, Industry Canada has had 99% of its frequency assignments approved for which the process is completed. It should be noted that some successful filings were not successful on initial filings but were successful in subsequent filings. In addition, there are a number of filings that are currently not successful but they are still being pursued by Industry Canada and hence their final outcome is not yet known.

In summary, the evidence collected indicates that Industry Canada has been successful in achieving its intended immediate outcomes at the ITU.

Immediate Outcomes (Canada)

Section 1.3 outlines the following five intended immediate outcomes for Canada:

• Canadian Companies Exploit Existing/Develop New Services and Technologies
• Protection and Access of Spectrum Consistent with Canadian Interests
• Economies of Scale for Canadian Companies
• Connectivity and Interoperability of Global Communications Networks and Services
• Protection and Access of Satellite Orbits and Frequency Assignments Consistent with Canadian Interests

Findings in relation to each of these immediate outcomes are documented below:

Canadian Companies Exploit Existing/Develop New Services and Technologies

The survey of stakeholders revealed that 97% of stakeholders believe that Industry Canada has had moderate or great success in facilitating Canadian companies in exploiting their existing services and technologies or in developing new ones (42% moderate, 55% great). The following are some of the stakeholder comments that illustrate the nature of these impacts:

• The benefits would be to the extent to which Industry Canada successfully advocates a global, harmonized, standardized approach to certain mobile spectrum bands. That would be a benefit to his company because it would give them access to a bigger pot of technology.
• This year his company benefited by working with Industry Canada to have a new question adopted and studied at the ITU–R, which will hopefully result in some radio regulations for the implementation of wireless intercommunication systems onboard aircrafts.
• His company's profits depend on their ability to sell products that conform to standards. None of their customers buy exclusively from them. In order for his company to be able to sell their products, they have to be able to understand the standards, build products that conform to them and they have to be able to defend the way they conform. They have been able to do this very successfully. The success of their operations depends on conforming to the appropriate standards.
• Various study groups impact the activities of the radio communications division of the ITU. One study group in particular, the fixed study group, develops recommendations. Those fixed recommendations create the channeling plans that manufacturers use to build with. Canadian industry was a major contributor in developing those channeling plans and has been a major beneficiary. His company has been able to sell equipment that resulted from some of those new channeling plans.

Protection and Access of Spectrum Consistent with Canadian Interests

The survey of stakeholders revealed that 100% of stakeholders believe that Industry Canada has had moderate or great success in the protection and access of spectrum consistent with Canadian Interests (31% moderate, 69% great). Other government departments interviewed believe that Industry Canada has had moderate to great success in the protection and access of spectrum consistent with Canadian interests. Following are some illustrative comments from stakeholders:

• It is important for them to have spectrum and regulatory requirements that support their customers. Industry Canada works with them to ensure that this takes place. In the case of his company, getting 700 megahertz of radio frequency for public safety has been a definite benefit.
• Industry Canada was successful in obtaining a small new amateur radio band that will become available at the beginning of 2009. It was successful in defending their position on a classic legacy amateur radio band, at the last WRC–2007.
• The ITU is involved in the allocation of radio frequencies spectrum. Industry Canada's involvement with the ITU enabled his organization to secure more spectrum for public safety.
• The government was successful in getting the spectrum that we wanted to use for good technical reasons. This helped Canadian telecommunication companies to carry out there business. Industry Canada is responsible for our success.

Economies of Scale for Canadian Companies

The survey of stakeholders indicated that 91% of stakeholders believe that Industry Canada's ITU activities to influence ITU Global Technical Standards has had moderate or great success in leading to increased economies of scale for Canadian telecommunication companies (55% moderate, 35% great). Some of the comments from stakeholders which explain these impacts are listed below:

• Having harmonized spectrum will eventually enable his company to rationalize some of its products. They will no longer have to create a specific product because someone is using spectrum from Canada, the U.S., or elsewhere. The harmonization of spectrum makes it so that his company will not have to create a variety of products with radio frequency front ends for their product. This simplifies production.
• Basically they have had sales opportunities. They get economies of scale (selling larger volumes of equipment and meeting a common standard in a world wide sense, in which Canada is a party).
• The development of standards is an indirect benefit to his company cost wise. It means that his company only has to generate a single product worldwide, as opposed to generating unique products for different countries.

Connectivity and Interoperability of Global Communications Networks and Services

In a 2008 study^{Footnote 13} created by Leonard Waverman, professor of economics at London Business School, 16 innovation driven economies were ranked on a connectivity index based on thirty indicators of connectivity. Canada was ranked fourth overall of these 16 innovation driven economies.

The survey of stakeholders revealed that 91% of stakeholders believe that Industry Canada's ITU activities have had moderate or great success in facilitating connectivity and interoperability of global communications networks, products and services (38% moderate, 53% great). Some examples of the benefits cited by stakeholders are listed below:

• Industry Canada involvement had a significant impact on the decisions made in terms of telephone numbering systems. Basically it made international calling work more effectively and made it cheaper to implement.
• Industry Canada's participation in ITU–R is key in ensuring that the systems can function with other systems. This is because access to spectrum is protected and standards are set for the interoperation of the systems. This is a great benefit to clients.
• The simple fact that you can pick up a phone, dial a number and reach anyone in the world is a significant benefit. We are basically converting that network into a new Internet protocol based network. If Industry Canada wasn't there, we would not be able to feed Canadian requirements into that process.

Protection and Access of Satellite Orbits and Frequency Assignments Consistent with Canadian Interests

As previously discussed, over the last 5 years, Industry Canada has had 99% of satellite frequency requests (for which examination is completed) approved by ITU. Furthermore, the survey of stakeholders revealed that 96% of stakeholders believe that Industry Canada has had moderate or great success in the protection and access of satellite orbits and frequency assignments consistent with Canadian Interests (38% moderate, 31% great). Finally, other government departments interviewed suggested that Industry Canada has had moderate success in the protection and access of satellite orbits and frequency assignments consistent with Canadian interests. Listed below are some of the comments from stakeholders that illustrate the nature of these impacts:

• His business is in satellite communications. His company has a need to coordinate the frequencies that are used for satellite. The benefit to his company is that they can operate satellite systems; their satellites are coordinated and operate harmoniously with the other entities. Industry Canada's participation helps to protect the spectrum we have today and secure additional spectrum for future growth.
• His company fought for frequency allocations through Industry Canada at the ITU. They were successful and this allowed them to put products on the market for clients.
• The satellite folks push internationally to have orbit positions in the geo-stationary arc. His company, for instance, would like to have as many orbit positions as possible so that they can populate them with satellites to grow their business. This cannot happen without coordinating with other countries and insuring that there is no interference from his company's satellites to other satellites. Industry Canada facilitates the coordination to ensure that everyone can co-exist on the arc.

In summary, the evidence collected indicates that Industry Canada has been successful in achieving the intended immediate objectives in Canada.

Intermediate Outcomes

The logic model identified the following intended intermediate objectives from Industry Canada ITU activities:

• Increased Sales for Canadian Businesses to International Markets
• Countries around the World Serve the Canadian Market
• Interference Managed and Communication Facilitated

The findings related to each of these intermediate outcomes are outlined below:

Increased Sales for Canadian Businesses to International Markets

The survey of stakeholders shows that 94% of stakeholders believe that Industry Canada's ITU activities to influence ITU Global Technical Standards has had moderate or great success in leading to increased international sales (68% moderate, 26% great). Some specific related comments from stakeholders include the following:

• He thinks his organization has benefited specially in terms of Industry Canada's participation in ITU–R. This has helped facilitate common international definitions of uses and allocations of frequencies, which facilitates his company's sales opportunities in a worldwide sense. Manufacturers like to sell the same product worldwide. If they only had the opportunity to sell in Canada, it would be a cost penalty. We would really like to sell one product everywhere. He thinks that participation in ITU–R, with Industry Canada's assistance, has helped achieve that.
• His company's space missions benefited by being able to put products on the market that were fitting with international requirements.

The survey of stakeholders also shows that 81% of stakeholders suggested that the profits of their organization (those for which the question was applicable) had had a positive impact as a result of Industry Canada's involvement with the ITU (28% very small positive impact, 37% somewhat of a positive impact, 16% major positive impact). This is demonstrated in the following comments:

• Virtually all decisions at the ITU will affect their profits globally. The decisions that Industry Canada makes can influence ITU decisions and subsequently affect the industry profits in Canada.
• His company's profits are based on the sale of their systems to operators. His company has two kinds of customers: enterprises and public operators (e.g., Bell, Dell, Rogers, and others internationally). Their revenue comes from selling those systems to them. The sales would not be possible without the participation of Industry Canada in the ITU–R. His company could not do it alone.
• The revenues are impacted by decisions made at the ITU. Without their efforts in the area of frequency allocation, his company would not be able to do their job in terms of flight test certification and would not be able to sell their helicopters.

Countries around the World Serve the Canadian Market

The survey of stakeholders revealed that 94% of stakeholders believe that Industry Canada's ITU activities to influence ITU Global Technical Standards has had moderate or great success in benefiting Canadian consumers and business that may buy telecommunications equipment that is sold by international companies around the world (48% moderate, 46% great). One illustration of this is that wireless phone manufacturers headquartered in the United States, Japan, Finland, and the United Kingdom have their wireless phone products available in Canada.

Interference Managed and Communication Facilitated

During the last five years, Industry Canada has dealt with only five reports of harmful interference to or from non-Canadian sources. None of these cases resulted in harmful interference reports being sent through the ITU. One issue was not resolved, although the complainant indicated that the interference was not a significant concern. The other four interference complaints were either resolved on their own or through Industry Canada intervention.

The survey of stakeholders shows that 95% of stakeholders believe that Industry Canada has had moderate or great success in contributing to facilitating communication and managing interference in Canada (32% moderate, 63% great). Some illustrative comments from stakeholders included the following:

• Industry Canada's participation in ITU–R is key in ensuring that the systems they sell function without interference.
• The bulk of their revenues are still derived from orbital slots that are licensed by Industry Canada. The raw material for his business is having spectrum that they can exploit to offer services to their customers. It is critical that his company's rights to spectrum be solid and that they are not challenged by other administrations. They need to coordinate in such a way that they can operate without interference. Underlying that is Industry Canada's activities in the ITU.
• It is essential for his company to have access to good quality spectrum without interference from other systems. The benefit is being able to deploy those systems and take advantage of the revenue it creates for the company from the operators who buy them.

In summary, there is clear evidence that Industry Canada has largely been successful in achieving two of the intended intermediate outcomes and somewhat weaker evidence that Industry Canada has been successful in contributing to the outcome of countries around the world serve the global market.

Ultimate Outcomes

The logic model identified the following intended ultimate outcomes from Industry Canada's ITU activities:

• Globally Competitive Telecommunications Carriers, Suppliers and Manufacturers
• Lower Costs and More Technologies Made Available to Canadians
• Other Government Department Outcomes That Benefit Canadians
• Communication, Broadcasting Benefits to Canadians.

The findings related to each of these ultimate outcomes are outlined below:

Globally Competitive Telecommunications Carriers, Suppliers and Manufacturers

When stakeholders were asked in the survey, 90% of them believed that Canada's activities with the ITU had had moderate or great success in contributing to globally competitive telecommunications carriers, suppliers and manufacturers (48% moderate, 43% great). This was reflected in some of the comments in the follow-up survey:

• If Canada no longer participated in the ITU, it would definitely hurt the Canadian telecommunications industry;
• If Canada no longer participated in the ITU, the telecommunications industry simply could not function.

Lower Costs and More Technologies Made Available to Canadians

Many of the stakeholders interviewed were representatives of firms that manufacture telecommunications equipment or provide telecommunication services. As such, they are in a position to assess some of these impacts on Canadians who are their customers. The survey of stakeholders showed that 84% of stakeholders believe that Canada's activities with the ITU have had moderate or great success in contributing to lower costs in telecommunications equipment and more technologies being made available to Canadians (42% moderate, 42% great). Furthermore, 87% of stakeholders reported that their clients benefited moderately or greatly from Industry Canada's involvement with the ITU (39% moderately, 48% greatly). The cost and technology benefits to Canadians are elaborated in the following comments from stakeholders:

• Their clients definitely benefit from Industry Canada's involvement with the ITU. His company's products would end up costing more if there were more fragmented standards because Industry Canada was not involved with the ITU.
• His clients benefit because they have access to a greater variety of products, such as cellular handsets. They also have access to better pricing because of greater economies of scale.
• Spectrum users benefit from being able to access more product applications, a wider range of services, and better quality services.

Other Government Department Outcomes That Benefit Canadians

Industry Canada interviewed representatives of the following federal departments:

• Transport Canada;
• Environment Canada;
• Department of Fisheries and Oceans;
• Canadian International Development Agency;
• Department of National Defence;
• National Research Council; and
• Canadian Space Agency.

Most of these departments rely on reliable access to frequency either to conduct some of their work or to serve their clients. The following outlines some of the other government department outcomes that benefit Canadians:

Transport Canada

Transport Canada relies on spectrum to maintain marine and air safety in Canada. Industry Canada representation at the ITU ensures that this spectrum is protected and that regulations are consistent with Canadian interests. OGD interviews indicated that Industry Canada has had some success in putting forward Canadian positions at the ITU (e.g., Article 31 and 33 and recommendation 493 at the World Radio Conference 2007) and that, "All items they (Industry Canada) have worked on normally have a positive outcome". The result is that Canadians benefit through safer air and marine travel.

Environment Canada

Environment Canada depends on satellite data to predict weather. OGD interviews suggested that, without the participation at the ITU, activities such as forecasting severe weather, which can have a socioeconomic impact, would be affected. It was further suggested that Environment Canada benefited significantly from Industry Canada's involvement with the ITU. An example was provided of Industry Canada being successful in protecting Radar 5.6 and 6.65 which are important radar frequencies to detect severe weather: Industry Canada was a proponent of measures against the U.S. at ITU–R which were successful in preventing potential interference.

The OGD interviews indicated that weather forecasters, climatologist, hydrologists, and science and technology scientists were all clients of theirs that benefited from Industry Canada's involvement in the ITU. It would follow that farmers and those who benefit from accurate weather prediction would also indirectly benefit. Furthermore, Canadians at large would indirectly benefit by being aware and able to prepare for severe weather conditions.

Department of Fisheries and Oceans (DFO)

DFO relies on Industry Canada to protect frequencies in the marine bands and the OGD interviews indicated that Industry Canada has had great success in representing the interests of DFO at the ITU. An example of protecting the digital usage of marine band was cited. Industry Canada was able to facilitate a compromise in a proposal that protected regions of Canada where a new technology had not yet been adopted.

The OGD interviews further indicated that DFO clients are the shipping industry, boating (recreational), fishing vessels and that any changes in frequencies would be a long and painful process and lots of effort from its partners. These clients benefit from having secure access to marine bands to facilitate communication and to ensure safety.

Canadian International Development Agency (CIDA)

CIDA relies on Industry Canada in relation to telecommunications in the developing countries. The OGD interviews indicated that Industry Canada has had great success in representing the interests of CIDA at the ITU. An example was provided where Canada was competing to get a model approved to measure the digital divide. It was suggested that Industry Canada was instrumental in getting the Canadian model that CIDA financed accepted. It was further suggested that the direct benefit to Canadians was not great but that the Industry Canada work was important in supporting CIDA's mandate to support developing countries.

Department of National Defence (DND)

DND relies on Industry Canada to protect the use of military spectrum and systems. The OGD interviews indicated that Industry Canada has had great success in representing the interests of DND at the ITU. Reliable military spectrum and systems facilitates the success of Canadian combat and peace-keeping missions and ensures the safety of Canadian soldiers.

National Research Council (NRC)

The NRC relies on Industry Canada to protect and obtain spectrum space related to radio astronomy. The OGD interviews indicated that Industry Canada has had great success in representing the interests of NRC at the ITU. The benefit is to the Canadian Astronomical Community to be able to continue to do this kind of science. It was further indicated that this is a significant benefit and it is critical for it to continue.

Canadian Space Agency (CSA)

The CSA launch and operate satellites and it depends on Industry Canada to obtain its licenses through the ITU. The OGD interviews indicated that Industry Canada has been moderately to very successful in representing CSA's interests at the ITU. An example was provided of the 2007 World Radio Conference in which Industry Canada consulted with CSA on two aspects which resulted in great success. It was further indicated that the data collected by the satellites was extremely beneficial for the Canadian scientific community.

Summary

In summary, Industry Canada appears to assist many other departments in satisfying their mandates and benefitting Canadians directly and indirectly in many different ways. The representatives of those departments interviewed generally believed that Industry Canada had been successful in representing their interests at the ITU.

Communication, Broadcasting Benefits to Canadians

An objective measure is available relating to this intended outcome: In 2003, the ITU released a study^{Footnote 14} ranking countries on digital access. Of 178 countries, Canada was ranked 10th overall, the only country outside of Asian and Europe in the top ten. Furthermore, 94% of stakeholders interviewed believe that overall, Canada's activities with the ITU have had moderate or great success in contributing to communication and broadcasting benefits to Canadians (47% moderate, 47% great). Some of the comments from stakeholders illustrating the nature of these benefits are listed below:

• Amateur radio survives so long as the international frequency assignments for it continue to be upheld. If amateur radio does not have strong representation and a government that looks out for their interests, they run the risk of seeing large parts of their service deteriorated or even eliminated. Amateur radio benefited simply by being able to operate on the frequencies that they have had for decades.
• Influencing the decisions at the ITU has been tremendously important over the years. It is important to note that many of the decisions taken at the ITU are very long term in nature. Unless you have a management focus that is more long term as opposed to short term, you don't see the immediate benefits. We fought as a Canadian delegation for orbital positions and frequencies for broadcasting satellites, for instance, years before we had any broadcasting or satellite industry in this country. Now all Canadians who want to can benefit from the uses of satellite for their dishes. None of that would be possible if Canada did not have a strong presence in the meetings and conferences that establish the standards and also the frequencies and orbital positions that Canada now uses.
• If you look in your own house you have remotes for television and alarm systems. As we progress, there will be homes where people can control heating remotely. All these innovations occur because of radio spectrum becoming available. This helps to better the lives of users. This is all possible because of what takes place at the ITU. Industry Canada had played a significant role is shaping how spectrum is allocated.

In summary, the evidence collected indicates that Industry Canada has been successful in its interactions with the ITU such that they have led to communication and broadcasting benefits for Canadians.

Strategic Outcomes

As previously discussed, under Industry Canada's Program Activity Architecture, ITU activities are expected to contribute to the Industry Canada Strategic Outcome of a Fair, Efficient and Competitive Marketplace. The logic model described in section 1.3 shows that the ITU activity is also expected to contribute to the Industry Canada Strategic Outcomes of an Innovative Economy and Competitive Industry and Sustainable Communities.

There are two Industry Canada 2008–2009 MRRS performance indicators that would be relevant to ITU activities, specifically the following:

• Barriers to competition (OECD assessment of accessibility to Canadian market).
• International ranking of Canada in the use of information and communications technologies.

However, our research revealed that this data is no longer being collected by the external sources. Therefore, we are unable to find any direct evidence of Industry Canada's ITU activities contributing to its Strategic Objectives.

In section 1.3, the logic of how the Strategic Outcomes are expected to be achieved was explained:

• Fair, Efficient and Competitive Marketplace: This is expected to be realized through the lower level outcomes of globally competitive telecommunications carriers, suppliers and manufacturers; and lower costs and more technologies made available to Canadians.
• Innovative Economy: This is expected to be realized as a result of the ultimate outcome of globally competitive telecommunications carriers, suppliers and manufacturers.
• Competitive Industry and Sustainable Communities: The other government department outcomes that benefit Canadians and the communication and broadcasting benefits to Canadians can be seen to logically lead to the sustainable communities' aspect of this Strategic Outcome.

In summary, while there is no direct evidence that the ITU activities contribute to Industry Canada's Strategic Outcomes, the evidence collected in this evaluation that lower level outcomes are being achieved and the program logic would lead the evaluators to conclude that the ITU activities are contributing to the achievement of the outlined Strategic Outcomes.

Footnotes

1. ^{13 back to footnote reference 13} Benefiting from the full economic and social impact of affordable ICTs, Ilkka Lakaniemi, Head of Global Political Dialogue and Initiatives, Corporate Affairs, Nokia Siemens Networks, 2008
2. ^{14 back to footnote reference 14} Press Release by the ITU, November 19, 2003: ITU Digital Access Index: World's First Global ICT Ranking

3.4 What Leadership Role Is Canada Playing At The ITU?

The primary source of evidence related to the leadership role played by Canada at the ITU was through the interviews of retired ITU officials and representatives of other ITU member countries at the ITU. These individuals believe that Canada has a medium to high level of influence at the ITU. They suggested that this is based on the merit of Canada's proposals and its high quality and well-known representatives.

Retired ITU officials and representatives of other ITU member countries further believe that Canada plays a medium to very strong leadership role at the ITU. They suggested that this is evidenced by the important roles including chairmanships played by Canadians and is based on the high quality, hard-working Canadian delegates. Many surveyed stakeholders and representative of other government departments also suggested that this leadership role is evidenced by the important positions Canada has achieved at the ITU. Table 4 below outlines many of the key leadership positions Canada has achieved in relation to the ITU in recent years:

Retired ITU officials and representatives of other member countries also believe that Canada is well respected by other ITU members. They suggested that this respect is based on the balanced positions Canada presents, the high quality of its delegates and the ability of Canadian delegates to speak both French and English.

Finally retired ITU officials and representatives of other member countries believe that Canada is good to excellent in its ability to find middle grounds and form consensuses at the ITU. This is based on the influence of Canada's delegates and its ability to be flexible.

The survey of stakeholders also revealed that 90% of stakeholders believe that Canada's influence has been large or extremely significant relative to other countries at the ITU (35% large, 55% extremely significant). Some comments below illustrate the nature of this influence relative to other countries:

• His company was a key player on the world stage for global standards. It is through Industry Canada's involvement in the ITU and support of their activities that has allowed his company to maintain that highly visible role. We punch well above weight-in terms of the size of the country and the size of the company. With Industry Canada's support, we manage to be way more effective on a per capita per dollar invested basis than many of our competitors.
• Industry Canada is respected throughout the world. They always seem to be a body that can work out negotiations amongst all of the different countries. Canada does a good job of working out all sides to the issue. Industry and users benefit from good, solid regulations that are developed at the ITU–R with the help of Canada and eventually implemented nationally.

Representatives of other government departments have varying beliefs on how much influence Canada has relative to other countries at the ITU. For a few they feel it is commensurate with its size, however, for the most part they feel that for Canada's size, its influence has been extremely significant.

3.5 What Value Does Industry Canada's Involvement in ITU Provide?

Evidence addressing this issue was primarily collected through the stakeholder survey, the follow up interviews, and interviews of other government departments.

Stakeholders surveyed, when informed of the cost of Industry Canada's involvement in the ITU, suggested that there is very good (47%) to excellent value (47%) for money in Industry Canada's involvement with ITU for Canadian telecommunication carriers, suppliers and manufacturers. Further, they also believed that these expenditures were very good (42%) to excellent (51%) value for Canadian users of spectrum and satellite.

Stakeholders indicated that Industry Canada was moderately successful (45%) and very successful (38%) in terms of representing their organizations' interests at ITU. They also suggested that Industry Canada was able to represent the diverse needs of the Canadian stakeholders to an adequate extent (41%) and large extent (57%). Moreover, stakeholders indicated that Industry Canada was moderately successful (26%) and very successful (70%) at balancing the competing interests of different stakeholder groups.

When asked about the benefit organizations realized from Industry Canada's involvement with ITU, the stakeholders surveyed indicated that representing/mediating Canadian interests (40%) was the greatest benefit. Additionally, stakeholders recognized other benefits such as spectrum allocation (21%), creating standards/regulation, global alignment/harmonization (17%), access to/protecting frequency and satellite orbit (15%), spectrum/satellite management/harmonization (15%) and access to information (10%).

To assess the value of Industry Canada's participation in the ITU, stakeholders were asked a number of questions to determine the level of success or impacts that their organization has realized as a result of Industry Canada's participation at ITU. The following is a summary of the findings in relation to these questions:

• The survey of stakeholders shows that 81% of stakeholders suggested that the profits of their organization (those for which the question was applicable) had had a positive impact as a result of Industry Canada's involvement with the ITU (28%—very small positive impact, 37%—somewhat of a positive impact, 16%—major positive impact).
• 96% of stakeholders surveyed indicated that Industry Canada has been successful in the protection and access of satellite orbits and frequency assignments consistent with Canadian interests (38%—moderate success, 58%—great success).
• 100% of stakeholders believe that Industry Canada has been successful in the protection and access to spectrum consistent with Canadian interests (31%—moderate success, 69%—great success).
• 90% of stakeholders indicated that Industry Canada's ITU activities had been successful in leading to increased economies of scale for Canadian Telecommunication companies (55%—moderate success, 35%—great success).
• 94% of stakeholders believe that Industry Canada's ITU activities were successful in contributing to increased international sales (68%—moderate success, 26%—great success).
• 88% of stakeholders indicated that Industry Canada's ITU activities have been successful in benefiting Canadian consumers and business that may buy telecommunications equipment that is sold by international companies around the world (55%—moderate success, 33%—great success).
• 93% of stakeholders believe that Industry Canada has been successful in facilitating connectivity and interoperability of global communication networks, products and services (48%—moderate success, 45%—great success)
• 95% of stakeholders indicated that Industry Canada's was successful in facilitating communication and managing interference in Canada by influencing ITU regulations, the ITU allocation of Radio Spectrum and ITU Global Technical Standards (32%—moderate success, 63%—great success).
• 91% of stakeholders indicated believed that Industry Canada ITU activities have been successful in contributing to globally competitive telecommunications carrier, suppliers and manufacturers (48%—moderate success, 43%—great success).
• 82% of stakeholders believe that Industry Canada's ITU activities have been successful in contributing to lower cost telecommunications equipment and more technologies being made available to Canadians (41%—moderate success, 41%—great success)
• 93% of stakeholders suggested that Industry Canada's activities had been successful in contributing to communications and broadcasting benefits to Canadians (53%—moderate success, 40%—great success).

Listed on the following page are some illustrative comments from stakeholders related to the value that Industry Canada's involvement in the ITU provides:

• He thinks that this is money well spent. If Industry Canada did not spend this money, they wouldn't be able to take a lead role.
• In his personal view, the money is well spent. Face to face meetings are very important in this context. Smaller organizations like his do not always have the money to send people to attend conferences. He has to depend on Industry Canada to have his voice heard. He feels that they have been very good at representing groups like his.
• He cannot think of any better way to spend this money to represent the Canadian telecommunications industry and users of spectrum and satellite.
• He thinks that the money spent is spent appropriately and supports the telecom industry. While things are not as bright as they once were, telecom remains one of the bright spots in Canadian industry. Industry Canada's participation in ITU activities is fundamental to the future of the economy in general because telecom is one of the more successful aspects and it is one of the bootstrapping structures that make any economy go.

It should be noted that six of the sixteen stakeholders interviewed in the follow-up interviews suggested that Industry Canada should be spending more money in relation to its ITU activities to further increase its effectiveness.

ITU officials and representatives from other ITU member countries suggested that Canada has a bigger influence than one would expect for a country of Canada's size. They also recognized that Canada has held many leadership positions in terms of chairing committees. They suggested that Canada is recognized for its flexibility to modify its position and compromise and its ability to build consensus. Finally, they indicated that the Canadian team is highly respected and has a lot of weight when decisions are based on the merit of its proposals.

Representatives of other government departments that were interviewed generally believe that there is good value provided to Canadians in relation to the cost to participate in the ITU. (Six out of seven that could answer the question and who were informed of the cost of Industry Canada's involvement in the ITU suggested that Industry Canada's expenditures are very good value (four) to excellent value (two) for Canadians in relation to the cost.) Similarly, seven out of eight of these representatives that could answer the question also suggested that it is very good (three) to excellent value (four) for Canadian users of spectrum and satellite.

3.6 Does Industry Canada Have an Adequate Succession Plan to Ensure Continued Influence at ITU?

Succession planning and management involves an integrated, systematic approach aimed at identifying, developing, and retaining talent for key positions and areas in line with current and projected business objectives.

This section of the evaluation examines the program delivery aspect of Industry Canada's involvement in the ITU related to issues of whether there is a succession plan in place to support the continuity of the program, and if so, how adequate it is to ensure the continued influence at the international level. Findings are derived primarily from document reviews, survey of stakeholders, ITU official telephone interviews, other government department interviews, program manager interviews; and telephone interview and survey of staff members.

3.6.1 Demographic Profile of Executives and Engineers in Spectrum, Information Technologies and Telecommunications (SITT) Branch

This analysis focuses on the demographic information of Executive and Engineer groups which were identified during the Evaluability Assessment in 2007–2008. This profile provides a demographic overview of these two groups in SITT as of March 2008. It not only depicts the current Spectrum, Information Technologies and Telecommunications (SITT) work force for these two groups but also provides a comparison analysis with the federal public service as a whole. The rationale for looking at SITT instead of the six directorates in the three branches involved in ITU activities is that it is not feasible to analyze the demographic information at the branch level based on the ITU involvement. For example, for Radiocommunications and Broadcasting Regulatory Branch, only the Space and International Regulatory Activities directorate which consists of one executive is actively involved in ITU activities. It would be misleading to compare the age and the percentage of those eligible to retire with such small representation. Furthermore, based on the interviews with the program managers and staff, the succession planning is a strategic exercise; and it would be more useful to analyze the data for the SITT sector as a whole.

Demographic Profile of SITT as of March 31, 2008

• 764 indeterminate employees
• Average age 45.4
• 25 Executives (25 Executives and 35 EX Equivalent)
• Average age of executives 54
• 210 Engineers
• Average age of engineers 43.3

Table 5 below shows the average age and the retirement eligibility rate between employees of Public Service of Canada and SITT in the Executive and Engineer categories.

The Executive Group

At March 31, 2008, there were 60 executives in SITT. The average age of this group was 54.6, of which 68% was over 50 years old. By comparison, the average age was 50.3 for all executives in the federal public service and 48.9% of them were over 50 as of March 31, 2008.

In March 2008, approximately 36% of the SITT executives were eligible to retire while the figure for all public service executives was 16.4%. Along the same notion, SITT will have 60% of its executives eligible to retire by 2012 compared to approximately 37.8% of the executives for the public service of Canada retirement eligibility for the same period.

The Engineer Group

As of March 2008, SITT had 764 indeterminate employees, of which 210 were engineers (27.5%). (The Spectrum Engineering Branch had 68 Engineers representing 55.7% of the branch work force).

The average age of 43.3 for SITT engineers was a little younger than that of the public services as a whole (45.2). However, 12.3% of this group was eligible to retire in March 2008 and over 25% of them will be eligible to retire by 2012. Compared to the public service as a whole, the retirement eligibility rate was 8.4% for March 2008 and 22% of the employees will be eligible to retire by 2012.

In summary, the demographic profile reveals that a considerable proportion of executives and staff will be eligible to retire in the near future, indicating a potential succession issue.

3.6.2 SITT HR Plans

The SITT Human Resources Plan for 2007–2010 identified four over-riding priorities. One of the four priorities is to develop succession and development plans to address planned departures in key regulatory and research functions.

The SITT HR Plan for 2008–2011 indicated that succession planning is the key to maintaining a knowledgeable and skilled workforce in SITT, therefore developing and implementing a robust succession plan was identified as a top priority for 2008–2009 and beyond. The plan further identified that it will be important for SITT to develop a succession plan over the coming months/years to ensure a highly qualified replacement personnel are at the ready for critical and/or specialized positions.

At the time of this report, a succession plan for SITT had not yet been developed.

3.6.3 Existing Gap in Age and Skill Sets

In the survey of stakeholders, 71% of stakeholders believed that Industry Canada sends enough junior staff to ITU meetings to ensure an ongoing high-level contribution. 29% believed that Industry Canada send no junior staff, send few junior staff or infrequently send junior staff. Some of the specific comments from stakeholders in the follow-up interviews included the following:

• There are relatively few junior people being trained to partake in the process. He has been at this a long time and most of the Industry Canada people that were there when he started are still there. They are all just getting older.
• Many of Industry Canada's very experienced people are retiring. There was a period in the 1990s when they hardly hired anyone. There is a huge age gap in the staff. There is no one in their mid 40's who has built a career there. They started hiring again in 2000. There are now people in their late 20s and early 30s coming up behind. The challenge is to get younger staff up to speed on the dynamics and the negotiations (the technical part is easy to learn). There are going to be some rough years ahead in terms of Industry Canada's participation in the ITU, which will be based on younger staff learning.
• Succession planning is a concern. He would like to see Industry Canada develop people to step up and take on the roles of those who will be retiring. This is an area that Industry Canada should pay more attention to.

Retired ITU officials and representatives of other ITU member countries reported that Canada has a very strong technical capability compared to other countries. They also suggested that Canada also has very committed, highly regarded delegates such as Bruce Gracie, Marc Dupuis and Bill McCrum who have great influence at the ITU. They suggested, however, that there is a dip in expertise when someone leaves. This is not only for Canada but all countries experience this. They recommended that Canada needs to get more new people involved for succession.

Program managers reported that no gap analysis has been conducted to identify the gaps in skills and competencies related to conducting Industry Canada's ITU activities. The main skills required ITU related skills are negotiation, presentation and technical skills. Industry Canada employees mainly have the technical skills but all do not have the special skills required for effective ITU participation. Those who represent Canada at the ITU need to be extroverts and be able to grasp the opportunity during the conferences or the chance of being heard would be lost.

In summary, there is anecdotal evidence that Industry Canada may have a skills gap but there has not been a detailed gap analysis to confirm this.

3.6.4 Succession Planning Activities

Industry Canada managers reported that there has been no formal succession planning related to ITU activities but the managers reported that the priority for succession planning is recognized at senior management level and practiced. For example, former and current ADM level of approval for the WRC delegation has included junior staff participation for development purposes (see World Radiocommunication Conferences 2000, 2003 and 2007 and Radiocommunication Assembly 2003); the DG of DGES is fully aware of the succession issue and is very supportive of succession planning activities; the three executives interviewed also think succession planning is a key component for Industry Canada–ITU's strategic planning exercise.

A survey of Industry Canada staff working in the directorates with ITU responsibilities revealed that these staff believe that there is an informal plan to ensure ITU activity continuity by knowledge transfer, on job training (coaching), outside training (ITU seminar, workshops), and opportunities to participate in ITU meetings, Canadian National preparatory meetings, etc. The staff suggested that development is provided to employees who demonstrate an ability to perform and who have a personal desire to get involved.

Junior staff reported that they have been given opportunities to participate in international meetings for training in international techno-diplomacy. The document review revealed that for the WRC–07, the Canadian delegation included seven junior engineers and four more senior staff who attended a WRC for the first time. Further, program managers reported that, from 2003–2007, Industry Canada has given the junior level staff responsibilities to coordinate the ITU conferences including briefing materials, preparing positions and participating ITU meetings.

Competencies and skills for key positions are reportedly reinforced in HR system such as recruiting, learning and development:

• Program manager and staff interviews showed that skills and competencies are fully documented, including ITU technical and negotiation process.
• Each employee has a Personal Learning Plan (PLP) and has on-the-job training for negotiation skills.

Staff engineers report that, in most cases, their career developmental requirements have been adequately assessed and their learning and developmental requests are encouraged and supported by management:

• Staff interviews indicated that the assessment is being done through yearly performance appraisal and learning plan processes.
• Staff interviews also showed that mentoring and coaching have been strongly supported by management. Management often provides the opportunity and guidance required by junior employees to learn and develop their expertise related to international activities and standardization.

3.7 Other Findings

There are two key findings from the evaluation that are not directly related to the evaluation issues:

• The most commonly cited Industry Canada–ITU weakness by stakeholders was that Canada does not send enough representatives to the ITU forums; and,
• Post-Conference reports do not provide a good indication of the level of Industry Canada's success at the ITU

Inadequate Representation

As previously mentioned, the most commonly cited Industry Canada ITU weakness by stakeholders in the survey of stakeholders was that Canada does not send enough representatives to the ITU forums (25% of stakeholders suggested this). Stakeholders indicated that Industry Canada must depend on industry to help out at ITU meetings and this is becoming more difficult for industry to do as the economy constricts.

Table 6 on the following page outlines the number of representatives sent by the top seven contributing ITU countries at some major ITU conferences over the past nine years. The table demonstrates that while Canada has sent proportionately slightly less delegates than its contributions in relation to the top seven contributing countries, its rank in terms of number of delegates has been almost the same as its rank in contributions.

The table would seem to indicate that while Canada has sent proportionately and absolutely less delegates than the United States and France, its number of delegates has not been severely proportionately less than other top contributing countries.

There are a couple of things to keep in mind also when considering this assertion: first, 75% of stakeholders did not identify inadequate representation as an Industry Canada weakness; and second, a large majority of stakeholders suggested that Industry Canada has been successful at the ITU. (For example, 100% of stakeholders indicated that Industry Canada has had moderate or great success in representing the diverse interests of Canadian stakeholders at the ITU and 91% said that Industry Canada has had moderate or great success in representing the interests of their organization at the ITU. Please see section 3.3 for a full discussion of the success that Industry Canada has achieved at the ITU.)

In summary, while inadequate Industry Canada representation at the ITU was the most frequently cited weakness, the evidence would seem to contradict this. Furthermore, this was only identified as a weakness by 25% of stakeholders and the evidence collected suggest that Industry Canada's ITU activities have been highly successful despite any possible shortage of representatives at ITU forums.

Inadequate Post-Conference Reports

Section 1.2.2 presented an overview of the methodology used to review post-conference reports. The analysis revealed that there is a lack of consistency in the contents and format of the reports. The section noted that post-conference reports, with the exception of the one for the 2007 World Radio Conference did not provide comprehensive coverage of pre-conference positions with post-conference results. These reports primarily provided information on the nature of ITU outcomes and anecdotal evidence of positive outcomes for Canada. From these reports, it is not possible to determine the degree of success that Canada achieved at the conferences.

For the 2007 World Radio Conference, the post-conference report provides an assessment of Canada's success in relation to each agenda item. For each item, Canada's success in achieving its position was documented as "completely satisfied", "partially satisfied", "not satisfied", "not applicable", or "no change". In addition, a Pre-briefing Memorandum was prepared that highlighted the key objectives for the conference. A post-conference executive summary reported on the degree of success in achieving the key objectives. The post-conference report that reported success by agenda item can provide a measure of success in achieving Canadian positions sought, i.e., the proportion of agenda items complete satisfied or partially satisfied. However, there is no consideration of the relative importance of the different agenda items. The Executive Summary of the post-conference report was better because it reported Industry Canada success in relation to the key pre-conference objectives. However, it was noted that some of the objectives reported in the Executive Summary had been modified from the pre-conference objectives so they weren't entirely comparable.

3.8 Summary of Findings/Conclusions

The purpose of this section is to recap the findings from sections 3.1—3.7. The following are the key findings/conclusions from this evaluation:

• Industry Canada's involvement in the ITU is consistent with its mandate and Government directions.
• Evidence collected indicates that there is a continued need for Industry Canada's involvement in the ITU.
• Evidence collected indicates that Industry Canada's involvement in the ITU provides good value to Canada.
• Evidence collected indicates that Industry Canada has largely been successful in achieving most of its intended outcomes in relation to its ITU activities.
• Evidence collected indicates that Canada, as represented by Industry Canada, is a recognized leader at the ITU.
• Evidence collected indicates that there is an opportunity for Industry Canada to improve its succession planning in relation to its ITU activities.
• Evidence collected indicates that many stakeholders believe that Canada does not send enough representatives to the ITU forums.
• Evidence collected indicates that post-conference reports do not provide a good indication of the level of Industry Canada's success at the ITU.

4.0 Recommendations

The only key findings from the evaluation that identified weaknesses related to post-conference reports and succession planning, hence the following are the two evaluation recommendations:

Some further details and substantiation of these recommendations are provided below:

Develop an improved methodology for measuring and reporting the success of Industry Canada activities at the ITU

As discussed in sections 1.2.2 and 3.3, the Industry Canada post-conference reports over the past nine years have not been consistent in terms of content and format. In addition, up until the 2007 World Radio Conference post-conference report, the reports primarily provided anecdotal information on Industry Canada's success in getting its positions adopted at the ITU. While the 2007 World Radio Conference post-conference reports and the related executive summary are an improvement, there are still difficulties in matching key conference objectives with post-conference results and the impacts on stakeholders.

The evaluators discussed the 2008–2009 ITU MRRS performance indicator (Proportion of Canadian ITU delegation objectives met) with program managers who voiced concerns that this measure is not an appropriate measure of success as different objectives have different levels of importance and in fact some positions may simply be negotiating positions that Industry Canada does expect to achieve. The evaluators would suggest that as part of this recommendation, ITU program managers review this MRRS measure and determine if there is a more appropriate measure to employ.

Follow through on the SITT HR Plan commitment to develop and implement a succession plan for SITT

The evaluation revealed that 60% of SITT EX's and 25% of SITT ENG are eligible to retire by 2012. This represents a challenge to continuing Industry Canada's ongoing success at the ITU. While Industry Canada managers suggested that there is ongoing succession planning activities and staff supported this view, to date there is no official succession plan. The 2008–2009 SITT HR Plan indicated that developing a robust succession plan is a top priority for 2008–2009 and beyond. It is the recommendation of this evaluation that SITT follow through on this commitment.

Management Response

Evaluation of Industry Canada's Involvement in the International Telecommunication Union

Background

The International Telecommunication Union (ITU) is the United Nations' Specialized Agency for telecommunications. The ITU is an inter-governmental organization, based in Geneva, Switzerland, with 191 Member States party to its treaty instruments. As the global focal point for governments and the private sector, the ITU's role in helping the world communicate spans three core sectors: radiocommunication, standardization and development. The key functions of the organization are its treaty-binding regulations and the facilitation of world-wide standardization of telecommunications.

Consistent with Industry Canada's statutory responsibilities under the Department of Industry Act and the Radiocommunication Act, active Canadian participation in ITU activities supports both government and industry objectives. Among the most important objectives are the following:  securing access to the radio-frequency spectrum and satellite orbit slots; avoiding harmful interference to radio communication services; ensuring management and functioning of the ITU meet Canadian requirements and are consistent with best practices for governance in the United Nations system; and, reducing costs for equipment manufacturers to bring products to market for the ultimate benefit of the Canadian consumer.

Two Industry Canada Sectors are involved in the delivery of ITU activities: Strategic Policy Sector and the Spectrum, Information Technologies and Telecommunications Sector. This Management Response is an integrated response from the two Sectors.

Management response to the recommendations

It is noted that the overall evaluation findings were very positive. Canada, as represented by Industry Canada, is a recognized leader at the ITU and has been largely successful in achieving intended outcomes. Evidence collected also indicated that there is a continued need for Industry Canada's involvement in the ITU which is recognized as providing good value to Canada. Two areas were noted where improvements could be made and are addressed in the recommendations. Specific responses to the recommendations are detailed below.

Develop an improved methodology for measuring and reporting the success of Industry Canada activities at the ITU

Management Response: Agreed

It is agreed that efforts should be enhanced to improve the methodology of reporting the success of Industry Canada activities at the ITU. In this regard, actions are being undertaken to compare objectives, as developed and elaborated prior to a major ITU event, and the results achieved at the conclusion of such events. An example of the application of this methodology is the ITU World Radiocommunication Conference (WRC) held in 2007. In the approval memorandum to senior management for the overall Canadian approach to the Conference and the proposed composition of the delegation in terms of roles and functions, a series of objectives were specified. In the report on the success of the Canadian delegation in achieving these objectives, a comparison with the results of the WRC from a Canadian perspective was undertaken and reflected in the Executive Summary to the comprehensive Canadian Delegation Report. Going forward, Industry Canada will replicate this approach for all major ITU events.

It is important to note, however, that in certain instances, a direct comparison between objectives and results may of necessity be nuanced. In the case of a WRC, for example, the achievement of some results is more important than others; in other cases, some Canadian objectives/proposals are actually used as part of a negotiating strategy to achieve results which are more critical to the realization of Canadian public and private sector interests. Consequently, it is important to interpret results from a broad perspective recognizing that some weighting of objectives and trade-offs must occur. In other cases, such as ITU plenipotentiary conferences, the achievement of objectives must be considered in the context of their political, and therefore, non-quantifiable outcomes.

With regard to lower level meetings, such as the Study Group meetings for ITU-R and ITU-T, Canadian delegation reports will contain, as a matter of common practice, a general comparison of objectives and results; however, the reporting requirements for such meetings will be different from those for major events.

It should also be recognized that the success of a Canadian delegation cannot be measured solely on the results of issues and objectives known before the meeting or conference. The ITU often looks to Canada to resolve issues that arise during the course of meetings by placing us in positions of influence, chairing of groups, etc. A successful meeting includes the ability to address foreign positions, proposals and ideas, ensuring that Canadian interests are advanced and defended, which takes time and effort. The ultimate goal remains a win-win solution whereby immediate or short-term Canadian objectives are also weighed against the greater good of worldwide improvements to telecommunications, which may in turn benefit Canadians in the longer term.

Follow through on the SITT HR Plan commitment to develop and implement a succession plan for SITT

Management Response: Agreed

The need to consider succession requirements has been specifically reflected in the proposed delegations to several recent ITU events referred to senior management for approval. For example the Canadian delegations to the 2002 and 2006 ITU Plenipotentiary Conferences, as well as the 2003 and 2007 ITU World Radiocommunication Conferences, included delegates whose principal roles were for developmental purposes. A balance between succession requirements and capacity to represent the diverse interests of Canadian stakeholders guide the composition of delegations. Canadian delegates with significant international experience as well as governance and technical knowledge provide guidance and advice on issues and procedures.

It is noted in section 3.6.4 of the Report that succession planning and training and development of staff for ITU activities have been ongoing within Industry Canada. The ITU is structured in tiers with the high-level conferences at the top, Study Groups below that prepare work for these conferences and multiple Working Parties that report into each Study Group. For each of these groups, Canada has a Canadian National Organization that prepares the Canadian contributions, positions and views. Junior staff members are included in this preparatory process and are given opportunities to attend both the preparatory and, whenever possible, international meetings, thus gaining experience and knowledge leading up to involvement in a high-level conference. Further, junior staff members participating in a WRC are expected to carry responsibility for developing proposals, negotiating with industry groups and presenting and defending proposals at meetings leading to, and at, the WRC. It is noted that engineering staff must gain experience not just in the technical aspects of ITU work but also in the procedural and political aspects of its functions.

Lastly, staff training and development in ITU activities is not limited to ITU meetings. The key skills used for success in such meetings, such as preparation and presentation of technical contributions, negotiation skills, and inter-personal abilities, are also required for other organizations where staff participate. In fact, much of the work preparing for ITU high-level conferences is performed by regional organizations such as CITEL for the Americas, CEPT for Europe and APT for the Asia-Pacific countries. Participation at regional organization meetings is both essential for success at ITU and also provides opportunities for staff development.

It is agreed that all of the succession plans and activities currently being implemented are not documented in a formal staffing and training plan for all levels of ITU activities. This will be accomplished by the end of June 2009.

Approved:

_________________
Robert Dunlop

__________________
Helen McDonald

Long Descriptions

Figure 1: Organizational Structure for Industry Canada's ITU Activities

Chart which describes the organizational structure for Industry Canada's ITU activities. The top tier is a box indicating Industry Canada.

The second tier presents the two Industry Canada sectors that are involved in ITU activities. On the left is the Strategic Policy Sector and on the right is the Spectrum, Information Technologies and Telecommunications Sector.

The third tier indicates the branches that are involved in ITU activities under each sector. The Telecommunications Policy Branch (DGTP) is the only branch indicated in the Strategic Policy Sector. There are two branches indicated under the Spectrum, Information Technologies and Telecommunications Sector: The Spectrum Engineering Branch, and The Radiocommunications and Broadcasting Regulatory Branch.

Finally, the fourth tier indicates the directorates under each branch that are directly involved in ITU activities. The two directorates indicated under the Telecommunications Policy Branch are the International Telecommunications Policy and Coordination Directorate (DIT) as well as the Spectrum and Radio Policy Directorate (DSRS).

There are three directorates indicated under the Spectrum Engineering Branch: The Spectrum Planning and Engineering Directorate (DDGSE), The Broadcast Multimedia Planning and Technical Policy Directorate (DBCP), and The Telecom Engineering and Certification Directorate (DSI). The only directorate indicated under the Radiocommunications and Broadcasting Regulatory Branch is the Space and International Regulatory Activities Directorate (DSIR).

Back to main text.

Figure 2: Graphical Logic Model

This diagram displays the Logic model for Industry Canada Involvement in the International Telecommunications Union (ITU). The first row of boxes shows the Activities with arrows pointing to the Outputs which these Activities are expected to achieve in the second row. The third row consists of Immediate Outcomes achieved relating to the ITU, the fourth row is Immediate Outcomes achieved relating to Canada, the fifth row is Intermediate Outcomes, the fifth row is Ultimate Outcomes, and the final row of boxes are Industry Canada's Strategic Outcomes. The Logic Model is designed to show how each activity, output, and outcome leads to the next one, and ultimately to Industry Canada's Strategic Outcomes.

Back to main text.

Summary of Evaluation Issues and Questions

Table outlining the different evaluation issues in the first column and the associated evaluation questions in the second column.

Back to main text.

Table 1: ITU Cost from 2003–2004 to 2007–2008

Table outlining the cost incurred relating to the ITU. Each row corresponds to a fiscal year, starting in 2003–04 and ending in 2007–08. The final row is the average over the five years.

Back to main text.

Table 2: Summary of Evaluation Issues and Questions

Table outlining the evaluation issue and questions pertaining to this evaluation. The first row states the two evaluation questions relating to the issue of relevance. The second row states the two questions relating to the issue of success. The third row states the one question that stems from the issue of cost-effectiveness. The fourth row states the question that relates to program delivery.

Back to main text.

Table 3: Distribution of Staff Interviewed by Classification/Level

Table which shows the distribution of staff interviewed for the evaluation by classification/level. Annex 18 contains a list of all staff interviewed.

Back to main text.

Table 5: Comparison of Average Age and Retirement Eligibility Rate Between Public Service of Canada and SITT

Table which compares the average age and retirement eligibility rate between public service of Canada and SITT. The first row indicates the average age. The second row indicates the percentage of workers over 50 years old. The third row indicates the percentage of workers eligible to retire. The fourth row represents the total percentage eligible to retire by 2012.

The workers are then broke up by column. The first column is the public service of Canada. The second column are public service workers in the EX classification. The third column is SITT employees in the EX classification. The fourth column represents SITT employees in the ENG classification.

Back to main text.

Table 6: Contributions to the ITU Versus Number of Delegates Sent to Conferences for Top Seven Contributing Countries

Table that shows the contribution to the ITU versus Number of Delegates sent to conferences for top seven contributing countries. Each row represents a different country. The second and third row indicate the contribution towards the ITU given by each country. The following two rows indicate the number of delegates sent to the 2000 WRC conference. The sixth and seventh row deal with the 2003 WRC conference. The eighth and ninth row are associated with the Plenipotentiary Conference in 2006. The final two rows deal with the 2007 WRC conference.

Back to main text.