Industrial and Technological Benefits Policy Evaluation Report

From: Innovation, Science and Economic Development Canada

May 2020

List of abbreviations and acronyms

DND

Department of National Defence

GDP

Gross Domestic Product

IRB

Industrial and Regional Benefits (Policy)

ISED

Innovation, Science and Economic Development Canada

ITB

Industrial and Technological Benefits (Policy)

PSPC

Public Services and Procurement Canada

R&D

Research and Development

RFP 

Request for Proposal

SME

Small- and Medium-Sized Enterprise

VP

Value Proposition

List of figures

• Figure 1: ITB Policy Logic Model

List of tables

• Table 1: ITB Branch Expenditures

---

Executive summary

This report presents the results of an evaluation of the Industrial and Technological Benefits (ITB) Policy. The program is administered by Innovation, Science and Economic Development Canada (ISED) and was announced as part of the Defence and Procurement Strategy in February 2014 to replace the Industrial and Regional Benefits (IRB) Policy, which had been in place since 1986. The ITB Policy helps ensure that defence and security procurements going forward are better leveraged to create jobs and economic growth in Canada.

Program overview

The ITB Policy requires that companies awarded defence procurement contracts undertake business activities in Canada, equal to the value of the contract (as under the IRB Policy). The ITB Policy also includes a requirement for companies to provide specific, targeted proposals (known as the Value Proposition or VP) for how they will contribute to economic growth in Canada through targeted investments. In addition, some changes to the banking policy, the goal of which is to encourage bidders to engage Canadian companies early and invest in Canada well ahead of securing a defence contract, were made with the introduction of the ITB Policy.

The ITB Policy has the following five core objectives:

• Support the long-term growth and sustainability of Canada's defence sector;
• Support the growth of prime contractors and suppliers in Canada, including small- and medium-sized enterprises (SMEs) in all regions of the country;
• Boost innovation through research and development (R&D) in Canada;
• Increase the export potential of Canadian-based firms; and
• Leverage opportunities in skills development and training to advance employment opportunities for Canadians.

Between February 1, 2014 and January 31, 2019, 32 contracts valued at almost $6.2 billion were initiated under the ITB Policy.

Evaluation purpose and methodology

The objectives of this evaluation were to examine the relevance, performance and efficiency of the ITB Policy in accordance with the Treasury Board Policy on Results and address issues identified by ITB Branch management (e.g., engagement with industry prior to contracting). The evaluation covered the period from February 1, 2014 to January 31, 2019. The evaluation employed multiple data collection methods: a document review, interviews, administrative data review, survey of industry, input/output (I/O) analysis, and case studies.

Findings

Relevance

The ITB Policy, including expenditure requirements and VPs, is aligned with ISED priorities related to the competitiveness of Canadian firms, economic growth, job creation, innovation and R&D. Further, the ITB Policy is aligned with the needs of the Canadian defence industry by supporting the growth of prime contractors and suppliers in Canada.

Performance

Roughly one-third of ITB Policy recipients have developed new business relationships with prime contractors as a result of ITB Policy-related work – about half have become qualified suppliers, while a majority said that ITB Policy contracts allow them to sell to new clients.

The ITB Policy is contributing to the development of a Canadian supplier base, as it enforces sourcing from Canada-based suppliers. Most recipients report that the ITB Policy contracts improved their work processes and contributed to their R&D and innovation capacity. According to data analyses, the ITB Policy was estimated to have contributed to the maintenance/creation of as many as 6,450 jobs per year in the private sector between 2014-15 and 2018-19. The impact on Canada's Gross Domestic Product (GDP) was estimated to have been as high as $3.3 billion for this period.

The ITB Policy has led to improvements in the engagement process with industry. Early engagement has allowed industry to provide views on the development of the VP framework before the Request for Proposals are issued, and engagement is generally viewed as being timely.

Further, Value Propositions increase benefits for Canada in many areas, including R&D, innovation, skills development, and economic development. Two-thirds of prime contractors interviewed were satisfied with the Value Proposition approach. However, one-third expressed concerns about the lack of clarity of the VP statements, their complexity and limited flexibility of the approach.

In terms of the banking mechanism, it is generally being utilized as planned and is encouraging prime contractors to make investments in Canada.

Finally, the ITB Branch research and analytics function is used for decision-making and reporting purposes and is deemed useful within ISED.

Efficiency

ITB Branch operations were efficient over the 2014-15 to 2018-19 period, with no major issues or gaps identified with respect to the efficiency of operations. However, there are opportunities to improve the consistency of the crediting process (i.e., the process whereby contractors submit reports to get their expenses credited as per their VPs and obligations).

Recommendations

The evaluation findings led to the recommendations noted below.

Recommendation 1: Value Proposition Approach

ISED should continue to work with contractors to ensure that the Value Proposition approach on a procurement-by-procurement basis is as flexible as possible for prime contractors to fulfill requirements, while continuing to meet the policy objectives.

Recommendation 2: Crediting Process

ISED should enhance internal guidance related to the application of crediting guidelines for contractor expenses to improve the consistency of the crediting process.

Recommendation 3: Program Understanding

ISED should continue industry engagement and outreach activities to enhance stakeholder understanding of the ITB Policy, particularly the Value Propositions.

1.0 Introduction

1.1 Report overview

This report presents the results of an evaluation of the Industrial and Technological Benefits (ITB) Policy. The program is administered by Innovation, Science and Economic Development Canada (ISED) and was announced in February 2014 to replace the Industrial and Regional Benefits (IRB) Policy, which had been in place since 1986. The ITB Policy helps ensure that defence and major Canadian Coast Guard procurements going forward are better leveraged to create jobs and economic growth in Canada.

The purpose of this evaluation is to assess the relevance, performance, and efficiency of the ITB Policy. The report is organized into four sections:

• Section 1 provides the context and profile of the ITB Policy;
• Section 2 presents the evaluation methodology and the challenges for the evaluation;
• Section 3 presents the findings; and
• Section 4 summarizes the conclusions and provides recommendations.

1.2 Context

Federal procurement of goods and services has the potential to generate significant industrial benefits for Canadian firms. Since the mid-1970s, the Government of Canada has been pursuing the use of industrial benefits as part of federal procurement contracts that fall outside of international trade agreements or for which a National Security Exception is invoked. Under this approach, firms bidding on government defence and major Canadian Coast Guard contracts are evaluated on the basis of the economic benefit of their proposal to Canada, as well as price and technical requirements. Since 1986, 144 projects have leveraged a total of $43.8 billion in economic obligations^{Footnote 1}.

Canada's Defence Procurement Strategy, announced in February 2014, transformed Canada's IRB Policy into the ITB Policy. As with the IRB Policy, the ITB Policy requires that companies awarded defence procurement contracts undertake business activities in Canada, equal to the value of the contract. However, the ITB Policy also includes a requirement for companies to provide specific proposals (known as the Value Proposition or VP) for how they will contribute to economic growth in Canada through targeted investments^{Footnote 2}.

1.3 Objectives and description

The ITB Policy is part of the procurement process used to purchase new equipment and services for the Canadian Armed Forces and Canadian Coast Guard. Companies that bid on a defence contract submit a proposal to the Government of Canada, which includes a technical submission which is evaluated by the Department of National Defence (DND), a financial submission evaluated by Public Services and Procurement Canada (PSPC), and a VP evaluated by ISED. The proposal with the highest combined score wins the contract.

The ITB Policy requires the winning bidder to invest an amount equal to the value of their contracts back into the Canadian economy. This is accomplished by forming partnerships and awarding work to firms in Canada. The winning bidder could also invest in new technologies and research with post-secondary institutions, as well as with Canadian companies both inside and outside the defence industry, helping to grow the Canadian economy and make Canada more innovative.

The ITB Policy has the following five core objectives:

• Support the long-term growth and sustainability of Canada's defence sector;
• Support the growth of prime contractors^{Footnote 3} and suppliers in Canada, including small- and medium-sized enterprises (SMEs) in all regions of the country;
• Boost innovation through research and development (R&D) in Canada;
• Increase the export potential of Canadian-based firms; and
• Leverage opportunities in skills development and training to advance employment opportunities for Canadians.

A core element of the ITB Policy approach is a rated and weighted VP that requires bidders to put forward activities that support the five core objectives of the ITB Policy and provide economic benefits to Canada (weightings for each objective can vary on a procurement-by-procurement basis). Under the IRB Policy, bids were evaluated on the basis of price and technical merit only. Under the ITB Policy, VPs are also factored into the evaluation of bids. Bidders receive points for VPs that support the five core objectives of the ITB Policy. After a contract is awarded, the contractor is required to start fulfilling its commitments (i.e., business activities directly related to the equipment or service being procured by Canada) and to identify further business activities in Canada, as may be required to meet the overall ITB Policy obligation (i.e., 100% of the contract value).

The change from the IRB Policy to the ITB Policy included a new emphasis on federal government engagement with prime contractors and Canadian industry as a whole, before contracts are awarded, which gives industry an opportunity to provide input into the development of a Request for Proposal (RFP), including the VP framework. The change to the ITB Policy also included enhanced monitoring and reporting on the economic impacts of the ITB Policy after contract award (e.g., updates on the level of Canadian SME involvement).

Banking Policy

The banking policy aspect of the ITB Policy (created in 2009 under the IRB Policy) allows companies to undertake business activities in Canada prior to a procurement and "bank" these investments for credit against their IRB/ITB obligations later, when they secure a defence contract. The goal of the banking policy is to encourage bidders to engage Canadian companies early and invest in Canada well ahead of securing a defence contract.

Five changes^{Footnote 4} to the banking policy were made to the ITB Policy in 2015 to significantly enhance the attractiveness of investing in Canada in advance of a contractual requirement and motivate prime contractors to extend successful business relationships with Canadian suppliers above and beyond their contractual requirements:

• A change of the time period for validity of investments in the bank from five to ten years;
• Removal of the annual depreciation of banking transaction values;
• Introduction of annual crediting of investments;
• Removal of the requirement to identify future projects; and
• Clarification that contractors can use only up to 50% of banked credits to satisfy their overall obligation on a project.

As before, banked transactions may be used to fulfill up to 50% of a contractor's overall IRB/ITB obligation, thereby ensuring that at least one-half of a company's obligation comprises new investment commitments. For example, if a company has invested $20 million into the Canadian economy prior to winning a $100 million ITB Policy contract, the company can withdraw these investments from their ITB bank account and apply them against the $100 million obligation.

The banking policy also allows companies that achieve more than their contractual obligations to bank the overachievement for future credit. This encourages prime contractors to fulfil their obligations ahead of schedule and continue investing in Canadian suppliers with which they have successful relationships.

1.4 Delivery and governance

The ITB Branch is responsible and accountable for the administration and management of the ITB Policy. The ITB Branch established a set of processes, including controls and approvals, which are recorded in a Process Management Information System. Employees are required to use this system, which in turn has a series of tasks officers must complete for each procurement to which the ITB Policy applies. During the pre-contract stage and the verification activity stage, team leaders are responsible for ensuring officers are in compliance with the ITB processes while providing value-added input into ITB procurement documents (e.g., VP evaluation framework). Senior management has the responsibility of reviewing procurement documentation, providing comments, and approving procurement documents. In the Contract and Verification Activity stage, team leaders are required to review verification material and reports developed by officers and make any necessary changes before being approved by Directors.

ITB Branch management conducts regular reviews to ensure all processes are followed, correct information is being entered, and proper amounts are being credited in the ITB Process Management Information System. These processes include required consultations with the following organizations:

• DND/Canadian Coast Guard: defines the technical requirements, conducts cost and options analyses.
• PSPC: identifies contracting policy, organizes industry engagements, develops a procurement plan and contractual documents, oversees the bid evaluations process, monitors prime contractor compliance with contracts, and updates procurement contracts.
• Global Affairs Canada: assists in developing, evaluating and implementing VP export strategies.
• Non-government entities: organizations such as industry associations assist to communicate the ITB Policy, establish and maintain relationships, identify Canadian activities, and provide relevant documentation to assist ITB verification efforts.
• Regional Development Agencies:^{Footnote 5} assist to communicate the ITB Policy, help bidders search for Canadian capabilities, assist in developing a procurement strategy, assist in evaluating the VP, and help prime contractors and eligible parties identify additional Canadian capabilities to satisfy ITB obligations.
• ISED Legal Services: provide legal advice on ITB contractual terms and conditions.
• Audit and Evaluation Branch: conducts internal audits and program evaluations.

1.5 Resources

Between February 1, 2014 and January 31, 2019, 32 contracts valued at almost $6.2 billion were initiated under the ITB Policy^{Footnote 6}. These contracts were distributed among four sectors:

• Aerospace sector: 14 contracts valued at $4.67 billion.
• Land sector: 10 contracts valued at $853.6 million.
• Marine sector: 7 contracts valued at $590.5 million.
• Space sector: 1 contract valued at $57.6 million.

Projects under the ITB Policy's predecessor (IRB Policy) awarded since the launch of the Defence Procurement Strategy in 2014, valued at $6.1 billion, were excluded from this evaluation since they exclude some of the key elements of the ITB Policy being evaluated, notably the VPs.

Between 2014-15 and 2018-19, total ITB Branch expenditures more than doubled, from about $3.9 million to $8.1 million, with operations and maintenance expenditures almost tripling^{Footnote 7}. The majority of the financial resources (60.6%) over this five-year period were dedicated towards staff salaries.

Table 1: ITB Branch Expenditures

Fiscal Year	2014-15	2015-16	2016-17	2017-18	2018-19	Total
Operations and Maintenance	$1,057,196	$2,723,138	$3,646,804	$3,703,863	$3,147,245	$14,278,246
FTEs	31	48	54	56	56	245
Salaries	$2,853,131	$4,257,189	$4,577,735	$5,331,207	$4,967,399	$21,986,661
Total	$3,910,327	$6,980,327	$8,224,539	$9,035,070	$8,114,644	$36,264,907

1.6 Logic model

The 2018 logic model for the ITB Policy is presented in Figure 1. A logic model is a visual representation that links a program's activities, outputs and outcomes; provides a systematic and visual method of illustrating the program theory; and shows the logic of how a program, policy or initiative is expected to achieve its objectives. It also provides the basis for developing the performance measurement and evaluation strategies.

2.0 Methodology

This section provides information on the evaluation context, objectives, scope and approach, issues and questions, data collection methods, and limitations.

2.1 Evaluation context

The previous evaluation of the ITB Policy (formerly known as the IRB Policy) was completed in January 2015. The evaluation found a continued need for federal support for industrial development in Canada's defence and security sector. In terms of performance, the evaluation noted that prime contractors were fulfilling their IRB obligations, that IRB recipients had benefitted from IRB transactions to a varied extent, and that the IRB Policy had led to the growth of IRB recipients and the development of new business relationships. The evaluation recommended that the ITB Branch should:

• Ensure that the ITB Policy encourages high-quality outcomes (including those identified in the review of previous IRB transactions) by defining quality and success factors, establishing baselines, and monitoring performance;
• Ensure that the monitoring and reporting system currently in place tracks the impacts on recipients of IRB transactions and the new ITB Policy; and
• Clarify and streamline its administrative procedures.

2.2 Objectives

An evaluation of the ITB Policy is a Treasury Board submission requirement. The objectives of this evaluation were to examine the ITB Policy in accordance with the Treasury Board Policy on Results and address issues identified by ITB Branch management (e.g., engagement with industry prior to contracting) not covered under the logic model.

2.3 Scope and approach

The evaluation was conducted by Goss Gilroy Inc. and covered the period from February 1, 2014 to January 31, 2019. It examined the relevance, performance and efficiency of the ITB Policy, including the economic and social benefits of the Policy. In the assessment of performance, the evaluation focused only on progress made towards the achievement of the immediate outcomes identified in the logic model, as the ITB Policy has been in existence for only five years. The logic model suggests that intermediate outcomes require 5 to 15 years to be measureable.

Contracts for the IRB Policy that were active during the evaluation period are not covered by this evaluation, since they exclude some of the key elements of the ITB Policy being evaluated, notably the VPs.

2.4 Evaluation issues and questions

The evaluation addressed the following questions:

Relevance

1. Is the ITB Policy, including the VP, aligned with ISED priorities?
2. Is the ITB Policy in line with the needs of the Canadian defence industry, in particular the SMEs?

Performance

3. To what extent is engagement with industry prior to contracting effective and timely?
4. To what extent has the VP approach been implemented as planned?
   1. Are some VP components more beneficial than others?
5. To what extent has the ITB Policy led to the development of new business relationships between prime contractors and recipients?
   1. How many recipients have then become qualified suppliers of prime contractors?
   2. How many recipients have then become primary (tier 1) or secondary (tier 2) contractors?
6. What are the economic and social impacts of the ITB Policy?
   1. What are the direct economic benefits of the ITB Policy in terms of employment and economic growth?
   2. To what extent have recipients benefited from the ITB Policy, including VP? From supplemental business opportunities with other firms after the initial ITB Policy transactions?
   3. To what extent have prime contractors benefitted from the ITB Policy, including VP?
   4. To what extent have other organizations benefitted from the ITB Policy, including VP?
7. To what extent does use of the banking mechanism lead to an impact on recipients?
8. To what extent is the ITB Branch research and analytics function providing value to ISED and other departments?
   1. To what extent does ISED or the ITB Branch data support decision-making?

Efficiency

9. To what extent has the implementation of the ITB Policy demonstrated efficiency? Are there alternatives to improve efficiency?

2.5 Data collection methods

Multiple lines of evidence were used to address the evaluation questions. The data collection methods included a document review, interviews, administrative data review, survey of industry, input/output (I/O) analysis, and case studies.

Document Review

A document review was conducted to gain a thorough understanding of the program and to provide insights into relevance and performance. The review included ISED program and foundational documents, ITB annual reports, evaluation reports, corporate plans and administrative data.

Interviews

The objective of the interviews was to gather in-depth information related to the relevance, performance, and efficiency questions. The interviews were semi-structured in nature to help collect qualitative information from a range of key stakeholders. Interviews were conducted either in-person or by telephone.

The identification of key stakeholders was confirmed with program management. In total, 32 interviews were conducted as part of the evaluation with the following three groups:

• ISED management and staff (n=15);
• DND staff (n=3); and
• Prime contractors (n=14, including two with Defence Industry Advisory Group^{Footnote 8} members).

Administrative Data Review

The administrative data reviewed included ITB Branch financial tables and data on results against service standards. The data was summarized using tables and analyzed to assess program efficiency. Contract dollars data was also analyzed as part of the I/O analysis.

Survey of Industry

An online survey (based on a sample of 194 private sector firms and organizations that were part of an ITB Policy transaction) was conducted of all companies (including prime contractors) that were recipients on one or more ITB Policy transactions. The survey resulted in 45 completed surveys (and 14 partially completed surveys) for a response rate of 23% (30% response rate if including partially completed surveys). The response rate is in line with other comparable surveys.

The distribution of survey respondents was almost identical to the distribution of the original sample of 194 (e.g., 50% of the original sample was companies in the air sector versus 49% for the completed or partially completed surveys).

Input/Output (I/O) Analysis

The economic impact analysis undertaken for this evaluation utilized data from the results of the survey of industry undertaken as part of this evaluation. The data obtained was then analyzed using Statistics Canada's National Accounts and I/O tables. The analysis also includes an I/O analysis of the ITB Policy contracts (as a separate analysis). The I/O analysis was conducted by ISED economists within the ITB Branch. More detailed information on the I/O analysis and the economic impact of the ITB Policy is provided in Appendix B and Appendix C.

Case Studies

Five focused case studies were completed of specific ITB Policy contracts to get a more in-depth understanding of the ITB Policy. The cases were selected in consultation with the ITB Branch and covered contracts in three domains (air, marine, and land). The respondents from the five cases and the 14 prime contractors interviewed as part of the interviews covered more than half of the prime contractors working under ITB Policy contracts.

After the case selection, the ITB Branch provided file information related to each of the cases, including contract amounts, project descriptions and lists of sub-contractors. The main representatives of the private sector firms were contacted for an interview by phone. Other interviews were conducted based on their recommendations, including some with the Canadian sub-contractors and recipients of the VPs. Up to three interviews were conducted per case.

2.6 Limitations

Limited Quantitative Information

The interviews were conducted and analyzed using qualitative methods. While this allows for a more in-depth data collection and analysis, it presents a risk of bias on the part of the evaluation analysts. This was mitigated by focusing the analysis on factual information and results that were supported by explanations (versus simple opinions without any rationale provided). When possible, the evaluation team also reviewed documentation to ensure that the information was in line with formal program documentation.

Case Studies

The case studies included only five ITB Policy contracts awarded. Therefore, they are not representative of all ITB Policy contracts; nonetheless, they helped inform and corroborate other lines of evidence.

3.0 Findings

3.1 Relevance

3.1.1 Is the ITB Policy, including the Value Proposition (VP), aligned with ISED priorities?

According to documents reviewed, the ITB Policy aligns with ISED's departmental result that Canadian companies are globally competitive and achieve high growth^{Footnote 9}. The application of the ITB Policy has enabled the development of new technologies and skilled labour, acquisition of new technologies, and increased innovation^{Footnote 10}. In addition, the Policy generates intangible benefits to Canadian companies such as knowledge transfer, improves internal processes, and enhances access to other markets and supply chains. These tangible and intangible benefits allow Canadian companies to become competitive and grow in the aerospace and defence sector.

In ISED's 2019-20 Departmental Plan, there is a strong focus on redesigning the innovation policy landscape in Canada. It states that the Innovation and Skills Plan is "a new microeconomic framework to drive growth – one that aims to succeed by building a culture of innovation and a globally recognized brand as one of the most innovative and competitive countries in the world."^{Footnote 11} Moreover, this multi-year plan builds on Canada's innovation strengths, including exporting and scaling-up globally competitive companies. Since the ITB Policy is used as an investment attraction tool for the Canadian economy, it is well aligned with ISED's priorities. More specifically, the Policy has been developed to ensure that purchases of defence equipment and services results in economic growth, innovation and success in export markets.

In addition, the incorporation of Key Industrial Capabilities^{Footnote 12} into the ITB Policy framework enables the Government of Canada to bring a more strategic approach to leveraging procurement that reflects key upcoming defence procurement requirements for which significant economic and commercial benefit may be derived while still allowing for broad investments. This is documented in the VP Guide^{Footnote 13}.

The ITB Policy also aligns with the Innovation and Skills Plan, one of ISED's key priorities, which emphasizes the development of partnerships through bringing stakeholders together from across the innovation system. The ITB Policy indeed supports this element through encouraging stronger partnerships between contractors and Canadian companies, as well as between industry and research institutions. In fact, the VP Guide also notes that since the introduction of the Policy in 2014, potential bidders are "engaging earlier with Canadian companies to form partnerships and identify high quality business investments up-front on a project." It is also stated that through VPs, R&D investments will be encouraged that involve strategic and long-term partnerships with Canadian SMEs, for example, to provide engineering support that results in knowledge transfer that will help an SME to scale-up and advance the commercialization of innovative ideas and products. Industry partnerships with Canadian post-secondary academic and public research institutions are encouraged under the ITB Policy, including the formation of research consortia, to create an environment that supports world-class innovation and allows for the fusion of hands-on, industrial experience with academic knowledge and inquiry. Furthermore, to motivate bidders to partner with Canadian universities and colleges, additional points towards the evaluation of the VPs can be awarded based on the R&D that bidders and their major suppliers propose to undertake with accredited Canadian post-secondary institutions.

ISED's Departmental Plan also highlights the need to embrace inclusivity and fosters the participation of traditionally underrepresented groups in the innovation economy. The introduction of the Gender and Diversity Plan in the second edition of the VP Guide supports this objective as bidders are now required to provide these plans, at the prime contractor level, describing their approach to achieving gender balance and increasing diversity within their Canadian corporate structures and broader supply chains in Canada. Some interviewees emphasized the addition of the Gender and Diversity Plan requirement, which aligns with GBA+ initiatives being undertaken by the broader government agenda and priorities. However, none of the firms covered in the case studies had a Gender and Diversity Plan, as this was not a requirement when they signed their contracts – which was during the first edition of the VP Guide before the Plan was introduced. Other firms had a Plan, but these existed before their ITB Policy contract came into effect.

The majority of ISED interview respondents indicated that the transition from the IRB Policy to the ITB Policy brought about more opportunities to align with ISED priorities due to the introduction of the VPs. Some government respondents discussed how the ITB Policy allows for flexibility for companies to respond to ISED priorities, which is in contrast to the previous pass/fail structure of the IRB Policy. Respondents from DND generally agreed that the ITB Policy is aligned with DND priorities as well.

3.1.2 Is the ITB Policy in line with the needs of the Canadian defence industry, in particular the small- and medium-sized enterprises (SMEs)?

Canadian defence SMEs^{Footnote 14} have multiple needs that must be met in order for them to enter markets, grow, and be successful. According to interviews, two of these key needs relate to accessing and integrating into global supply chains by making themselves known to the main contractors and becoming qualified suppliers of the main contractors. Becoming a qualified supplier entails a number of requirements, including having the appropriate certifications, and management and quality control systems, among other requirements.

The document review indicated that, through leveraging defence and major Canadian Coast Guard procurement to create jobs and economic growth, the ITB Policy supports the long-term growth and sustainability of Canada's defence sector, supporting the growth of prime contractors and suppliers in Canada^{Footnote 15}. Moreover, the ITB Policy requires companies awarded defence contracts to do business in Canada equal to the value of their contracts.

According to interview findings, the ITB Policy requirements for direct expenditures in Canada meet the needs of Canadian businesses, as it allows them to contact prime contractors and offer their services, knowing that the prime contractors will need services from Canadian suppliers in order to meet their ITB obligations. According to survey findings, 46% of recipients have become qualified suppliers as a result of ITB Policy contracts. Case study evidence shows that many suppliers provide support and training for Canadian SMEs in order for them to become qualified:

One contract in the area of aircraft maintenance shows how prime contractors can help SMEs become qualified suppliers. This long-term maintenance contract for the Hercules and Aurora aircrafts, has provided the opportunity for Canadian SMEs to receive training from the prime contractor to allow them to work on these aircraft that were designed to DND's special requirements. They are also required to meet the standards of the International Organization for Standardization. The contracts received allow SMEs to develop new capabilities and expand their markets.

The survey findings also indicate that the ITB Policy contracts have contributed to improving recipients' profiles and credibility (76% agreed) and 26% said that the ITB Policy contracts contributed to their participation in the prime contractor's global supply chain. Almost all interview respondents agreed that this requirement (for direct expenditures in Canada) led to more business for Canadian suppliers.

3.2 Performance

3.2.1 To what extent is engagement with industry prior to contracting effective and timely?

ISED works in collaboration with PSPC and DND for the engagement process with industry. Engagement with industry informs these three departments and leads to the overall development of a tailored approach for the RFPs.

Findings from the key informant interviews indicated that the ITB Policy has led to improvements in the engagement process between government and industry prior to issuing a contract. A few respondents explained that under the IRB Policy, engagement typically only occurred after the contract was awarded. Under the ITB Policy, there are multiple touch-points for this dialogue to occur – during industry days, one-on-one meetings, and written solicitations (e.g., via questions and answers prior to RFP issuance). Several ISED officials interviewed noted that early engagement provides industry the opportunity to substantiate the feasibility of the VPs (e.g., investing in R&D, SMEs, etc.), which has improved the RFPs and VPs. The majority of industry respondents interviewed agreed that earlier engagement allows government partners to better understand the context of industry at an earlier stage. Government partners can then use this feedback to gain a general sense of whether a project's requirements are feasible. If industry concerns are found to be valid, RFPs can be adjusted by PSPC to better reflect what the industry can deliver for a given contract.

According to the majority of prime contractors interviewed, earlier engagement activities under the ITB Policy (a joint responsibility with DND and PSPC) are useful and beneficial. However, some projects' timelines were impacted by the transition from the IRB Policy to the ITB Policy. These specific instances occurred when the Defence Procurement Strategy was announced and the ITB Policy went into effect. Some IRB Policy procurement contracts were already underway at the time of this transition; as such, there was a small delay while the procurements were adjusted to include ITB Policy elements. Once these projects resumed, the ITB Policy (including the VP element) was applied to the new contracts and industry engagement occurred well in advance of the final RFP, which was the overall intent of the Defence Procurement Strategy.

3.2.2 To what extent has the VP approach been implemented as planned?

Findings from the interviews and case studies suggests that the VP approach has been implemented as planned – the inclusion of VPs in the evaluation of bids is motivating prime contractors to make greater and higher quality investments earlier, and increasing their willingness to partner with Canadian companies. The VP evaluation criteria was found to be clear, fair and well understood by bidders and encourages high value proposals based on individual firms' strengths.

As noted earlier, the VP approach requires bidders to put forward activities that support the five core objectives of the ITB Policy and provide economic benefits to Canada. In terms of the objective related to boosting innovation through R&D in Canada, a number of positive aspects were noted:

• Most ISED officials interviewed saw a clear change from the IRB Policy in that the VPs allow the federal government to steer suppliers towards strategic investments in R&D, as RFPs can now be more tailored.
• ISED officials also noted that many firms have partnered with universities and research centres to conduct R&D. There is evidence of successful R&D and innovation VPs – because there is a multiplier applied to these credits, some firms target R&D for their credits.
• Many prime contractors see benefits stemming from the R&D investment portion of the VP approach. The interviews and case studies found that, in some instances, projects involving R&D led to the development of prototypes and new products that may be offered to other clients in the future (e.g., 3D printing of parts, new fabrics, and new approaches to quality-control).
• Some prime contractors interviewed said that they work actively with universities on R&D engineering projects and have improved their manufacturing processes as a result of the collaboration.
• For DND, the VPs can contribute to the development of a stronger defence industry in Canada, as they provide incentives (points in bid score) to support the growth of Canadian SMEs in the defence sector.

However, according to document review findings stemming from meetings of the Defence Industry Advisory Group in June 2018, the multipliers may not be sufficiently recognizing all forms of R&D. The meetings culminated in industry recommending that the R&D investment portion of the VP approach be made broader to allow for more meaningful investments across different technology areas, and that a wider use of the multiplier would encourage greater R&D investments in Canada.

Overall, two-thirds of prime contractors interviewed were satisfied with the VP approach. However, one-third mentioned that they found the overall VP approach complex and sometimes inflexible, starting at the RFP stage. Some prime contractors noted that there is a lack of clarity of some VP statements in the RFPs which has led to disagreements between some prime contractors and ISED at the contracting stage. Additionally, some prime contractors indicated that RFPs can involve many ITB Policy objectives, which makes it difficult for some firms to meet requirements across the board. Prime contractors understand that the objectives are aligned with ISED and/or government priorities, but sometimes not all of the objectives fit well within a project. Some firms would prefer more flexibility to concentrate on one or two objectives, if such options existed. Further, it was mentioned by some prime contractors that the VP approach was not well adapted for their contract/type of work. For example, some service-related contracts are not good candidates for the conduct of R&D, as they are less likely to involve or benefit from R&D than, for example, manufacturing contracts. A few prime contractors indicated that the requirement to work with SMEs can be challenging, particularly for large-value contracts, as SMEs are not seen as having the capacity to meet the requirements of the contract.

Finally, for the ITB Policy objective related to identifying skills development and training opportunities for Canadians, it was mentioned by interviewees that this objective has led to some investments in training for under-represented groups (e.g., women, visible minorities, Indigenous workers, and those with a disability).

3.2.3 To what extent has the ITB Policy led to the development of new business relationships between prime contractors and recipients?

There is evidence from the document review and survey that the ITB Policy enables SMEs to access the global value chains of multi-national firms and create export opportunities in multiple key industrial capability areas. For instance, according to the ITB Policy 2019 Annual Report, the ITB Policy enables SMEs to access the Global Value Chains of multi-national firms and creates export opportunities in multiple key industrial capability areas such as training and simulation, and space systems. Since the launch of the ITB Policy, the Value Proposition has resulted in over $2 billion in export opportunities. SMEs benefiting from these export opportunities include Alta Precision and Gastops.

Slightly more than one-third of survey respondents (35%) reported that they have developed new business relationships with prime contractors and have become qualified suppliers (46%). About one-quarter (26%) said that it increased their participation in their prime contractor's global supply chain (especially among large firms – i.e., 500 employees or more). Further, most (58%) said that the ITB Policy-related contracts helped with selling products to new clients.

According to the sub-contractors and prime contractors interviewed during the case studies, the ITB Policy contracts have allowed many sub-contractors to work for the first time for larger prime contractors. This enabled them to become compliant with the prime contractors' quality assurance system and other standards.

One Canadian company obtained a first-time contract to conduct maintenance work on aircrafts under a major prime contractor. The prime company provided training to the Canadian supplier's staff in order for them to do the work. While some sub-contractors may have worked on these aircrafts for other countries before, it was explained that all sub-contracting firms receive training due to the unique requirements of DND for these types of aircraft (about 30% of the components are unique).

Although their long-term business relationships will ultimately depend on their performance, the ITB contracts allowed them to develop new business relationships (new contracts).

3.2.4 What are the economic and social benefits of the ITB Policy?

In order to measure the impact of the ITB Policy on employment and GDP, an I/O analysis was conducted utilizing data from the results of the survey of industry (conducted specifically for this evaluation) and Statistics Canada's National Accounts and I/O tables (see Appendix A for a description of the methodology).

Overall, evaluation findings suggest that the ITB Policy is contributing to significant economic impacts, through direct expenditures in Canada arising from the ITB Policy contract obligations, as well as through spin-off contract work for firms that received contracts from prime contractors. These two impacts were reported for the 2014-15 to 2018-19 period.

The ITB Policy investments that occurred during this period represent 26% of the ITB obligation value^{Footnote 16} ($1.6 billion in investments out of $6.2 billion in ITB obligations). Investments stemming from ITB Policy contracts awarded between 2014-2015 and 2018-2019 are expected to continue until 2037.

According to the I/O analysis, contracts covered by the ITB Policy were estimated to have supported or maintained up to roughly 4,100 jobs per year during the 2014-15 to 2018-19 period (see Appendix B). In addition, jobs created via the spin-offs from these contracts were estimated to potentially add up to roughly 2,350 jobs per year. Therefore, considering the impact of the ITB Policy contracts, as well as the estimated spin-off sales, the total impact on employment was estimated to be as high as approximately 6,450 jobs per year for the 2014-15 to 2018-19 period.

The impact on Canada's GDP from the ITB Policy contracts was also estimated. For the 2014-15 to 2018-19 period, the 32 ITB Policy contracts had an estimated GDP impact of as high as

$2.1 billion (see Appendix B). In addition, the spin-offs from these contracts were expected to contribute as much as $1.2 billion to GDP during this same period. Thus, considering the impact of the ITB Policy contracts, as well as the estimated spin-off sales, the impact on Canada's GDP was estimated to be as high as $3.3 billion for the 2014-15 to 2018-19 period.

The case studies provide multiple examples of projects leading to job creation and/or maintenance, as well as business growth for the prime contractors and sub-contractors. For example:

• A contract for maintenance and engineering services for the C130 Hercules and CP140 Aurora will contribute to the creation and maintenance of hundreds of maintenance and engineering services jobs for aircraft engines.
• Contracts to repair and overhaul engines of DND's CF-188 Hornets will provide work for a significant number of full-time employees at Magellan Mississauga, in addition to workers in sub-contracting firms. One sub-contractor also expanded its workforce as a result of the contract and associated spin-off contracts.
• Contracts to build temporary shelters have allowed a Canadian firm specializing in that area to double in size.

According to interviewees, the ITB Policy, and the VPs in particular, is contributing to the development of a Canadian supplier base, as it enforces sourcing from Canada-based suppliers. The VPs helped some recipients to acquire new manufacturing techniques and improve their technical capabilities. Some Canadian recipients have moved up the supply chain (e.g., from tier 3 to tier 2 suppliers – i.e., levels of sub-contracting) and many have expanded their client base.

The survey of industry also provides evidence of impacts among recipients in terms of improved internal capacity:

• Most ITB contract recipients surveyed reported that the ITB contracts helped improve their work processes (60%) and allowed them to develop highly qualified personnel (70%).
• About half (47%) made investments to conduct the work, and about half of the projects (46%) led to the acquisition or development of new technologies.
• Slightly more than half of recipients (54%) reported that the ITB contracts contributed to their R&D and innovation capacity.
• Almost half (46%) developed prototypes.
• On average, firms provided research experience to over 10 post-secondary students on projects during the 2014-2018 period.
• About one-third (35%) reported that their contracts allowed them to hire and retain staff beyond the length of the contract.

Additionally, most recipients (57%) reported that the ITB contracts contributed to increased sales in existing lines of business and some (35%) indicated that it helped increase their share of international markets.

3.2.5 To what extent does use of the banking mechanism lead to an impact on recipients?

ISED officials interviewed generally agreed that the banking mechanism is appropriate and is being used as planned. Many felt it was a great tool within industry, as it provides motivation for contractors to work with Canadian suppliers. It allows prime contractors to "think outside the box" and think longer term. Some ISED respondents mentioned that they have seen improvements in the mechanism over time and one respondent indicated that banking activities have tripled since 2015. Most prime contractors interviewed agreed that, while it is difficult to predict, if the banking mechanism was not an option they likely would not have spent (and banked) these amounts in Canadian companies.

Prime contractors indicated that their banked transactions included a range of activities, such as exporting Canadian products, investing in universities for R&D activities, or putting them towards other areas of business that are not related to the defence sector. These were intended outcomes of the banking mechanism.

In terms of understanding, the majority of the 32 interviewed respondents indicated that the rules and regulations regarding the banking mechanism are clear. Three interview respondents indicated that some of the rules and regulations are unclear. For example, it was suggested that there is a lack of clarity regarding what business activities undertaken prior to a procurement qualify as bankable for credit against their IRB/ITB obligations later.

Among prime contractors interviewed who had banked credits, a few indicated that they have not yet used them due to an industry perception that all ITB/IRB Policy-related activities must be completed first. According to one prime contractor, their company compares various options when deciding where to manufacture and will bank only when it is attractive to do so, while another stated that they will not bank for projects that are less than $100,000 because of the difficulty involved. A third prime contractor explained that the process is complicated because it is difficult to understand what is eligible and what is not, and there is uncertainty about whether the efforts required to apply for credits will be successful.

3.2.6 To what extent is the ITB Branch research and analytics function providing value to ISED and other departments?

According to ISED interviewees, the ITB Branch research and analytics function plays a valuable supporting role to the ITB Branch and other ISED branches by conducting marketing studies, providing advice about monitoring, and producing data for reporting purposes.

When asked if the services provided by the research function were useful, most ISED respondents agreed that the ITB research function is responsive and that the information has been used for decision-making purposes. The function provides useful information for the purposes of developing RFPs, assessing industry capacity, and reporting:

• Developing RFPs: At the RFP preparation stage, the research function provides information about industry profiles and data about Canadian suppliers and markets. The information is used to validate the information provided by prime contractors when they are consulted about RFPs and the VPs.
• Industry capacity: The function provides market studies containing information about Canadian capacity to produce products and services related to upcoming contracts. For example, in the area of aerospace, the research function provided information on Canadian industry capacity to produce landing gear, engines and systems. The function has conducted searches to see if anyone in Canada has done certain types of work before and if the company can take on the obligations (i.e., if they have the resources). Information is gathered from various sources, including Statistics Canada and the federal Regional Development Agencies.
• Reporting: The function helped improve the quality of data. The Research and Analysis group analyzed the information provided by prime contractors and worked with other ISED staff to improve the forms and the data requirements. Information is also used for the ITB Policy performance report and to brief senior managers at ISED.

Outside of ISED, two of three DND respondents interviewed were less likely to require the information from the research and analytics function for their processes (i.e., being able to assess the market's ability to meet the needs of the Department), as DND is more interested in technical information. They observed that information obtained from the prime contractors during the engagement phase often met their requirements, limiting the need for market information provided by the research and analytics function. However, not all projects which leverage the function involved DND.

3.3 Efficiency

3.3.1 To what extent has the implementation of the ITB Policy demonstrated efficiency? Are there alternatives to improve efficiency?

Efficiency, or the degree to which the program is efficient in its use of resources, was assessed through a review of the administrative costs against the level of contracts covered by the Policy, as well as via interviews about internal processes.

Over the 2014-15 to 2018-19 period, actual expenditures for the ITB Branch for work involving ITB Policy contracts and the remaining IRB Policy contracts (i.e., salaries plus operations and maintenance costs) totalled $36.3 million (as shown in Table 1). Compared to the 32 contracts initiated during this period (valued at $6.2 billion), ITB administrative costs represent 0.6%.

Interviewees noted that there were no major issues or gaps with respect to the efficiency of the ITB Branch operations. Further there are no redundancies between activities within the ITB Branch, according to respondents. The ITB Branch is taking concrete steps to reduce administrative burden and improve client service. This includes streamlining the annual reporting process for contractors and implementing a six-month service standard for completing verification of annual reports. The goal of the six-month service standard is to award contractor credits in a timely fashion, standardize the annual reporting process, and be accountable and transparent in the verification processes. According to service standards data, verification activities are compliant to the six-month service standard in more than 95% of cases, significantly outpacing the targeted standard of 80%.

Many ISED respondents made suggestions to improve the consistency of the crediting process (i.e., the process whereby prime contractors submit reports to get their expenses credited as per their VPs and obligations). It was noted that the crediting process is based on a set of criteria that is subject to interpretation by ISED program officials – leading to inconsistencies. Having clear criteria ensures a more efficient crediting process, as there is less interaction required with prime contractors submitting the credits.

4.0 Conclusions and recommendations

4.1 Conclusions

Relevance

• The ITB Policy, including expenditure requirements and VPs, is aligned with ISED priorities related to the competitiveness of Canadian firms, economic growth, job creation, innovation and R&D. Further, the ITB Policy is aligned with the needs of the Canadian defence industry by supporting the growth of prime contractors and suppliers in Canada.

Performance

• Roughly one-third of ITB Policy recipients have developed new business relationships with prime contractors as a result of ITB Policy-related work – about half have become qualified suppliers, while a majority said that ITB Policy contracts allow them to sell to new clients.
• The ITB Policy is contributing to the development of a Canadian supplier base, as it enforces sourcing from Canada-based suppliers. Most recipients report that the ITB Policy contracts improved their work processes and contributed to their R&D and innovation capacity. According to data analyses, the ITB Policy was estimated to have contributed to the maintenance/creation of as many as 6,450 jobs per year in the private sector between 2014-15 and 2018-19. The impact on Canada's Gross Domestic Product (GDP) was estimated to have been as high as $3.3 billion for this period.
• The ITB Policy has led to improvements in the engagement process with industry. Early engagement has allowed industry to provide views on the development of the VP framework before the Request for Proposals are issued and engagement is generally viewed as being timely.
• Further, VPs increase benefits for Canada in many areas, including R&D, innovation, skills development, and economic development. Two-thirds of prime contractors interviewed were satisfied with the Value Proposition approach. However, one-third expressed concerns about the lack of clarity of the VP statements, their complexity and limited flexibility of the approach.
• In terms of the banking mechanism, it is generally being utilized as planned and is encouraging prime contractors to make investments in Canada.
• Finally, the ITB Branch research and analytics function is used for decision-making and reporting purposes and is deemed useful within ISED.

Efficiency

• ITB Branch operations were efficient over the 2014-15 to 2018-19 period, with no major issues or gaps with respect to the efficiency of operations identified. However, there are opportunities to improve the consistency of the crediting process (i.e., the process whereby contractors submit reports to get their expenses credited as per their VPs and obligations).

4.2 Recommendations

The evaluation findings led to the recommendations noted below.

Recommendation 1: Value Proposition Approach

ISED should continue to work with contractors to ensure that the Value Proposition approach on a procurement-by-procurement basis is as flexible as possible for prime contractors to fulfill requirements, while continuing to meet the policy objectives.

Recommendation 2: Crediting Process

ISED should enhance internal guidance related to the application of crediting guidelines for contractor expenses to improve the consistency of the crediting process.

Recommendation 3: Program Understanding

ISED should continue industry engagement and outreach activities to enhance stakeholder understanding of the ITB Policy, particularly the Value Propositions.

Appendix A – Input-Output Analysis

Input-Output Analysis Methodology

The economic impact analysis undertaken for this evaluation utilized data from the results of the survey of industry undertaken as part of this evaluation. The data obtained was then analyzed using Statistics Canada's National Accounts and Input Output (I/O) Tables. The analysis also includes an I/O analysis of the ITB Policy contracts (as a separate analysis). The I/O analysis was conducted by ISED economists.

The objective of the analysis was to identify how ITB Policy transactions affect the Canadian economy via direct purchases of goods and services and indirect demands inherent in supply chains for inputs into direct goods and services. In addition, throughout the supply chains, contractors may gain both knowledge in producing sophisticated equipment and intellectual property that allows them to gain international accreditation, and expand their offerings in both civilian and non-civilian markets, possibly through the utilization of the prime contractors' marketing channels, for spinoff sales. This analysis also measured the values of such sales.

The economic impacts comprise direct, indirect, and induced impacts:^{Footnote 17}

• Direct impacts arise when a prime contractor is awarded a defence procurement contract (as a prime contractor or sub-contractor).
• The indirect impacts are linked to the suppliers to the prime contractor but are further up the supply chain.
• The induced impacts emanate from the incremental (additional) increased incomes due to the direct and indirect impacts, and the sequential increases in income from additional rounds of induced expenditures out of derived incomes. Using its 2010 I/O Model^{Footnote 18}, Statistics Canada has estimated both the direct and indirect impacts (Open Model) and all impacts inclusive of the induced impacts arising from consumer expenditures (Closed Model). This is a partial analysis because both spending of increased government revenues and increased private sector investments are not estimated.

Process

The first step in this process was to identify the contract and follow-on or spinoff sales revenues generated at the establishment and industry level as a result of the ITB Policy contracts. The North American Industry Classification System (NAICS) codes, available at the firm level on the Industry Canada Business Profiles, or failing that contractors' websites, were used to link ITB Policy establishment data on revenues to Statistics Canada's I/O tables. Where a firm listed more than one NAICS code, the one most applicable was applied based on the description of each ITB Policy contract. Therefore, a firm with several establishments could have several NAICS codes. This process allowed the defining of the economic shock (expenditure), or ITB Policy stimulus, as incremental revenues by NAICS code at purchaser prices.

Contribution to GDP

The impacts on national income at the national level were captured by estimating incremental GDP. GDP is a gross measure of what is produced incrementally domestically in Canada prior to taking depreciation into account. This figure relates not only to the firms receiving the ITB Policy contracts, but also to all suppliers in each contractor's supply chain. Some of these contract recipients and suppliers are likely to have imported materials and supplies, upon which duties would have been paid. These and other taxes have been estimated via the Statistics Canada I/O tables. When they are subtracted from the stimulus, the I/O results yield the direct and indirect GDP impacts at market prices.

As long as there are imports, the GDP multiplier relative to the total economic shock will be fractional. The multiplier approaching or exceeding unity in the analysis comes from induced impacts when the incomes earned from the project are spent. The GDP at market prices is calculated by subtracting indirect taxes and imports on products and adding in any subsidies to the products over and above ITB Policy qualified expenditures. This metric is important since its elements identify various income streams. The direct and indirect impact on GDP and employment by industry indicate the reach of defence procurements (with ITB Policy requirements) in Canada. Since I/O analysis traces supplies back up the supply chain by project, the analysis identifies GDP and employment by industry.

Direct employment estimates

This includes both new full-time equivalents and employees who retain their positions due to ITB Policy defence procurement contracting. In this sense, they are incremental to the counterfactual of, "What would have been the relative level of employment without the funding and follow-on sales, had companies failed to find any other markets?" They may not all be incremental since the companies throughout the supply chain may have found other work or horded Highly-Qualified Persons (HQPs) and other staff in hopes of retaining intellectual capital. Conversely, some prime contractors may not have had a Canadian presence at all without the ITB Policy's impact on defence procurement.

Other assumptions and considerations included the following:

• ITB Policy investments are based on Canadian Content Value credited according to the ITB Policy before credit multipliers are applied;
• The economic model is based on the latest (2015) and closest related Statistics Canada Input-Output (I/O) GDP and jobs multipliers that correspond to the investment activities (forecasted investments have been adjusted for inflation based on the Bank of Canada's 2% inflation rate and all economic model results are in 2015 dollars);
• The GDP impact is reported on an annual average and cumulative basis; and
• The job impact is reported on an annual average basis and measured in terms of full-time equivalent employment.

Data Sources

In addition to Statistics Canada I/O tables, the analysis was conducted using ITB Policy program data between 2014 and 2024 and data from a survey of industry that was conducted as part of the evaluation. The I/O analysis includes only the 32 contracts awarded under the ITB Policy. All IRB Policy contracts are excluded from the analysis, as they are outside the scope of this evaluation.

Each of the 32 ITB Policy contracts are listed below, along with their year of award and obligation value.

Year Awarded	ITB Policy Project	Obligation Value
2018	Chemical, Biological, Radiological and Nuclear Respirator - Acquisition (GSR-A)	$25,946,343
2018	Chemical, Biological, Radiological and Nuclear Respirator (GSR-I)	$2,164,166
2018	Land C4ISR Sustainment Project - Cyber Security Engineering Services (LC4-SC)	$50,000,000
2018	Land C4ISR Sustainment Project - Engineering and Integration (LC4-SE)	$325,000,000
2018	Land C4ISR Sustainment Project - ISTAR (LC4-SI)	$50,000,000
2018	Land C4ISR Sustainment Project - Software Support (LC4-SS)	$175,000,000
2018	Maritime Unmanned Aerial System - Interim Capability (CAF UAS)	$60,437,840
2018	Mercury Global—Strategic Deployable Terminals (MG SDT)	$28,257,799
2018	Underwater Warfare Sensor Suite Upgrade (UWSU)	$97,523,219
2017	Arctic Offshore Patrol Ship and the Joint Support Ship - Maintenance (AJISS)	$4,821,650
2017	Buffalo and Cougar Armoured Vehicles - Maintenance (EROC ISS)	$11,750,000
2017	C130-CP140 Aircraft Engine— Long Term Maintenance (T56-O)	$107,282,053
2017	CF-18 Engines - F404 Long-term Maintenance (F404-O)	$106,434,421
2017	Contracted Airborne Training Services (CAT S)	$407,680,394
2017	Headquarters Shelter Systems—Acquisition (HQSS-A)	$170,031,829
2017	Headquarters Shelter Systems—Maintenance (HQSS-I)	$12,044,381
2017	Phalanx Naval Weapon System - Support (PHALX2-I)	$330,038,170
2017	Victoria Class Command and Control System – Maintenance (VCCCS)	$18,270,000 (USD)
2016	Combat Net Radio Enhanced - Maintenance (CNRE-I)	$33,867,771
2016	CT114 Tutor Fleet - Maintenance (CT114 PAV)	$18,400,000
2016	Fixed-Wing Search and Rescue Aircraft - Acquisition (FWSAR-A)	$1,925,195,879
2016	Fixed-Wing Search and Rescue Aircraft - In-Service Support (FWSAR-I)	$578,793,661
2016	Maritime Satellite Communications Upgrade (MSCU)	$25,012,840
2016	Naval Remote Weapon Station (NRWS)	$37,191,779
2016	Polar Epsilon 2 (PE2)	$57,593,968
2015	C130J Tactical Airlift — Maintenance 2 (C130J-IP2)	$708,676,610
2015	CP-140 Aurora - Data Management System - Block IV (CP140BIV)	$285,111,825
2015	Diesel Generator Sets (DGSet)	$52,465,693
2015	Medium Range Radar - Acquisition (MRR-A)	$138,500,782
2015	Medium Range Radar - Maintenance (MRR-I)	$8,406,404
2014	C17 Strategic Airlift—Acquisition—5th Aircraft (C17-A 5)	$264,500,000
2014	Canadian Coast Guard Helicopter—Medium (CCGHM)	$102,341,743
TOTAL	-	$6,218,741,220

Appendix B – Economic Impact of ITB Policy

Three scenarios were developed to assess the economic impact of the ITB Policy, including the Value Proposition. The assessment only includes 32 contracts awarded since 2014-15 under the ITB Policy and to which the Value Proposition was applied. This does not include contracts awarded under the National Shipbuilding Strategy, or contracts awarded under the IRB Policy.

2014-15 to 2018-19 Period	Pessimistic Scenario: Spin-off for sample respondents only and 50% of direct benefits attributable to the ITB Policy	Mid-Range Scenario: Spin-off for sample respondents and 50% of non-respondents and 75% of direct benefits attributable to the ITB Policy	Optimistic Scenario: Spin-off impacts of sample extended to population of recipients and 100% of direct benefits attributable to the ITB Policy
Impact on GDP from ITB Policy contracts	$1,040,423,276	$1,560,634,914	$2,080,846,552
Impact on GDP from spin-off revenues	$161,303,877	$681,463,125	$1,201,622,373
Total impact on GDP	$1,201,727,153	$2,242,098,039	$3,282,468,925
Impact on employment from ITB Policy contracts	2,052	3,078	4,104
Impact on employment from spin-off revenues	358	1,355	2,353
Total impact on employment	2,410	4,433	6,457

Explanation of scenarios

As shown in the table, the economic impacts are of two types: direct impacts from the contracts covered by the ITB Policy, and the spinoff contracts (past and projected) for the recipients (sub-contractors). While there are program and survey data that can be used to assess these impacts, there are two factors that prevent the evaluation from producing a single statement/number about the net impacts of the program.

• First, the net impacts of the contracts covered by the ITB Policy cannot be measured as there was no comparison group to assess the net incremental impact (this was not deemed feasible). In other words, while one can assume that the policy led to economic activity in Canada, it is not known to what extent the policy alone actually increased the economic activity in Canada. There are, however, key informant interview findings (with prime contractors) that confirm that the larger international firms would have manufactured the products in their establishments in other countries where production costs are lower, if the ITB Policy was not in place. Some explained that other countries also have offset policies, and that they constantly seek opportunities to conduct work in these countries to meet the offset requirements of these countries. Without the ITB Policy, these firms would therefore seek to conduct work in other countries.
• The second factor is associated with the limitations of the survey conducted as part of the evaluation. While the response rate (23%) is definitely in line with similar surveys conducted for other evaluations (including the previous IRB Policy evaluation), the fact remains that the results are only based on a sample.

Based on the above, the evaluation provides results under three scenarios.

Scenario 1: Pessimistic

The pessimistic scenario is based on two assumptions: 1) spinoff impacts are limited to the sample of respondents only – no impacts are attributed to those who did not respond to the survey; and 2) impacts attributed to the directly associated ITB Policy contract dollars are estimated at 50% - in other words, the net impact of the policy is set at 50% of the contract dollars (i.e., the other 50% would have happened in the absence of the policy).

Scenario 2: Mid-Range

Under the mid-range scenario, the impacts of the spin-off projects include those reported by the survey respondents, and are extended to half of the remaining sample (that did not respond to the survey). In other words, this scenario assumes that the sample is only representative of about 75% of the recipient population – and that no impacts are attributed to 25% of the remaining non-responding recipients. This remains fairly conservative given that the statistical margin of error of the obtained sample is approximately 12%. It should also be mentioned that there was no evidence (based on the results of the follow-up phone calls during the conduct of the survey) that the non-responding recipients had closed their operations. As for the direct economic impacts of ITB Policy contracts, this scenario attributes a net impact of 75%. In other words, 75% of the impacts associated with the contract dollars are attributed to the ITB Policy. Again, this is considered fairly conservative considering that most of the prime contractors are large international firms, and that interview evidence indicates that they would likely conduct their work through foreign establishments. However, it is still likely that some of the work (e.g., maintenance work) would have been conducted in Canada by Canadians.

Scenario 3: Optimistic

The optimistic scenario basically assumes that the survey sample is representative of all recipients, that is, that the results of the sample (spin-off contracts) can be fully extended to the entire recipient population. It also assumes full attribution of the economic impact to the ITB Policy.