Evaluation of the Fairness at the Pumps Act (FAPA)

May 2020

This publication is available online at http://www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03911.html.

To obtain a copy of this publication, or to receive it in an alternate format (Braille, large print, etc.), please fill out the Publication Request Form at www.ic.gc.ca/publication-request or contact:

ISED Citizen Services Centre
Innovation, Science and Economic Development Canada
C.D. Howe Building
235 Queen Street
Ottawa, ON K1A  0H5
Canada

Telephone (toll-free in Canada): 1-800-328-6189
Telephone (international): 613-954-5031
TTY (for hearing impaired): 1-866-694-8389
Business hours: 8:30 a.m. to 5:00 p.m. (Eastern Time)
Email: ISED@Canada.ca

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from the Department of Industry, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that the Department of Industry is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, or as having been made in affiliation with, or with the endorsement of, the Department of Industry.

For permission to reproduce the information in this publication for commercial purposes, please fill out the Application for Crown Copyright Clearance at www.ic.gc.ca/copyright-request or contact the ISED Citizen Services Centre mentioned above.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of Industry, 2020.

Cat. No. Iu4-307/2020E-PDF
ISBN 978-0-660-35169-8

Aussi offert en français sous le titre Évaluation de la Loi sur l'équité à la pompe.

List of abbreviations and acronyms

AEB
Audit and Evaluation Branch
AMPs
Administrative Monetary Penalties
ASD
Alternative Service Delivery
ASP
Authorized Service Provider
EGIA
Electricity and Gas Inspection Act
FAPA
Fairness at the Pumps Act
ISED
Innovation, Science and Economic Development
MMP
Marketplace Monitoring Program
MRR
Management Review Report
MC
Measurement Canada
ORA
Online Reporting Application
QMS
Quality Management System
RT
Recognized Technician
SBMS
Small Business and Marketplace Services Sector
WMA
Weights and Measures Act

List of figures

  1. Figure 1: FAPA Implementation Milestones
  2. Figure 2: WMA Inspection Process
  3. Figure 3: Trade Measurement Logic Model
  4. Figure 4: Measurement Compliance Rate in Unbiased Inspections Conducted by MC in sectors not covered by FAPA for 2018-19
  5. Figure 5: Demographics of MC Inspectors and RTs

List of Tables

  1. Table 1: Number of Inspections by Inspection Type – 2010-11 to 2018-19
  2. Table 2: Number of WMA Enforcement Actions – 2014-15 to 2018-19
  3. Table 3: Measurement Compliance Rates in Re-inspections
  4. Table 4: Operating Budget for the MC ($ millions), 2009-10 to 2018-19

Executive Summary

This report presents the results of an evaluation of the Fairness at the Pumps Act (FAPA), which amended the Electricity and Gas Inspection Act (EGIA) and the Weights and Measures Act (WMA) and related regulations to protect Canadian consumers from inaccurate measurement when purchasing measured goods.

Overview

In April 2010, the government announced FAPA which introduced mandatory re-inspection in eight trade sectors (retail petroleum, mining, dairy, downstream petroleum (wholesale), fishing, forestry, grain and field crops, and retail food) and new enforcement measures in order to strengthen consumer protection. FAPA also enables non-government inspectors (i.e., Authorized Service Providers – ASPs and their Recognized Technicians – RTs) to inspect and certify weight and measurement devices (e.g., gas pumps). Measurement Canada (MC) is responsible for the implementation of FAPA, which came into force in August 2014.

MC is a special operating agency within the Small Business and Marketplace Services Sector of Innovation, Science and Economic Development Canada (ISED). It ensures the integrity and accuracy of trade measurement in more than 40 sectors in Canada through the administration and enforcement of the WMA and EGIA, and their related regulations. MC's role is to protect Canadians against loss due to inaccurate measurement of devices used in trade when purchasing measured goods. MC has 19 district and regional offices located across Canada, and its headquarters and laboratory test facilities in Ottawa. For fiscal year 2018-19, MC had approximately 300 employees and a budget of approximately $25.7 million.

Evaluation Purpose and Methodology

The objectives of this evaluation were to examine the performance and efficiency of FAPA in accordance with the Treasury Board Policy on Results.  The evaluation covered the period from April 1, 2014 to March 31, 2019 and employed multiple data collection methods: a document review; literature review; surveys; interviews; and an administrative data analysis.

Findings

Performance

FAPA increased the number of mandatory re-inspections of devices in the eight regulated trade sectors. In 2010-11 (i.e., pre-FAPA), about 6,000 re-inspections were conducted. When FAPA was introduced, the number of re-inspections increased from 23,000 in 2013-14 to more than 110,000 in 2018-19. However, more than 30 other trade sectors are not subject to the same level of oversight. The measurement compliance rate in some sectors not covered by FAPA was as low as 53.5% for quarries and sandpits, 60.5% for scrap metal, and 64.0% for waste management in 2018-19 (recognizing these can naturally be more challenging environments to maintain device accuracy), compared to an average of 94.3% in the 8 trade sectors covered by FAPA.

In addition, FAPA contributed to enhancing MC's enforcement capability. Overall, education and information provided to device owners, as well as warnings provided to them for offenses, have been effective compliance measures. Although administrative monetary penalties (AMPs) have been used to a limited extent to date, they are generally viewed as having a deterrent effect.

As a result of the mandatory re-inspections and MC's enforcement capability, FAPA has increased coverage in the marketplace (i.e., more devices are re-inspected every year) and to some extent device accuracy. Device accuracy should continue to be monitored as most devices used in the retail food sector, for example, have only been through one cycle of mandatory re-inspection since 2014.

Since FAPA came into effect, retailers and device owners have access to a large national network of ASPs and RTs, but with a limited presence and at a higher cost in northern and remote areas. As the inspection fees in those areas can be up to four times greater than the margin of profit obtained from a device, some device owners could question the viability of maintaining their devices in operation (e.g., those used in "Mom and Pop shops").

Although RTs are qualified and competent to perform inspections and their training is seen to be adequate, there is a need for additional learning tools and refresher courses to enable RTs to maintain their knowledge of regulations and procedures. Furthermore, it was noted that FAPA may have inadvertently limited MC inspectors' exposure to specialized devices, and thereby the knowledge needed to inspect them, due to the new role of MC inspectors (i.e., monitoring ASPs).

Efficiency

In 2010, $8 million was committed for the implementation of FAPA over 5 years (2010-11 to 2014-15), with $2 million in ongoing funding thereafter. This funding was used to increase resources for MC inspectors, in addition to recruiting, training and monitoring non-government inspectors. MC utilized financial resources provided for the implementation of FAPA efficiently, including absorbing some of these costs and implementing an Online Reporting Application (ORA) to replace the inefficient manual paper-based MC data entry approach.

When FAPA was launched, MC mainly relied on a few major associations and its website to communicate the changes, which led to challenges for device owners to understand the new requirements. Although MC has since then strengthened its external and internal communication with stakeholders, and device owners are more educated in respect of their responsibilities due to mandatory re-inspections, there is still a general lack of awareness among Canadians and device owners of MC's mandate and role, as well as the implications and results of FAPA. For example, during the evaluation process, there was no available data on MC activities (e.g., inspections, enforcement actions, audits, complaints) and results (e.g., compliance rate) for Canadians. Making available reliable data would support consumer confidence in the accuracy of measurement-based transactions.

Recommendations

The evaluation findings led to the recommendations noted below.

Recommendation 1: Accuracy and fairness in the marketplace

MC should explore risk-based approaches, including periodic re-inspections, to improve accuracy and fairness in trade sectors not currently overseen by FAPA.

Recommendation 2: ASPs and RTs availability in northern and remote areas

MC should explore alternative approaches to enable efficiencies for re-inspections in northern and remote areas and access to ASPs (e.g., greater coordination of re-inspections).

Recommendation 3: Training and knowledge transfer

MC should facilitate access to online refresher courses and learning tools for RTs and consider implementing strategies to facilitate knowledge transfer for MC inspectors.

Recommendation 4: Communication with device owners and Canadians

MC should improve communication with device owners and Canadians, including easily accessible information on activities (e.g., enforcement actions) and results (e.g., compliance rates).

1.0 Introduction

This report presents the results of an evaluation of the Fairness at the Pumps Act (FAPA), which came into force in August 2014. FAPA amended the Electricity and Gas Inspection Act (EGIA) and the Weights and Measures Act (WMA) and related regulations to introduce mandatory re-inspections in eight trade sectors and new enforcement measures in order to strengthen consumer protection. FAPA also enabled non-government inspectors (i.e., Authorized Service Providers – ASPs and their Recognized Technicians – RTs) to inspect and certify weight and measurement devices (e.g., scales in groceries and gas pumps). Measurement Canada (MC) is responsible for the implementation of FAPA, including legislative and regulatory changes and its enforcement.

This evaluation assesses the performance and efficiency of FAPA and is organized as follows:

  • Section 1 provides the context and background on the role of MC and FAPA;
  • Section 2 presents the evaluation methodology and the challenges of the evaluation;
  • Section 3 presents the findings; and
  • Section 4 summarizes the conclusions and provides recommendations.

At a glance:

  • FAPA enabled amendments to the Weights and Measures Act and Regulations, and the Electricity and Gas Inspection Act to introduce:
    • Mandatory re-inspection frequencies.
    • Recognition of non-government inspectors.
    • New enforcement measures (i.e., increased fines for non-compliance and administrative monetary penalties).
  • MC established mandatory re-inspections in eight sectors (retail petroleum, retail food, dairy, downstream petroleum (wholesale), fishing, forestry, grain and field crops, and mining).

1.1 Background

MC is a special operating agency within the Small Business and Marketplace Services Sector of Innovation, Science and Economic Development Canada (ISED). Through its administration and enforcement of the WMA and EGIA (and their related regulations), MC ensures the integrity and accuracy of trade measurement in more than 40 sectors in Canada, including transportation and precious metals.

MC's headquarters and laboratory test facilities are based in Ottawa, supported by a total of 19 district and regional offices located across Canada. For 2018-19, the agency had approximately 300 employees and a budget of approximately $25.7 million.

To fulfill its mandate, MC provides a wide variety of programs and inspection services, including:

Text version

This figure is a visual representation of representation of Measurement Canada's programs and services.

Calibration: MC calibrates and certificates standards of mass, length, volume, pressure, temperature and various electrical quantities, so that measurement in domestic and international trade will be uniform. These standards from the basis of all trade measurement in Canada.

Device Approvals: MC tests prototype measuring instruments for compliance with legislated requirements to ensure that they are capable of measuring accurately under normal conditions of use and throughout their service lifetime. For those that comply, MC certifies and approves them for retail and commercial use in Canada.

Inspection, Certification, Enforcement & Complaints: MC inspectors inspect and certify devices to ensure that they measure accurately and are not used fraudulently. When inaccurate measurement or fraudulent activity is found, MC uses a graduated enforcement approach to resolve non-compliance, including warning letters and removal of devices from service until corrective actions are taken. MC also investigates complaints it receives from those who suspect they have received an inaccurate measurement, or are dissatisfied with the results of their measurement transaction, and takes corrective actions to resolve non-compliance.

Marketplace Monitoring Program (MMP): MC uses the MMP to collect data accuracy performance of devices and net quantity transactions within trade sectors to identify and address perceived, potential and actual problem areas.

Oversight of Authorized Service Providers (ASPs): MC designates ASPs the authority to inspect and certify weighing and measuring devices and electricity and natural gas meters for use in trade measurement transactions. MC monitors all ASP organizations on a regular basis through surveillance audits, product audits, follow-up inspections and annual visits to ensure they continue to maintain the necessary inspection competencies and are certifying devices and meters correctly. 

These programs and services aim to instill confidence in the integrity and accuracy of measurement-based transactions, minimize the occurrences of financial loss due to inaccurate measurement, and provide an avenue of recourse to those who feel they have not received the amount of measured product or service they paid for.

1.2 Context

Prior to the mid-1970s, MC inspectors were required to annually re-certify the accuracy of every device used in a trade measurement transaction in Canada.  The goal of these re-inspections was to ensure that the device was still measuring goods and services accurately, and not being used fraudulently.

This approach was resource intensive and in the mid-1970s, the government changed the requirement for annual, mandatory re-inspections, to be performed every two years under the WMA. In the early 1980s, MC's requirement for mandatory periodic re-inspections was removed, instead opting for selected or targeted inspections based on devices deemed to be problematic and having a lower compliance rate.Footnote 1 

As a result of these changes, evidence demonstrates that measurement compliance rates trended downward over time. Further, the decline in measurement accuracy affected consumers and small businesses by lowering their confidence in the accurate measurement of goods and services traded in Canada.Footnote 2

The government announced FAPA in 2010 to improve consumer confidence and ensure Canadians receive the amount that they pay for, by making retailers more accountable for the accuracy of their devices and introducing stricter enforcement measures to deter measurement inaccuracy. Specifically, legislative and regulatory amendments enabled MC the ability to: perform mandatory re-inspections in eight marketplace sectors, phased in based on postal code and geographic location of the deviceFootnote 3 (refer to Figure 1); source non-government inspectors to inspect and certify weight and measurement devices; and introduce new enforcement measures.

Figure 1 – FAPA Implementation Milestones

Text version

Figure 1 presents a time line graph that shows FAPA implementation milestones.

  • In April 2010, FAPA was announced.
  • In March 2011, FAPA received Royal Assent.
  • In August 2014, FAPA came into force.
  • From August 2014 to August 2016, FAPA was phasing-in.
  • As of August 2016, FAPA's phase-in period ended for seven of the eight trade sectors (exception was retail food sector).
  • In August 2017, FAPA's phase-in period ended for retail food sector.
  • As of August 2017, FAPA was fully implemented. 

1.3 Legislative and Regulatory changes to the WMA and the EGIA

The objective of the legislative and regulatory changes was to reduce measurement errors and increase fairness in the Canadian marketplace in an efficient and innovative way. These changes also aimed to strengthen consumer protection against unfair retailer practices by placing responsibility for device accuracy on device owners. FAPA enabled amendments to introduce:

1.3.1 Mandatory re-inspections (WMA):

  • Introduced mandatory re-inspection frequencies in eight trade sectors (retail petroleum, mining, dairy, downstream petroleum (wholesale), fishing, forestry, grain and field crops, and retail food) for measuring devices such as scales and petroleum meters.
  • Re-Inspection frequencies vary by trade sector and by type of device, from every year (e.g., fishing or forestry products) to every two years (e.g., mining products, fuel pump), and up to every five years (in retail food sector).
  • Mandatory re-inspection has been in place for many years under the EGIA.

1.3.2. Authorized service providers (WMA):

  • Enabled designation of non-government inspectors (i.e., ASPs and their RTs) to inspect and certify weight and measurement devices.
  • ASPs can be recognized by MC to conduct inspections on devices through an accreditation program or a registration program.
    • An accredited organizationFootnote 4 must document, establish and maintain a quality management system which meets the requirements of MC's accreditation standard.
    • A registered organization is more limited in inspection services it can provide and does not require a quality management system. As a result, there is more frequent MC monitoring of RTs working for registered organizations. 

1.3.3. New enforcement measures (WMA and EGIA):

  • Established Administrative Monetary Penalties (AMPs) allowed MC to employ a graduated enforcement approach in which the government could apply a strategy appropriate to resolve non-compliance — with AMPs for relatively minor offences and prosecution for serious or repeated offences. These AMPs range from $250 for each minor violation up to $2,000 for each serious violation. To facilitate the implementation of AMPs, MC initially launched an AMPs pilot phase in 2015.
  • Increased court-imposed fines (through prosecution) for businesses that do not meet measurement accuracy requirements under the WMA and EGIA of up to $10,000 (from $1,000) for minor offences and up to $25,000 (from $5,000) for major offences. As well, FAPA introduced a new fine of up to $50,000 for repeat offences.

Following the introduction of FAPA, the newly introduced mandatory re-inspections have been primarily conducted by RTs working for ASPs. In addition, RTs have been mainly responsible for conducting initial factory inspections. This enabled MC inspectors to focus on performing follow-up inspections and audits of ASPs (to ensure inspections are conducted correctly).

In addition, MC inspectors continued to assess the marketplace performance through random and selective inspections through the MMP (especially on the 33 other trade sectors not covered by FAPA under the WMA) and responding to complaints of suspected inaccurate device measurement. Figure 2 provides additional information on the WMA inspection process. It should be noted that MC is solely responsible for enforcement actions taken to resolve measurement inaccuracy or other types of non-compliance.

Figure 2 – WMA Inspection Process
Text version

Figure 2 provides additional information on the WMA Inspection Process for Accredited and Registered ASPs (Authorized Service Providers) and for MC (Measurement Canada) Inspectors.

Accredited and Registered ASPs

  1. Initial factory inspection:
    The first inspection of a device performed at a manufacturer's or dealer's premises by an RT (includes all examinations until the device has been certified for trade use for the first time).
  2. Initial field inspection:
    The first inspection of a device, performed by an RT, the performance of which cannot be determined until it is installed on site for use in trade (includes all inspections until the device has been certified for trade use for the first time).
  3. Re-inspection:
    Any inspection of a device performed by an RT, other than an initial factory or field inspection. Mandatory for eight trade sectors under FAPA.
  4. Re-inspection of non-complying device:
    Inspection of a device performed within six calendar months following an inspection that has determined the device did not meet the requirements of the WMA and Regulations. The inspection is to determine whether the device now conforms to the Act or the regulations.

MC Inspectors

  • Audits:
    MC performs regular surveillance audits of accredited organizations' quality management systems as well as product audits of devices examined by the accredited organizations.
  • Follow-up inspection:
    MC conducts annual meetings with registered service providers and performs in presence and follow-up inspections to monitor the work these organizations are authorized to conduct.
  • Inspection following a complaint:
    MC investigates consumer and business complaints of suspected inaccurate measurement and institutes corrective actions when inaccurate measurement is found.
  • Marketplace Monitoring Inspection:
    MC conducts random and targeted inspections of devices and commodities as part of its marketplace monitoring. This allows MC to evaluate the overall status of accuracy and equity in the marketplace, to detect problems and take appropriate action, and to maintain an independent presence in the marketplace.

1.4 Program Resources

To implement FAPA, in 2010, the government committed $8 million over 5 years (2010-11 to 2014-15), and $2 million per year in ongoing funding. MC used funding to increase the number of MC employees, in addition to recruiting, training and monitoring non-government inspectors.

2010-11 2011-12 2012-13 2013-14 2014-15 Total Ongoing

$550,000

$1,500,000

$2,000,000

$2,000,000

$2,000,000

$8,050,000

$2,000,000

1.5 Logic Model

The trade measurement logic model (Figure 3) was developed in 2017 to describe MC's activities, outputs, and intended outcomes, including those related to FAPA such as the activities, outputs, and outcomes related to ASPs (third stream).

Figure 3. Trade Measurement Logic Model – Measurement Canada

Text version

Figure 3 is a visual representation of Measurement Canada's Trade and Measurement program's activities, outputs and intended outcomes, including those related to FAPA such as the activities, outputs, and outcomes related to ASPs (third stream: Immediate outcome: APS's competences to certify measuring devices are confirmed annually, Intermediate outcome: Retailers and other business owners have measure of choice when selecting an ASP to perform mandatory device re-certifications). 

The four activities are:

  1. Test and evaluate prototype measuring devices (scales, gas pumps, electricity, and natural gas meters, etc.)
  2. Conduct inspections; investigate complaints of suspected inaccurate measurement, institute corrective and enforcement actions to resolve non-compliance.
  3. Audit, evaluate and recognize private sector organizations to perform measuring device inspections.
  4. Stakeholder consultation, environmental scans (e.g., new measurement technology development, changing business practices), participation in international technical working groups.

The four outputs are:

  1. Approval of measuring devices for use in trade measurement transactions.
  2. Inspection and certification of measuring devices used in trade measurement transactions.
  3. Monitoring of ASPs in accordance with program criteria to ensure retention if required competences and correct certification of devices.
  4. Regulations, technical specifications, policies and procedures governing trade measurement transactions.

The four immediate outcomes are:

  1. International and domestic device manufacturers receive timely prototype device approvals enabling them to sell their measuring devices for use in Canada.
  2. Measuring devices used in trade measurement transactions comply with legislative requirements for accuracy, performance and use.
  3. ASP's competencies to certify measuring devices are confirmed annually.
  4. Timely and informed development of requirements governing trade measurement transactions that reflect stakeholder needs and new measurement technologies and business practices.

The four intermediate outcomes are:

  1. International and domestic device manufacturers are satisfied with approval evaluation services received from Measurement Canada.
  2. Retailers and other device owners maintain accurate devices.
  3. Retailers and other business owners have measure of choice when selecting an ASP to perform mandatory device re-certifications.
  4. Rules and requirements governing trade measurement are relevant to reflect international model laws.

The one ultimate outcome is: "Accurate and reliable trade measurement in Canada."

2.0 Methodology

This section provides information on the evaluation objective and scope, issues and questions, data collection methods, and limitations.

2.1 Evaluation Objective and Scope

The evaluation of FAPA is a Treasury Board submission requirement and was conducted by ISED`s Audit and Evaluation Branch. The objective of the evaluation is to assess the impact of FAPA, including effectiveness and efficiency between 2014-15 and 2018-19. The evaluation focused on progress that MC has made toward the FAPA-related immediate and intermediate outcomes (identified in blue in the logic model in Figure 3). Given that FAPA only came fully into effect in August 2017, it is too early to assess the ultimate outcome.

In 2017, ISED's Audit Directorate completed an audit of MC. The objective of the audit was to provide assurance that MC's activities support the effective administration of the legislation governing trade measurement. The audit found that MC had:

  • established a management control framework where roles and responsibilities are defined and communicated, key decision-making mechanisms are in place, and a structured model is used for mandatory inspection training;
  • key controls in place to support the administration of the WMA and associated regulations, and to monitor ASPs and RTs as part of MC's Alternative Service Delivery Program; and
  • complaint mechanisms in place and publicly available for consumer complaints of inaccurate measurement.

In addition, the audit noted some opportunities for improvement in the areas of risk management, marketplace monitoring, new enforcement measures, and change management (e.g., implementing a national communication strategy).

2.2 Evaluation Issues and Questions

The evaluation addressed the following questions:

Performance

  1. To what extent has FAPA contributed to increased mandatory inspection frequencies for measuring devices?
  2. To what extent has FAPA contributed to enhancing MC's enforcement strategy?
  3. To what extent have retailers and other device owner's maintained accurate measuring devices as a result of FAPA?
  4. To what extent do retailers and other device owners have access to a national network of authorized service providers as a result of FAPA?
  5. To what extent has FAPA contributed to authorized service providers' competencies to conduct mandatory inspections?

Efficiency

  1. To what extent is FAPA being implemented efficiently?

2.3 Data Collection Methods

Multiple lines of evidence were used to address the evaluation questions.  The data collection methods included a document review, literature review, surveys, interviews, and an administrative data analysis.

Document Review

A document review was conducted to gain a thorough understanding of FAPA and provide insights into its implementation and performance. The review included:

  • Foundational documents (e.g., Treasury Board Submissions, Terms and Conditions, Contribution Agreements, etc.);
  • Government priority setting documents (e.g., Budgets, Speeches from the Throne, mandate letters, etc.); and
  • Other key program documents (e.g., industry reports, etc.).

Literature Review

The literature provided further insights into the performance and efficiency of FAPA. The review included:

  • Review of best practices in the areas of inspection and enforcement activities and their impact on compliance;
  • Review of alternative delivery models in other countries (e.g., United States, United Kingdom, France, and Australia); and
  • Academic literature and articles which could support the areas noted above.

Surveys

Two online surveys were conducted between July 16 and August 9, 2019, inviting 604 technicians and 151 MC inspectors to participate. A total of 194 technicians and 90 inspectors completed a survey, for response rates of 33.9% and 76.8% respectively.

Interviews

The objective of the interviews was to gather in-depth information related to the impact of FAPA in meeting MC outcomes and on the efficiency of FAPA's implementation. The interviews were semi-structured in nature to enable the collection of qualitative information from a range of key stakeholders.  Interviews were conducted either in-person or by telephone, as required. 

Stakeholders interviewed as part of the evaluation included 36 representatives from the following groups:

  • ISED and MC management;
  • MC regional staff;
  • ASP organizations;
  • Association of measuring device owners; and
  • Industry consumer network and experts.

Administrative Data Analysis

The administrative data review included an analysis of measurement accuracy and compliance data to assess the extent to which FAPA has achieved its expected immediate and intermediate outcomes. As well, administrative and operational data were reviewed to assess the efficiency of FAPA implementation in relation to mandatory inspections of relevant trade sectors. The data analyzed included:

  • administrative data collected through the Online Reporting Application (ORA), which was put in place in 2015-16 to replace the manual paper-based MC data entry approach, allowing for real-time and self-service data entry and reporting by RTs and MC inspectors; and
  • a review of reports prepared by MC such as the Management Review Reports from 2009 to 2019.

2.5 Limitations

Implementation of mandatory re-inspections

The implementation of mandatory re-inspections was phased-in over a two-year period (three years in retail food) with full implementation in August 2017. As a result, less than two years of data has been collected since the full implementation of FAPA which does not allow for a time-series analysis (i.e., a series of data over an extended time period). The evaluation mitigates this by identifying impacts occurring during the implementation period and those occurring after that period, and corroborating the findings with statistical analyses.

Data Issues

Several key informants (KIs) stated that ORA might present data inconsistencies with some records.  KIs indicated that in the past a portion of ASPs and RTs entered the final inspection result (after fixing the device if required) as the "as found" option (measurement compliance upon arrival at an establishment, prior to any adjustment or repair). As a result, the "as-found" compliance rate reported by RTs in ORA might be erroneously too high. To address this, the evaluation triangulated and validated performance data with other lines of evidence.

Furthermore, some information on device inspection was not required to be collected prior to the implementation of ORA. To address this, in the data analysis conducted for the evaluation, missing information from a respective device has been substituted (imputed) with available information from other inspections conducted on the specific device, where it was possible.

Potential respondent bias

Some of the stakeholders interviewed for this evaluation have a vested interest in the group that they represent or were significantly impacted by the legislative change (e.g., device owners, ASPs, etc.). The evaluation mitigated this by interviewing and surveying participants across the different stakeholder categories and by incorporating other lines of evidence (e.g., document review, literature review, and data analysis) into the analysis.

3.0 Findings

3.1 Performance

3.1.1 To what extent has FAPA contributed to increased mandatory inspection frequencies for measuring devices?

Key Finding: FAPA increased the number of mandatory re-inspections of devices in the eight regulated trade sectors under the WMA. However, more than 30 other trade sectors under the WMA are not subject to the same level of oversight.

Re-inspections

Before any device can be placed in service for trade purposes, it must be initially verified in accordance with the Acts, regulations and device verification specifications to ensure it functions properly and has acceptable metrological characteristics. The vast majority of initial verifications are performed by ASPs (between 25,000 and 30,000 inspections per year). As device accuracy can deteriorate with usage and time, initial verification alone is often insufficient to ensure that the standards of measurement are being maintained. Consequently, the device must be subsequently re-inspected at periodic intervals throughout its useful life.Footnote 5

An Organization for Economic Co-operation and Development (OECD) report on Regulatory Enforcement and Inspections Toolkit notes that "inspections are one of the most important ways to enforce regulations and to ensure regulatory compliance"Footnote 6. In addition, the report specifies that "the frequency of inspections and the resources employed should be proportional to the level of risk".

Between 1994 and August 2014, there was no legal obligation for re-inspecting a device.  As a result, the number of re-inspections has significantly increased with the introduction of FAPA. In 2010-11, about 6,000 re-inspections were conducted. When FAPA was introduced, the number of re-inspections increased from 23,000 in 2013-14 to more than 110,000 in 2018-19, an increase of 384% (see Table 1). Most of the increase occurred during the phasing-in period and has since declined slightly. It should be noted that the decline in the number of re-inspections is almost exclusively due to the retail food sector. As most of devices in that sector were inspected during the phase-in period, they do not have to be re-inspected for another five years. Although the 2018-19 Management Review Report mentions that the number of re-inspections conducted by weight and measurement ASPs may have reached its plateau since FAPA has been fully implemented, MC should expect an increase in the number of re-inspections in 2020-21 and 2021-22 as many devices in the retail food sector will need to be re-inspected.

Total inspections

The administrative data analysis also shows that FAPA has increased the total number of inspections by 119% since 2013-14 (see table 1). While the number of inspections conducted by registered and accredited ASPs increased significantly (180%) over the same period, the number of inspections conducted by MC decreased by 58%.

This was expected given the introduction of mandatory re-inspections conducted by ASPs and the change in MC's role from conducting inspections to monitoring ASPs doing this work. As a result, the number of follow-up inspections performed by MC (to monitor registered ASPs) and those conducted as part of audits (to monitor accredited ASPs), increased (from about 250 inspections per year to about 1,700 per year in 2018-19).

However, the number of unbiased (randomly) and selective inspections conducted by MC through its MMP declined since FAPA was introduced. As periodic inspections are conducted by RTs, MC decided in 2015-16 to reduce its MMP inspections to 2% of devices (from a sample of 5%) in the eight trade sectors regulated by FAPA while increasing it in the other trade sectors to 20% (from 5%). In 2016-17, MC reduced MMP inspections in the other sectors to a 10% sample for that year and the following years.

Table 1– Number of Inspections by Inspection Type – 2010-11 to 2018-19
Fiscal year Total ASP MC
Re-insp. Initial insp.Footnote * Other ASP insp. Part of Follow-up Part of Product audit Unbiased MMP insp. Selective MMP insp. Insp. following a complaint Other MC insp.

2010-11

57,241

6,264

25,489

184

87

295

16,901

3,609

1,940

2,472

2011-12

53,877

7,529

25,371

162

181

258

14,248

2,579

1,907

1,642

2012-13

72,442

19,980

32,849

275

311

374

14,993

1,103

1,112

1,445

2013-14

67,690

22,803

26,000

315

266

247

14,798

1,344

803

1,114

2014-15

104,878

62,229

30,596

455

360

340

8,794

619

873

612

2015-16

160,225

117,348

30,205

985

644

669

8,084

1,273

485

532

2016-17

159,197

117,444

24,648

1,050

623

519

9,201

4,623

676

413

2017-18

154,274

114,096

26,195

1,067

972

1,131

7,180

2,622

625

386

2018-19

148,310

110,308

26,425

1,068

1,686

1,658

5,299

902

502

462

% var. since 2013-14

119%

384%

2%

239%

534%

571%

-64%

-33%

-37%

-59%

Inspections across Canada

Many KIs noted that as FAPA increased the number of devices to be inspected across the Canadian landscape, it created opportunities for business and employment for ASPs.

MC management and regional staff noted that mandatory re-inspections were phased-in based on the location of the devices, (i.e., postal code) and trade sectors, to facilitate the inspection work for ASPs and to ensure that not all inspections were due for re-inspection at the same time. The data analysis corroborates that mandatory re-inspections were phased-in equally across sectors and provinces.Footnote 7 Many KIs also indicated that due to the increase in re-inspections in the eight trade sectors, a large number of device measurement and non-measurement errors are being detected and corrected in those sectors by the RTs through mandatory re-inspections.

Prior to the introduction of FAPA, these devices were not required to be re-inspected once in the marketplace and could have been inaccurate for many years. In general, the majority of KIs thought that the frequencies of mandatory inspections was appropriate. However, some KIs noted that the five-year interval in the retail food sectors might be too long. If a revision of the frequencies of mandatory inspections in the eight trade sectors was considered by MC, the vast majority of KIs suggested that the revision should be based on compliance rate (e.g., increasing frequencies for sectors with lower compliance rates). Some KIs suggested that volume, usage, environment, and type of device (e.g., heavy duty scale) could also be considered in determining the frequency of mandatory re-inspections.

Other trade sectors

MC is also responsible for ensuring the integrity and accuracy of trade measurement in 33 other sectors in Canada such as food and beverage manufacturing, transportation, chemical products, and cannabis products, which are monitored by MC through MMP inspections.Footnote 8

Some of these sectors were examined as part of a Trade Sectors Review conducted by MC between 2001 and 2011.Footnote 9 It should be noted that, during that review, the majority of stakeholders in the chemical products, transportation, and waste management (including scrap metal) sectors had requested mandatory periodic inspections in their respective sectors as measurement errors over a long period of time could be costly and could damage the reputation of companies as well as the sector. In addition, inspections are more likely to detect measurement problems and incorrect device settings, installations or usage.

Similarly, the vast majority of KIs interviewed for the evaluation indicated that FAPA should cover (or should have covered) more trade sectors (if not all, as is the case under the EGIA). Trade sectors that were specifically identified by KIs to be subject to mandatory re-inspections include waste management, scrap metal, transportation/shipping, chemicals, fertilizers and water. Some MC KIs suggested that if additional sectors were added under FAPA, a risk-based assessment would be required to identify the sectors and the frequency of inspection.

The data analysis reveals that the compliance rate in some of these sectors is low. For instance, MC inspectors found through unbiased MMP inspections that the "as-found" measurement compliance rate in some of the sectors not covered by FAPA was as low as 53.5% for quarries and sandpits, 60.5% for scrap metal, and 64.0% for waste management in 2018-19.  In comparison, the average "as-found" measurement compliance rate in the eight trade sectors covered by FAPA was around 94.3%. Although conditions in the three sectors noted above can be more challenging with respect to maintaining device accuracy (i.e., environment and weather conditions), the compliance rate in the mining sector (which is under FAPA and has  similar challenging conditions) is significantly higher since FAPA was  implemented (see table 3).

Figure 4 – Measurement Compliance Rate in Unbiased Inspections Conducted by MC in Sectors not covered by FAPA for 2018-19

Source: ORA, computed by AEB.

Text version

Figure 4 compares the measurement compliance rate in unbiased inspections conducted by MC in sectors not covered by FAPA for 2018-19 and the FAPA compliance rate of 94.3%.

Sector not covered by FAPA for 2018-19 Compliance rate for unbiased inspections conducted by MC

Chemical

79.2%

Food&Bev

91.1%

Fruit&Veg

79.3%

Gen Merchand

87.0%

Hardware Stores

88.6%

Livestock

87.5%

Preciousmetal

76.5%

Qua&Sandpit

53.5%

Scrapmetal

60.5%

Transportation

95.9%

Waste

64.0%

Other

71.4%

One of the challenges to introduce similar oversight in sectors not covered by FAPA identified by a few KIs for this evaluation is whether the ASPs have the capacity to handle additional sectors (i.e., whether there are enough ASPs/RTs to keep up with all the devices). It should be noted that several U.S. states, Japan, Germany, France, Switzerland and many other European countries require that devices be periodically retested across all sectors to ensure continued proper function, while United Kingdom, Australia, New Zealand, and a few other countries carry out inspection based on risk assessments.Footnote 10

Recommendation: MC should explore risk-based approaches, including periodic re-inspections, to improve accuracy and fairness in trade sectors not currently overseen by FAPA.

3.1.2 To what extent has FAPA contributed to enhancing MC's enforcement strategy?

Key Finding: FAPA contributed to enhancing MC's enforcement capability. Overall, education and information provided to device owners, as well as warnings provided to them for offenses, have been effective compliance measures. Although AMPs have been used to a limited extent to date, they are generally viewed as having a deterrent effect.

The 2018 OECD report on Regulatory Enforcement and Inspections Toolkit recognizes that "in order for the enforcement response to be credible, and to achieve some deterrence effect, the potential sanctions must be sufficiently strong to outweigh the potential benefits from violations and sufficiently flexible"; and "inspectors should ensure that follow up inspections are carried out to ensure full compliance". Another OECD report (2014) that looked at regulatory enforcement indicates "there should also be requirements for enforcement structures to be accountable for their decisions (public guidelines for inspectors on decision making, annual reporting on enforcement actions, including justification)".Footnote 11  In order to promote trade measurement compliance in the marketplace, MC uses a legislated escalation approach to enforcement. This approach consists of using a number of enforcement tools:

  • stakeholder education (e.g., face to face meetings, information bulletins);
  • warning and information letters;
  • notices of infraction and non-compliance;
  • seizure;
  • violation warnings;
  • notices of violation (AMPs); and
  • prosecution (court-imposed fines).

Since the MC audit conducted in 2017, which recommended that "MC should finalize its enforcement framework to reflect the new measures enabled by FAPA, and develop and deliver related training", MC's enforcement framework has been updated, and MC has set up and updated enforcement policies, processes, and escalation tables (i.e., guidelines for escalating enforcement actions) to support regional staff to put forward an enforcement action. In 2017-18, enforcement training was created by MC and training courses were delivered nationally. Several KIs of the evaluation (especially among regional staff) indicated that clarification and improvements have been introduced to the enforcement policy and procedures since its implementation.

MC selects the appropriate tool based on the facts and circumstances of the case. Stakeholder education initiatives that foster cooperation are the first tools used to promote compliance. More stringent enforcement actions are taken only when co-operative actions have failed, and only to the degree necessary to gain compliance. For instance, AMPs or prosecution could be considered if the non-compliance is part of a pattern of behaviour by a trader or part of a larger set of instances of non-compliance. Notices of violation (AMPs) are used as an alternative to prosecution for serious offences or repeat offences when less severe enforcement actions have not corrected the problem. Based on the WMA Enforcement escalation table developed by MC, MC may use up to four other tools before delivering the AMPs.

Enforcement capability

Evidence from the document review, data review and interviews suggest that FAPA has positively contributed to MC's enforcement capability. MC's escalation strategy (education, information letter, warning, notice of violation, AMPs) was perceived by a majority of KIs as appropriate and that has improved compliance. For example, a few KIs indicated that education of device owners provided by RTs during their inspection, as well as information letters and warnings sent by MC, have been effective measures to improve compliance. Nonetheless, some MC KIs noted that MC needs to strengthen the strategy, including incorporating guidelines to handle non-payment of AMPs to ensure their deterrence effect, and streamline the procedure to issue AMPs as it was seen to be too lengthy (e.g., up to four tools used prior to issuing an AMP).

Enforcement activities gradually phased-in

A review of the enforcement data found that MC has gradually phased-in enforcement activities under WMA. As table 2 illustrates, MC provided various types of enforcement actions.

Very few AMPs have been issued to establishments under the WMA since FAPA came into force.  It should be noted that notices of non-compliance and seizures were already in place before the introduction FAPA. On the other hand, along with AMPs, violation warning letters and notice of violations were also newly introduced as one of the tools that could lead to an AMP. In total, an average of 3,000 enforcement actions per year were taken between 2014-15 and 2016-17, compared to a total of 2,675 actionsFootnote 12 in 2017-18 and 5,187 actions in 2018-19. Most of the actions under the WMA were notices of non-compliance (38.3%) and violation warning letters (33.8%) in 2018-19.

Table 2 – Number of WMA Enforcement Actions – 2014-15 to 2018-19
Fiscal year 2014-2015 2015-2016 2016-2017 2017-2018 2018-2019

Notice of non-compliance

2,454

2,016

1,773

1,571

1,987

Seizure

511

465

459

342

454

Stakeholder education

100

186

881

449

553

Information and warning letters

68

179

154

249

344

Violation warning letter

Not applicable

Not applicable

1

54

1,753

Notice of infraction

Not applicable

Not applicable

0

8

93

Notice of violation (AMPs)

Not applicable

Not applicable

0

1

3

Prosecution

Not applicable

Not applicable

0

1

0

Total

3,065

2,846

3,268

2,675

5,187

Source: Measurement Canada

In comparison, a total of 147 enforcement actions were taken by MC in 2018-19 under the EGIA. The most commonly used enforcement action under the EGIA was contractor education (approximately 130 actions per year in 2017-18 and 2018-19). Only one AMP was issued and one contractor was prosecuted in 2018-19 under the EGIA.  It should be noted that MC does not have an online central repository for enforcement activities taken by MC under the EGIA. A few KIs suggested that a similar system to ORA should be used for the activities under the EGIA. From the point of view of several KIs, ORA was a positive implementation outcome of FAPA as a central database was created, which resulted in less paperwork for RTs and MC inspectors.

3.1.3 To what extent have retailers and other device owners maintained accurate measuring devices as a result of FAPA?

Key Finding: Although there is evidence that FAPA has improved device accuracy, MC should continue monitoring progress to further assess effectiveness over time.

Mandatory re-inspections

Evidence from the survey showed that 83% of MC inspectors and 91% of RTs agree that mandatory re-inspections have contributed to improved device accuracy.  KI opinions were more divided. Some agreed that mandatory re-inspections increased the accuracy of measuring devices, others did not. A few of the latter group noted that although mandatory re-inspections themselves did not increase device accuracy, traders are more aware of their responsibilities and obligations due to the inspection recurrence.

Enforcement activities

Several KIs noted that a risk of receiving a notice of non-compliance, financial penalties, or being prosecuted encourages a greater level of compliance. However, in the survey, less than half of MC inspectors (41%) and RTs (47%) agreed that FAPA enforcement activities contribute to device accuracy. As a result, it appears that the threat of enforcements, as much as the enforcements themselves, contribute to some extent to compliance with the regulations.

Compliance rate

The data analysis revealed that the "as-found" measurement compliance rate during a subsequent re-inspection conducted by ASPs among the eight trade sectors was around 96.2% in 2013 and it then declined to 92.4% during the implementation period (see Table 3). This decline was largely explained by the fact that the vast majority of devices in the eight trade sectors had never been previously re-inspected. Specifically, the number of devices re-inspected by RTs increased drastically, from an average of 14,144 re-inspections pre-FAPA to 99,007 during the phase-in period, and a larger number of device errors were detected (and corrected) by the RTs through re-inspections.

Since full implementation of FAPA, the measurement compliance rate for devices re-inspected has increased to 94.3%. It should be noted that the average compliance rate is largely driven by the compliance rate in the retail petroleum sector, due to the large number of devices inspected in that sector (about 83% of devices covered by FAPA).  As table 3 also shows, the average compliance rate varies per sector. For instance, after the implementation period, the compliance rate was lower for devices used in mining (83.9%) and forestry (82.6%) than for those used in retail food (88.9%) or retail petroleum (95.9%). Nonetheless, one in ten retail food scales and about one in 25 fuel pumps were still not accurate when inspected in 2018-19. The slight decline of compliance rates in some trade sectors after the full implementation of FAPA compared to those before FAPA is largely due to the increased number of devices re-inspected by RTs in the later period as noted previously.  

The data analysis also reveals that the average measurement compliance rate during re-inspections conducted by RTs was lower in rural than in urban areas, for scales (gravimetric devices) than for meters (volumetric devices). Moreover, a statistical analysis conducted for the evaluation corroborated that FAPA increased the "as-found" measurement compliance rate after the implementation period, even when taking into account other factors such as trade sector, device type and location.

Table 3 – "As-found" Measurement Compliance Rates in Re-inspections
Measurement compliance rates in re-inspections Avg. # of re-inspections per fiscal year* Total Trade Sectors
Grain and crops Downs. Petrol. Forestry Mining Fishing Dairy Retail food Retail petrol.

Before FAPA

14,144

96.2%

74.4%

85.8%

77.7%

62.7%

94.4%

97.4%

81.6%

97.2%

FAPA Phase-in

99,007

92.4%

76.8%

80.8%

81.5%

71.6%

75.3%

92.2%

85.8%

95.9%

Full implementation

112,202

94.3%

79.4%

80.9%

82.6%

83.9%

85.1%

86.9%

88.9%

95.9%

Avg. # of re-inspections during full implementation

--

112,202

2,732

3,260

690

176

3,750

255

7,896

93,423

Source: ORA, computed by AEB. *For the purpose of the average number of re-inspections per fiscal year, before FAPA period refers to 2010-11 to 2013-14, FAPA phase-in period refers to 2014-15 to 2016-17, and full implementation period refers to 2017-18 and 2018-19. Of the 112,202 re-inspections, 20 are coded as generic mass (used in exceptional circumstances where it is not possible to determine the specific trade sector).

MC measurement compliance rates during marketplace monitoring 

As noted in Figure 2, MC inspectors conduct random inspections on devices every year to evaluate the overall status of accuracy and detect problems. Optimally, the compliance rate of these random inspections should be similar to those reported by RTs during the mandatary re-inspections. However, the data analysis showed that the measurement compliance rate on devices in the eight trade sectors reported by MC inspectors through the unbiased MMP was on average 10 percentage points lower than those reported by ASPs during the phase-in period. Since the phase-in period has been completed, that difference has declined to 6.6 percentage points, which might imply improvements in terms of the accuracy of the results reported by RTs. It might also underline the importance of maintaining an independent presence of MC in the marketplace to monitor the work of RTs.

Non-compliant devices

For non-compliant devices, the data analysis found that over the most recent years (2017-18 and 2018-19), a greater proportion of rejected devices (non-compliant devices due to error against consumers) were subsequently re-inspected within 14 days (standard time allowed to device owner to repair the device) compared to years during the phase-in period, indicating that more re-inspections are done in a timely basis.

Overall, the compliance rates in the eight trade sectors covered by FAPA have improved in recent years. The compliance rates should continue to be monitored as most devices in some trade sectors (e.g., retail food) have only been through one cycle of mandatory re-inspection since 2014.  

3.1.4 To what extent do retailers and other device owners have access to a national network of authorized service providers as a result of FAPA?

Key Finding: A large national network of ASPs and RTs is available for device owners, but with a limited presence and at a higher cost in northern and remote areas.

ASP and RT Network in Canada in WMA

As of March 31, 2019, MC reported 64 active accredited organizations, 114 active registered organizations and 679 recognized technicians across Canada. The administrative data review revealed that in 2018-19, a total of 657 RTs (182 ASPs) conducted an inspection compared to 298 RTs (97 ASPs) in 2010-11. In comparison, only 87 MC inspectors conducted an inspection in 2018-19, 10% less than in 2010-11. As a result, RTs conducted about 90% of the inspections in 2018-19, while this proportion was about 55% in 2010-11 due to the implementation of FAPA.

Looking at the demographics of MC inspectors and RTs, the survey revealed that 42% of RTs were aged 50 and above. Women represented about 1% of the RTs compare to 12% of the MC inspectors. While 7% of RTs identified themselves as a visible minority, this proportion was 27% among MC inspectors, which could reflect the governmental policy in place to build a diverse and inclusive Public Service. Based on the survey results, further efforts could be made to recruit more women and other underrepresented groups in MC offices located in Atlantic provinces. The survey also revealed that 89% of RTs intended to continue to work as an RT for at least the next three years.

The majority of KIs noted that there is a good selection of ASPs and RTs available across Canada. For basic devices, there are many options. Similarly, in the survey, most of the MC inspectors and RTs typically agree (61%) that device owners have a large choice of ASPs for device inspections. Agreement was higher among RTs inspecting scales (68%) compared with only 50% among those inspecting pumps/meters (volumetric devices).

Figure 5 – Demographics of MC Inspectors and RTs

Source: Survey of MC inspectors and RTs.

Text version

Figure 5 presents the demographics of MC inspectors and RTs who participated in the surveys conducted as part of the evaluation. 

  • 1% of RTs were women.
  • 12% of the MC inspectors were woman.
  • 42% of RTs indicated they were aged 50 or above.
  • 28% of MC inspectors indicated they were aged 50 or above.
  • 27% of MC inspectors identified as a visible minority.
  • 7% of RTs identified as a visible minority.
  • Almost 2% of MC inspectors and RTs identified as an Indigenous person or a person with a disability.
  • 89% of RTs intend to continue to work as an RT for at least the next three years.
Limited availability in remote and rural areas

The survey results also showed that 40% of RTs working in rural areas disagree that device owners have a large choice of ASPs. In addition, many KIs noted that there are still very few ASPs available in remote areas and up north and for specialized devices (e.g., bulk scales in mining industry). For instance, the ASP network is very limited or absent in Cape Breton Island and Labrador for scales. Two maps (one for scales and one for meters) in Appendix A clearly show that there are only a few RT points of service located in northern and remote areas of Canada.

Cost in remote areas

When FAPA was announced, the government committed that "Fees for independent inspection services would be determined by market forces, ensuring that there is competition in the marketplace and that retailers would be charged fairly for these services." Nonetheless, some interviews from regional management staff noted that ASPs exist to make a profit and suggested that MC should be more aware of ASPs range of fees to ensure that there is enough competition in the marketplace so that devices owners pay fair prices.

Most KIs indicated that device owners in northern and remote areas are disadvantaged as inspections are much more expensive than in urban areas, mainly due to the travel cost incurred by RTs to travel to northern or remote areas. Some noted that the inspections fees can be sometimes up to four times greater than the margin of profit obtained from a device, especially for devices used in "Mom and Pop shops" (e.g., scale for food), seasonal devices, marinas (e.g., fuel pump) in remote areas, or those that are related with a secondary activity of a business. As a result, some device owners could question the viability of maintaining their devices in operation.

To alleviate this issue, it was noted that MC had proposed in some instances to device owners in remote areas to coordinate their inspections at the same time to split the travel cost. Although it reduced the cost for device owners, this was only done on a few occasions rather than systematically implemented. In another example, a KI noted that instead of a device owner paying for the travel cost, the Nunavut government actually hired ASPs to have inspections conducted on fuel dispensers and for most of the scales, they were shipped to ASPs for inspection.

To further seek efficiencies and help relieve the costs for device owners in remote areas, some KIs suggested that MC needs to attract more ASPs in the north and remote areas and that MC could consider extending the re-inspection period in remote areas or offer exemptions of mandatory re-inspections for small business. It was also suggested that MC could monitor the range of fees in remote areas to ensure there is no price fixing.

Recommendation: MC should explore alternative approaches to enable efficiencies for re-inspections in northern and remote areas and access to ASPs (e.g., greater coordination of re-inspections).

3.1.5 To what extent has FAPA contributed to ASPs' competencies to conduct mandatory inspections?

ASPs Competencies

Key Finding: With RTs exposure to the range of measurement devices under FAPA and supported by MC's monitoring of their work, they have developed the competencies required to conduct timely and quality inspections.

Mandatory training of MC's technical inspection staff is part of the Technical Inspector Recruitment and Development Program. Furthermore, MC has developed a formal training approach for ASPs who have been designated with authority to perform inspections. MC provides training to ASPs and their staff to ensure they understand and can verify legal requirements. MC recognizes, trains and evaluates RTs for their competencies to inspect weights and measures devices and issue certi­ficates.

Although FAPA did not have a direct effect on ASPs competencies (e.g., training program did not change with FAPA), the majority of KIs indicated that FAPA has contributed to ASPs competencies since RTs are conducting the vast majority of the inspections as shown in table 1 (about 137,000 inspections of the 148,300 inspected conducted in 2018-19). This exposure to a variety of devices contributes to developing their competencies.

MC Monitoring

To ensure compliance, ASPs that have received accreditation status are subject to MC surveillance and product audits while those that are registered are subject to enhanced MC monitoring. According to the survey, 62% of MC inspectors agreed that RTs conduct timely inspections and 73% said the RTs they have monitored conducted quality inspections.

The 2018-19 Management Review Report showed that 105 surveillance and 109 product audits were conducted in   2018-19, and 105 annual visits were held with registered organizations. The report also indicated that 93% of accredited ASPs were: 1) following procedures, 2) equipment was properly maintained and used, and 3) reports/certificates were accurate and submitted on time.

Similarly, that proportion was 97% among registered ASPs. Moreover, the number of WMA devices that were part of product audits have increased by more than five times since 2013-14 (571% - see table 1). It was noted by some KIs that RTs are competent in their technical abilities, perhaps to a lesser extent in terms of the regulations as it is not part of their regular duties outside their "inspector" role.

Similarly, most of KIs and 83% of the RTs surveyed indicated that the product audit and follow-up inspections were appropriate. Nonetheless, some KIs noted that there are many rules that govern the auditing and monitoring program. A few (mainly ASPs and some of the MC regional staff) noted that the auditing program is extensive and too frequent. Similarly, the survey found that 28% of RTs indicated that audits and follow-up inspections are too frequent. In addition, several KIs (mostly MC management and regional staff) suggested a risk-based approach would be more appropriate than annual auditing, i.e., conducting less frequent audits on ASPs that meet MC requirements and follow the procedures.

MC inspectors Competencies

Key Finding: FAPA may have inadvertently limited MC inspectors' exposure to specialized devices, and thereby the knowledge needed to inspect them.

Based on the 2018-19 Management Review Report, regional MC teams received training and were able to complete their required training progression within the anticipated timeframe. MC employees and their managers felt that the training progression has adequately prepared them to perform their ASD functions competently. However, it was noted by some KIs (mainly ASPs in WMA and a few regional MC staff) that the decline in MC inspection activities (from 25,304 inspections in 2010-11 to 10,509 inspections in 2018-19) due to the new role of MC inspectors following the introduction of FAPA (i.e., monitoring ASPs) and staff turnover, has impacted MC exposure to specialized devices. Some KIs noted that the previous cohort of inspectors "knew/know their devices", which is less the case with new hired inspectors. In some instances, it was noted that RTs have more expertise on certain devices than some MC inspectors, affecting their ability to properly assess the RTs. The survey results seem to corroborate this finding by showing that 23% of the MC inspectors had been in the role for less than 3 years. This decline in expertise by MC inspectors could continue to be an issue for MC in the near future as close to 30% of MC inspectors are older than 50 years of age.

ASPs Training

Key Finding: Although training is adequate for RTs to perform inspections, there is a need for additional learning tools and refresher courses to maintain knowledge of regulations and procedures.

Training offered

The survey revealed that a large majority of RTs (75%) completed their most recent theoretical training through MC's fee-based training, whereas 9% RTs have received their training from their own organization (trained by an RT). The document review revealed that RTs who received training from MC in 2017-18 had an overall greater pass rate on the exams (80%) than those who received training from their own organization (69%). Nonetheless, 84% of RTs reported that their training provided them with the competencies they need to conduct quality inspections, regardless of who provided the training. MC inspectors also agreed with that statement, but to a lesser extent (67% agreed).

Similarly, most KIs noted that MC's training program is effective. Some KIs noted that the theoretical training was overly dense and the availability of training courses was limited. This was also reflected in the survey where 76% of RTs felt their training was appropriate and only 17% felt that their training was burdensome. Nonetheless, just over 50% of RTs agreed that they have had opportunities to enhance their competencies by taking training in recent years.

Additional training needs

The survey also showed that 48% of RTs agreed that MC should provide additional courses. Almost all ASPs and a few regional staff mentioned that a refresher course on regulations and procedures would be beneficial. Some suggested more training be delivered by either video conferences, or a mix of video conference and online courses. To note, a number of training videos are available to ASPs and their RTs, as well as to MC inspectors on the MC website, but through a formal request to MC.

Although a few KIs indicated that MC has increased training on ORA, several KIs reported that MC should continue monitoring data entry in ORA in order to increase the integrity level of the data. It was also suggested by a few KIs that additional training on data entry in ORA should be offered, especially since ORA has changed from when it started.

Recommendation: MC should facilitate access to online refresher courses and learning tools for RTs and consider implementing strategies to facilitate knowledge transfer for MC inspectors.

3.2 Efficiency

3.2.1 To what extent is FAPA being implemented efficiently?

Key Finding: MC utilized financial resources provided for the implementation of FAPA efficiently, including absorbing some of these costs.

Implementation Cost

The Legislation Review of FAPA in 2016 noted that "the use of non-government inspectors to perform mandatory inspections allows the government to increase the detection and correction of inaccurate measurement in the marketplace without significantly increasing resources dedicated to measuring device inspections".

While the total number of MC inspections and related costs declined as a result of FAPA, the total number of hours spent by MC employees on accreditation and registration of ASPs (and monitoring performance) increased by almost 50% since 2011-12, reaching 36,448 hours in 2018-19 (see table 4) and requiring MC to commit more resources to support these activities.

A total of $8 million was committed by the government for the implementation of FAPA over 5 years (2010-11 to 2014-15), and $2 million in annual ongoing funding. MC used this funding to increase resources for MC inspectors and recruit, train and monitor RTs.  As a result, the overall operating budget of MC increased from $26.3 million in 2010-11 to $27.6 million in 2011-12.Footnote 13

However, financial data shows the total operating budget of MC declined from $27.6 million in 2011-12 to $25.7 million in 2018-19. When taking inflation into account, the operating budget declined almost every year since 2011-12, reaching $23 million in 2018-19 (adjusted for inflation). As a result, the operating budget declined by 16.7% between 2011-12 and 2018-19 when adjusted for inflation. Knowing that ORA was also implemented during that period, it can be concluded that MC has implemented FAPA with limited financial resources and has partially absorbed the costs associated with the introduction of FAPA.

Table 4 – Operating Budget for the MC ($ millions), 2009-10 to 2018-19
Fiscal Year Total number of MC Inspections under WMA MC number of hours spent on accreditation and registration MC Salary Expenditures ($) MC Non-Salary Expenditures and Capital ($) Total (in current $) Total (in constant $2011-12)

2009-10

32,816

30,338

20.9

6.1

27.0

28.2

2010-11

29,352

30,992

20.3

6.0

26.3

27.1

2011-12

24,597

24,440

19.7

7.9

27.6

27.6

2012-13

24,041

28,798

20.7

6.5

27.1

26.7

2013-14

23,810

33,807

20.5

6.7

27.1

26.5

2014-15

19,041

28,024

21.1

6.0

27.1

25.9

2015-16

19,229

33,791

20.3

5.7

26.0

24.6

2016-17

22,988

29,106

21.2

5.9

27.1

25.3

2017-18

18,752

34,133

20.8

5.1

25.9

23.7

2018-19

16,139

36,448

20.1

5.6

25.7

23.0

Shaded cells represent the phase-in period from fiscal year 2014-15 to 2016-17 and the full implementation period from fiscal year 2017-18 to 2018-19. Source: Management Review Reports and MC 10-year Budget Analysis.

Communication and Delivery

Key Finding: There is a lack of awareness among Canadians and device owners of MC's mandate and role, as well as the implications and results of FAPA.

Many KIs indicated that when FAPA was introduced, it created confusion and misinterpretation across the country. There was a lack of education, communication and outreach activities on the changes coming with FAPA, especially among small and medium enterprises (SMEs), and "Mom and Pop shops". MC mainly relied on a few major associations and its website to communicate the changes.  Some regional staff noted that it was challenging for them to learn about FAPA's requirements and the new enforcement policy, as well as to make sure everything was implemented properly.

According to the audit recommendation, MC should have developed and implemented a national communication strategy for its new enforcement measures and mandatory inspections, targeting both internal and external stakeholders. Since then, MC has strengthened to some extent its external and internal communication with stakeholders (mainly ASPs) by publishing more information on MC's website and sharing communications tools with MC employees.

Gaps remains in education and awareness

Based on the Management Review Reports, more than 97% of the ASPs were satisfied with the program's delivery by MC since 2015-16. To a lesser extent, the survey reveals that the majority of RTs (70%) indicated satisfaction with their relationship with MC, 18% were neither satisfied nor unsatisfied and 12% were unsatisfied. The most common reason for dissatisfaction was communication (e.g., difficulty reaching MC), a view also expressed by several KIs. There are still some device owners that are not aware (or do not understand) that their devices are required to be regularly re-inspected. In addition, the document review revealed that while some outreach activities have taken place, these activities with MC's external stakeholders (e.g., consumers and device owners) were limited.

Several Canadian agencies and agencies in other countries make information publically available on their activities, including enforcement actions taken for non-compliance (e.g., website, annual report). For example, the Canadian Food Inspection Agency (CFIA) and Transport Canada publish enforcement data on their websites, the Financial Transactions and Reports Analysis Centre of Canada (FINTRAC) publishes information in their annual report and the agency responsible for trade measurement in Australia publishes data on their enforcement actions.

MC should make available reliable data on its activities (e.g., inspections, enforcement actions, audits, complaints) and results (e.g., compliance rate) to Canadians to align with the government's open data initiative. This would support consumer confidence in the accuracy of measurement-based transactions. During the evaluation process, there was no reliable available data on MC activities and results to Canadians.

Recommendation: MC should improve communication with device owners and Canadians, including easily accessible information on activities (e.g., enforcement actions) and results (e.g., compliance rates).

4.0 Conclusions and Recommendations

4.1 Conclusions

Performance

  • FAPA increased the number of mandatory re-inspections of devices in the eight regulated trade sectors and contributed to enhancing MC's enforcement capability. As a result of the mandatory re-inspections and MC's enforcement capability, FAPA has increased coverage in the market place and device accuracy. Although there is evidence that FAPA has improved device accuracy, MC should continue monitoring progress to further assess effectiveness over time.
  • However, more than 30 other trade sectors under the WMA are not subject to the same level of oversight. The measurement compliance rate in some of these sectors was as low as 53.5% for quarries and sandpits, 60.5% for scrap metal, and 64.0% for waste management in 2018-19 (recognizing these can naturally be more challenging environments to maintain device accuracy), compared to an average of 94.3% in the 8 trade sectors covered by FAPA.
  • Since FAPA came into effect, retailers and device owners have access to a large national network of ASPs and RTs, but with a limited presence and at a higher cost in northern and remote areas.
  • Although RTs are qualified and competent to perform inspections and their training is adequate, there is a need for additional learning tools and refresher courses to enable them to maintain their knowledge of regulations and procedures. In addition, FAPA may have inadvertently limited MC inspectors' exposure to specialized devices, and thereby the knowledge needed to inspect them, due to the new role of MC inspectors (i.e., monitoring ASPs).

Efficiency

  • MC utilized financial resources provided for the implementation of FAPA efficiently, including absorbing some of these costs.
  • In general, there is a lack of awareness among Canadians and device owners of MC's mandate and role, as well as the implications and results of FAPA.

4.2 Recommendations

The evaluation findings led to the recommendations noted below.

Recommendation 1: Accuracy and fairness in the marketplace

MC should explore risk-based approaches, including periodic re-inspections, to improve accuracy and fairness in trade sectors not currently overseen by FAPA.

Recommendation 2: ASPs and RTs availability in northern and remote areas

MC should explore alternative approaches to enable efficiencies for re-inspections in northern and remote areas and enable access to ASPs (e.g., greater coordination of re-inspections).

Recommendation 3: Training and Knowledge transfer

MC should facilitate the access to online refresher courses and learning tools for RTs and consider implementing strategies to facilitate knowledge transfer for MC inspectors.

Recommendation 4: Communication with device owners and Canadians

MC should improve communication with device owners and Canadians, including easily accessible information on activities (e.g., enforcement actions) and results (e.g., compliance rates).

Appendix A

Distribution of RT's points of service for scales and meters across Canada in 2018-19
Province or Territory Meters Scales

Alberta

57

43

British Columbia

30

39

Manitoba

16

22

New Brunswick

14

11

Newfoundland and Labrador

13

10

Nova Scotia

14

20

Ontario

76

103

Prince Edward Island

2

1

Quebec

92

78

Saskatchewan

16

17

Yukon

1

Not available

Northwest Territories

Not available

Not available

Nunavut

Not available

Not available