Evaluation of the Contributions Program for Non-Profit Consumer and Voluntary Organizations

Audit and Evaluation Branch
February 2005

Table of Contents

• 1. Introduction
  • A. Background
  • B. Study Approach
  • C. Study Limitations
  • D. Structure of the Report
• 2. Executive Summary and Recommendations
  • A. Executive Summary
  • B. Recommendations
  • C. Other Items of Note
• 3. Relevance
  • A. Introduction
  • B. Objective-Based Rationale
  • C. Continued Need
  • D. Possible Duplication
  • E. Encouraging Complementarity and Avoiding Gaps
  • F. Summary
• 4. Success Related to Support
  • A. Introduction
  • B. Success related to Research
  • C. Success related to Development
  • D. Success related to Consumer Movement
• 5. Alternatives
  • A. Introduction
  • B. Recent Changes
  • C. Suggested Improvements
  • D. Summary of Alternatives
• Management Response
  • Background:
  • 1. Find a way to increase the available funds
  • 2. Improve Dissemination of Information and Institute Tracking of the Impacts of Information
  • 3. Institute General-Application Development Projects and Communicate their Findings Broadly
  • 4. Increase Transparency of Application Approval Process
  • 5. Encourage Greater Self-Reliance through Development Projects

1. Introduction

A. Background

This report presents findings of the Evaluation of the Contributions Program for Non-Profit Consumer and Voluntary Organizations. A prior evaluation (completed in March 2001) resulted in refocusing of the Program. Prior to any further revision to the Program's Terms and Conditions, Treasury Board (TB) required an evaluation as part of a scheduled review. The evaluation described in these chapters is in response to and fully conforms to this TB requirement.

The Contributions Program for Non-Profit Consumer and Voluntary Organizations provides contributions to eligible groups to strengthen the consumer's role in the marketplace though the promotion of timely and sound research and analysis and to encourage the financial self-sufficiency of consumer and voluntary organizations. The Office of Consumer Affairs (OCA) of Industry Canada administers the Contributions Program.

Eligible groups (non-profit organizations working in the consumer interest) submit applications for support to the Program. Research Project Contributions provide support to particular projects involving consumer issues research or policy development. Development Project Contributions support activities related to the capacity development or self-sufficiency of organizations.

The total amount of funds available through the Program was increased to its current level of $1.69 million dollars in 2001/02. Up to $300,000 is available annually for Development Projects with the balance ear-marked for Research Projects. Individual projects are limited to $100,000 in support and no applicant may receive more than $500,000 in a single year.

The recent funding history under the Contributions Program is provided in Exhibit I-1. Since expansion in 2001/02 the Program has annually funded approximately 40 projects involving about one dozen organizations. Over the last four years the Program has been unable to support 61% of the applications received and 65% of funds requested.

Contributions to organizations support activities that are expected to lead to a more effective consumer voice and thereby encourage a fair, efficient and competitive marketplace (a strategic objective for Industry Canada).

B. Study Approach

Evaluation issues of the study can be grouped thematically (A full list of evaluation issues and questions may be found in Appendix C.) reflecting the Program's:

• Relevance (rationale, need, and avoidance of duplication).
• Success overall and for separate research and development components.
• Alternatives considering cost-effectiveness, lessons learned and suggestions for change.

These are explored through a multiple lines of evidence approach combining a documents and literature review (see Appendix D) and key informant interviews (see Appendices E and F) with Program staff (4), current Program recipients (7), past Program recipients (3), non-recipients (2) stakeholder and industry sector representatives (11), and academics (2). Case studies (see Appendix B) involved a sub-set of the previous data sources specific to three areas of support (automobile retailing, privacy, and weights and measures).

C. Study Limitations

The evaluation relies heavily on the views of a small number of the key informants. These individuals represent a significant proportion of those who are knowledgeable of the Program. A limited review of secondary sources of information was also conducted including the earlier more extensive evaluation. Although these sources are limited, we believe they provide sufficient evaluation coverage given the size of the Program and scope of the previous evaluation.

D. Structure of the Report

The evaluation relies heavily on the views of a small number of the key informants. These individuals represent a significant proportion of those who are knowledgeable of the Program. A limited review of secondary sources of information was also conducted including the earlier more extensive evaluation. Although these sources are limited, we believe they provide sufficient evaluation coverage given the size of the Program and scope of the previous evaluation.

2. Executive Summary and Recommendations

A. Executive Summary

1. Relevance/Need

The Contributions Program is intended to create a more effective consumer voice and thereby encourage a fair, efficient and competitive marketplace. All data sources confirm that the objectives of the Program remain relevant.

All data sources also confirm that there is a continued need for the Program. Evidence suggests that the need is expanding and there is some evidence that the Program is not keeping pace with this need.

There are few if any Federal programs that overlap with the Contributions Program. A new program by the Privacy Commissioner will require future co-ordination efforts to ensure support for privacy-related research is not duplicated. Some similar research projects might be supported by Federal departments or through Provinces and Territories but these are infrequent, specific to a research topic, and not part of a general program of support.

2. Success

Recipient organizations increase their capacity to do research and to represent the consumer interest. Many are developing expertise in particular consumer issue areas. Their credibility is also enhanced. However, few recipient organizations appear to have fully capitalized on these impacts through other funded activities. Government organizations that seek the input of consumer groups are not generally willing to pay for it. Although demand for their input on consumer issues is high, consumer groups are able to meet very little of this unfunded demand. To do so would take away from other activities of the organization.

Recipient organizations report an improvement in the human capital of their employees and co-op and summer students through activities supported by research projects. The research exposure in the supported organizations acts as a rich training environment and some employees have left for other research opportunities, many involving the consumer interest, elsewhere.

The degree of dissemination of information from Research Project Contributions varies. Most organizations identify posting a summary of the report on their web site and providing the capability to download the report from their site. Most also report holding a press conference or providing a media release. Many develop and send out reports to a list of key stakeholders (including industry) they think should get the information. A few identify making the report available to their membership. A few also mention sending the report to politicians or meeting with politicians to review the study findings.

Organizations do not record contact information for those who obtain their reports, frustrating attempts to follow-up to assess an impact. Neither the OCA nor supported organizations systematically record mentions of reports in the media or quotes in peer journals or articles. A few organizations mentioned keeping a clippings file but not compiling any information from them.

However, the limited available information on use of research reports does suggest that changes are being made to improve marketplaces through the findings from supported research projects.

Development Project Contributions have resulted in a number of examples of positive impact for supported organizations. Although most impacts are internal to the organization, given the nature of the development projects, the strength they provide to the organization is likely to have spill over effects to the consumer interest.

More generally the Contributions Program was felt to add credibility to non-profit consumer and volunteer organizations. Organizations are now better able to present research-backed arguments that are the equal of arguments by industry. As a result they are more effective lobbyists and better able to protect the interests of consumers.

3. Alternatives

The Program implemented changes recommended by the previous evaluation. This evaluation obtained the suggestions of a number of key informants related to further changes. These can be summarized as:

• More and more effective use of development support.
• Expansion of total support through innovative measures.
• Modify project selection.
• Improve dissemination of results.

These suggestions for change and the findings reported earlier support the following recommendations of this evaluation.

B. Recommendations

1. Find a way to increase the available funds

The Contribution Program is virtually the only source of government support for the research activities of non-profit consumer groups and voluntary organizations. The evaluation identified a strong and continued need for the Program. The need appears to be expanding and the Program appears not to be keeping pace.

Organizations are unlikely to become self-supporting, as research promoting the "common good" is not generally saleable. In such circumstances, external support is required to produce research results that are of benefit to consumers, the marketplace and society.

Demand for funds for useful research activities exceeds available contributions. The Program only supports two out of every five applications and provides one out of every three dollars of support requested.

Some of the supported research activities benefit government organizations such as those with a mandate to consult consumers.

2. Improve Dissemination of Information and Institute Tracking of the Impacts of Information

Information is of no value unless those who can benefit from the information use it. The degree of communication of project results by recipient organizations ranges from a little to a lot. No organizations follow-up to assess the impact of project results. Organizations could do more to disseminate information and to help assess the value of that information.

Some project results are of particular value and warrant increased dissemination efforts. This is difficult to determine at the time of the initial project application but becomes more apparent as research results become known.

Impacts of project results may occur for a diverse group who receive the information and may take some time to occur. However, obtaining the initial intentions of those who receive the information would both identify possible impact and facilitate future tracking to identify actual impacts.

These changes should be reflected in the Results Based Accountability Framework.

3. Institute General-Application Development Projects and Communicate their Findings Broadly

There are a number of topics of general applicability to all or most non-profit consumer and volunteer organizations. Topics might include governance structures, maintaining a membership base, how to raise funds, how to communicate research results or how to maximize the impact from a press conference. A development project involving a single organization would benefit that organization. However a project involving a number of organizations or one with a single organization but with results communicated to all organizations would provide even more benefit.

One such "development activity" could be a round-table discussion of topics of mutual interest to organizations. Topics might be lessons learned from a set of similar development projects.

4. Increase Transparency of Application Approval Process

The approval of applications is a highly competitive process. Funds are limited. Applications for research funding far exceed the contributions that are available. Stakes are high for organizations that rely on approval of their applications for their continuation. Although much effort is expended to make the process fair much more could be done to improve on the perception of fairness. Greater transparency is needed.

5. Encourage Greater Self-Reliance through Development Projects

A number of consumer organizations are fragile. Some may be dependent on Research Project Contributions for their survival. Reductions in the amount of support available through Research Projects would adversely affect such organizations and the consumer movement. More needs to be done to encourage the self-sustainability of organizations and to wean organizations from dependence on Research Projects.

C. Other Items of Note

Other suggestions for change are worthy of attention beyond those recommended above:

• Improper use of research findings is a potential danger in any research. To help prevent such misuse, research reports and any release of findings from them should contain a clear statement of the limitations of the study methodology and guide the reader toward proper interpretation of the results.
• In the future the program might consider requiring organizations to obtain liability insurance coverage. If required, premiums should be an allowable expense.
• Follow-on activities to an original project may be highly cost-effective as they can build on the previous research methodology and base to confirm or help modify previous findings. But line departments, that benefit directly from the research, appear unwilling to pay claiming consumer research is not within their mandate. A perception also exists that OCA is unlikely to fund a similar study to follow-up on earlier work. As a result, potentially valuable research that could capitalize on previous research supported under the Program may not proceed through the lack of a willing funding source. This should be reviewed.
• The fixed timing of the application process and research schedules which conform to fiscal year funding realities, limit the ability to be responsive to an emerging issue. Potentially OCA could consider a second, more limited, call for applications mid-fiscal year. These would have shorter time frames and focus on emerging issues only. Alternatively moving to two application windows in a fiscal year for all projects might help reduce the lags created by the current application cycle.
• There may be research opportunities spanning a number of years that are not being addressed through the current annual application process. The Program might consider commitments to multi-year funding (conditional on annual applications and annual audits).

3. Relevance

A. Introduction

This chapter explores whether the Contributions Program is still relevant. Key issues investigated through a review of the economics literature and other documents as well as through key informant interviews are:

• Program rationale and the continuing need for the Program.
• Potential duplication through alternative federal programs of support addressing this need.
• Based on evidence of duplication or the lack of duplication, the potential need to encourage complementarity or avoid gaps in support.

B. Objective-Based Rationale

Government policy requires that Canadians be provided an opportunity to participate in developing or modifying policies, regulations and regulatory programs. Canada has adopted a United Nations' Guidelines for Consumer Protection supporting a consumer voice in making and executing government policy.

The Contributions Program is intended to create a more effective consumer voice and thereby encourage a fair, efficient and competitive marketplace. As the Program's Applicant's Guide 2005-06 suggests, the Program provides:

This section explores whether the objective of the Contributions Program is still relevant.

1. Overview

Consumers of goods and services are important participants in the marketplace. However, they may be at a disadvantage relative to other market participants. These disadvantages may lead to inefficiencies and even market failures that penalize both consumers and industry and consequently society as a whole. The classic example is inequality in the information available to consumers and industry resulting in a less than optimal result in the market.¹ One solution to such problems is a stronger consumer voice providing information to the benefit of consumers and ultimately the market.²

Under-provision of information in the market place may be overcome by providing support to an intermediary (the non-profit consumer or voluntary organization) who will act in the interest of consumers to gather and provide the information. The Contributions Program supports the gathering and dissemination of information through its Research Project Contributions.

However, much information has no intrinsic value in itself. The value of information is derived by its use in some purpose or activity that results in a benefit. Value increases as uses of the information increase, suggesting that dissemination of information should not be restricted. However, because of the market failures noted earlier and because dissemination has cost, support may be needed to get the information to those who can benefit from it. Support is available under Research Project Contributions to overcome this form of market failure by encouraging the dissemination of information. Generally this information is provided at zero or low cost to maximize the benefit (widest possible dissemination).

In a more typical enterprise, the sale of their product or service provides some contribution to cover the overheads and therefore sustain the business. However, given the difficulties in charging full value for the information of the intermediary (the nonprofit consumer or voluntary organization) and the rationale not to limit dissemination, information is typically provided at a zero or low price. There is little or no contribution available from this dissemination to support the overheads and thereby sustain the intermediary. To the extent that information provision is a key activity for non-profit consumer or voluntary organizations, their financial position and security will remain fragile.

Development Project Contributions provide support for activities to help the non-profit consumer or voluntary organization become more self-supporting. An example is the development of a product that will bring a positive return to the organization while not suffering the same saleability problem as information.

2. Evidence from the Literature Review

The underlying rationale for support to non-profit consumer or voluntary organizations to overcome information difficulties is well supported in the economics and consumer policy literature.

Information is a key element of the consumer problem. Well-informed consumers will fare better in market transactions. However, information has classic properties of public goods. In the economics literature, public goods are characterized by high costs of exclusion and are non-rival in nature. High exclusion costs are generally associated with the existence of free riders—consumers can benefit from the information without paying for it. The non-rival element of information (use of information by one consumer does not lessen the value of its use to other consumers) means that once produced, it should be used widely to maximize the benefits from it. However, since the production of information and the dissemination of information are both costly, markets will generally fail to produce and disseminate enough information.

Much information is produced and is available that is related to the decisions of consumers. However, the public goods literature notes that even when information is produced, markets will still under-allocate resources to the analysis and interpretation of the information or to the value added of the information. This may be particularly the case for the types of information that are most valuable to consumers. Information is a public good that is likely to be under-provided and as the Consumer Trends Report notes, the information requirements of consumers, if they are to be adequately informed, are likely growing.

In their review of consumer protection policy that focuses on information issues, Hadfield, Howse and Trebilcock (HHT, 1996) provide an overview of the range of consumer policy issues. This wide-ranging study considers changes in the environment facing consumers and interprets these changes in the light of new economic perspectives on consumer issues.

As the study notes, in spite of many important changes in the consumer environment and in how economists view the operation of consumer markets, information remains the driving force in providing the rationale for consumer protection policy. More specifically, the literature now focuses on information asymmetry. This refers to a situation in which producers and sellers have more information than consumers.

The Contributions Program relates to the rationale for information provision in two ways. Research projects are intended to generate information that is directly relevant to consumers as they operate in the marketplace. Information is likely to be under-produced because of its public goods characteristics. Problems with exclusion imply the existence of free riders. In addition, the nature of transactions in information is difficult for both buyers and sellers. Potential buyers have difficulties in determining their willingness to pay for information with unknown characteristics. Sellers find it difficult to provide such pre-sale knowledge to buyers without fully revealing the information prior to payment. For this reason, many potentially valuable market transactions in information do not occur.

Related to the provision of information through research are the organizational challenges that voluntary consumer groups face. As Olson in The Logic of Collective Action (1965) points out, large and diffuse groups like consumers face serious free rider problems in mobilizing support. This is in contrast with producer organizations that have smaller numbers and more obvious and substantial financial interest. Development projects that help organizations with long range plans for generating revenues are a response to this type of market failure.

Continuing research can be a key element in identifying marketplace problems facing consumers. The economics literature, however, focuses on the problems that arise and much less on the specifics of how such problems are best dealt with. Is information to consumers best provided through government directly or through support to consumer organizations? The literature that identifies under-provision does not provide an answer to this question of how best to remedy under-provision. Although there is no specific policy answer here, there may be grounds for believing that a de-centralized and diverse set of organizations that are closer to consumers may have an advantage in dealing with these issues. This provides the core rationale for contributions to a variety of consumer organizations.

3. Evidence from Key Informants

Consensus existed among key informants regarding the strong rationale for support to non-profit consumer or voluntary organizations to achieve an effective consumer voice and through it a fair, efficient and competitive marketplace.

Some key informants identified the "information as a public good argument" found in the literature as the main reason supporting this rationale. In brief, organizations should be provided funding to overcome information problems that adversely affect the marketplace. About as many others saw the rationale from a public policy perspective suggesting a need for a countervailing force to stand up to a strong industry lobby. According to their view, government needed to hear both arguments before it made a decision. In support of their argument, they noted that industry is organized and had money to buy research supporting its view. In comparison, consumers are weaker and less organized. To be given equal weight in government's decision-making, consumers need a research-supported argument to counter industry's argument. The Research Project Contributions provide the funds to support this research-based argument. Further key informants holding this view suggested it was important for the social democratic process that a strong consumer movement should make the counter-argument to the position by industry. Some form of assistance, such as Development Project Contributions, was needed to overcome the current weakness of the consumer movement in Canada and to allow consumer groups to participate effectively in the process.

4. Findings on Rationale

All data sources confirm that the objectives of the Program remain relevant. Achieving a fair, efficient and competitive marketplace through the actions of an effective consumer voice remains an important objective.

Such actions typically include gathering, analysing and disseminating value-added information. This information helps resolve market failures which otherwise would prevent marketplaces from operating or operating effectively. These actions need to be supported because they too are the subjects of market failures. Government helps achieve a fair, efficient and competitive marketplace by supporting these actions and the groups who provide them.

C. Continued Need

This section explores the continued need for the Program.

Key informants felt that while a fair, efficient and competitive marketplace was still a relevant goal it may be harder to achieve it now compared to an earlier time. This is because of the increased complexity, pace of change, and new advances in technology that may place the consumer at an increasing disadvantage in terms of staying current, knowledgeable and protected. Although difficult for the average consumer to stay informed, some groups of consumers (the elderly, lower income groups) may be increasingly marginalized by these changes.

The literature review supports this general view. Consumer choices have expanded. Together with this generally positive development, comes a requirement for information related to this expanded choice set. Globalization increases the availability of products and services and may often lead to the provision of more information. However, consumer evaluation of information, much of which is provided by suppliers, may become more problematic. This expanding choice set and the increasing information processing requirements around it suggest, at least, a continuing need for the Program.

Some key informants suggested that issues have expanded greatly since the 1980s while the Program has shrunk in real terms. In the 1980s the Department of Consumer and Corporate Affairs supported non-profit consumer and volunteer organizations through core funding (not tied to a specific output) and research support. The real dollar amount of support was higher. At the same time the issues were fewer and less complex. As a result, they suggest that while needs are expanding the ability to deal with those needs has contracted. According to this view, if the consumer movement is a prerequisite in a fair, efficient and competitive marketplace then the Program may not be keeping pace with these objectives.

In terms of the objective to create an effective consumer voice, a few representatives of eligible organizations felt that the reliance on Research Project Contributions to pay for overheads of organizations was in fact hurting the consumer movement in general. Some organizations were increasingly driven by a requirement to conduct research to survive. Failure to be successful on the requisite number of applications in a given year could spell disaster for the organizations. Contributions to overheads on research projects were also so slim that there was never enough organizational capacity left to develop new initiatives to allow them to break their dependence on research support.

Also as the only source of overhead support, the Research Project contributions had, in the view of a few key informants, resulted in consumer organizations being transformed into research organizations. Although this satisfied a need for informed consumer input by the government, the resulting heavy research orientation by consumer groups was, in the opinion of the few, not in the best interest of the consumer movement.

Poor financial health and lack of sustainability of non-profit consumer and volunteer organizations was a general view held by all key informants. Some suggested that organizations had become less viable in recent years. To the extent that an effective consumer voice requires such organizations and given that the Contributions Program is one of the only programs of support for such organizations (see next section) there continues to be a need for the support provided by the Program.

From the literature review (HHT, 1996), there is the suggestion that organizing consumers may be more difficult now than in earlier time periods. Although no trend data exist on which to base firm conclusions, the HHT assertion that "consumer activism seems curiously in abeyance" appears accurate. In part, this may reflect a view that government is providing consumer protection policies. But whatever the cause, most consumer organizations appear to face substantial organizational and development barriers. This too points to a continued need for the Program.

Both the literature review and interviews suggests that there is a continued need for the Program. Evidence suggests that the need is expanding and there is some evidence that the Program is not keeping pace. This expanding need is in part due to an increasingly complex marketplace and perhaps a greater information-processing requirement for consumers to remain effective in the marketplace.

D. Possible Duplication

This section explores whether the continued need for the Program is being met through some other initiative or initiatives.

The consensus across all groups is that there is little or no support available from federal departments for general consumer research of the type funded by the Contributions Program. A few identified a new program started by the Privacy Commissioner, apparently modeled on the Contributions Program, but covering a much narrower set of research topics. Other funds may sometimes be available for very specific issues but these are project-related and not part of a general program of research support. Organizations supported by the Contributions Program rarely are able to access these project-specific funds.

Support is sometimes available through Federal Departments related to consultation with consumer organizations. However it appears to be infrequent and generally limited to out-of-pocket expenses.

Some organizations may get time and expenses paid if they are given intervenor status, for example in a rate hearing. This happens infrequently for the organizations covered in our interviews.

Although no longer available, there was a Voluntary Sector Initiative (VSI) that provided support similar to Development Project Contributions. Only a few organizations had been able to make use of the VSI. The few other organizations that were aware of VSI had not received support through it.

Organizations in Quebec are able to access operating grants through the Secrétariat à l'action communautaire autonome du Québec. About 40 organizations share in approximately $800,000 available under the program annually. We did not find any evidence of similar programs operating in other jurisdictions. Some provincial/territorial ministries may fund individual research projects by organizations but these are few in number and value.

Support to university researchers to conduct consumer-related research is possible through the Social Sciences and Humanities Research Council (SSHRC). However a partial review of grants made through SSHRC did not identify any research grants that would be similar to the projects funded by the Contributions Program.

The Community-University Research Alliances (CURA) program of SSHRC was identified as possibly promoting similar research to the Contributions Program. CURA promotes research and social innovation by funding vital, creative partnerships between universities and communities. It helps universities and their local partners to work together for the social, cultural and economic development of communities. Potentially this program might better fit with Quebec-based organizations given their community focus. However, there appears a limited potential for overlap between CURA and the Contributions Program.

As a result, there appear few if any Federal programs that overlap with the Contributions Program. Some funds may be available through individual research projects by Federal departments or through Provinces and Territories. However these are infrequent and not part of a formal and continuing program of support. The only significant program of support, but not directed at research, is the operating grants available through the Province of Quebec.

E. Encouraging Complementarity and Avoiding Gaps

In the past, the absence of other Federal programs of support for consumer-related research activities meant that no linkages to avoid duplication were needed. However, staff of the Office of Consumer Affairs consulted with other departments to identify relevant topics for research projects that then became Suggested Priority Areas of the Applicant's Guide. This helped ensure that other department's needs for consumer-related research were being addressed in the absence of other Federal programs of support.

In the future, staff of OCA and the Privacy Commissioner will need to co-ordinate activities to avoid duplication of privacy-related research given the latter group's new program of support.

F. Summary

All data sources confirm that the objectives of the Program remain relevant and there is a continued need for the Program. Evidence suggests that the need is expanding; due to society's increased complexity, pace of change, and new advances in technology; and there is some evidence that the Program is not keeping pace with this need.

There are few if any Federal programs that overlap with the Contributions Program. Some similar research projects might be supported by Federal departments or through Provinces and Territories but these are infrequent, specific to a research topic, and not part of a general program of support. In the absence of similar programs offered by other Departments in the past, staff of the Office of Consumer Affairs consulted with departments to identify relevant topics for research projects. These become Suggested Priority Areas of the Applicant's Guide. In the future, co-ordination between the Contribution Program and a new program by the Privacy Commissioner will be required to avoid duplication related to privacy issues research.

¹ An example involves buyers and sellers of a good. Sellers have experience with the good; the buyers do not. The sellers are in a much better position to understand the quality of the good compared to the buyers. As a result, sellers will ask a price for the good that is greater than the buyers are willing to offer. The sellers refuse to accept a price lower than would be fair given the quality of the product. The buyers refuse to pay more in the absence of information suggesting the quality of the product is worth a higher price. No agreement can be reached, the good is not exchanged, the market fails and both buyers and sellers are penalized.

Another example of market failure involves a single seller of a product and many potential buyers. The single seller can exploit knowledge of the marketplace to sell only the amount of product that will maximize profit and not the larger amount of product that would maximize the benefit of consumers. In the example, consumers are many, disorganized and each with insufficient market power to affect the outcome. As a result an inefficient market result occurs with fewer needs being met than would be the case in a more competitive market. (Return to reference 1)

² In the first example, testing of the product by an independent group could reveal what the sellers know and what the buyers need to know to settle on a fair price. In the second example, research by a independent group leading to a strong lobby before a regulator of the monopoly might lead to a price for the product which would benefit more buyers while bringing a fair return to the seller.

The solution requires involvement by an intermediary (the non-profit consumer or voluntary organization) because the lack of information is itself a result of a market failure. Consumers collectively benefit from the information when the cost of the information is less than the sum of the benefits consumers derive from it. However if the cost to obtain the information is more than the benefit the information provides to a single individual, there is no incentive for any one consumer to obtain it. A "market" for information is unlikely to form, as the value of the information is hard to prove in advance. As a result, information as a good, suffers the same market failure problems as in the first example. In addition, once information is released it is hard to charge others for the same information because others can "free ride" on the previous purchasers. Again a market for information is unlikely to form necessitating the involvement of an intermediary. (Return to reference 2)

4. Success Related to Support

A. Introduction

This chapter explores topics related to the achievement of success through the Contributions Program. Success is assessed separately for Research Projects, Development Projects and for the Program as a whole.

B. Success related to Research

1. Impact on Recipient Organization

We measure the success of Research Project Contributions through the effects on recipient organizations in terms of:

• Consumer research capacity.
• Intellectual property/capacity.
• Solicitations for research, analysis, and consultative work.
• Demand for input by marketplace stakeholders.

a) Impact on Consumer Research Capacity

There appear to be two general models for the research conducted through a Research Project Contribution. The first involves researchers within the recipient organization conducting all aspects of the research. The second involves the recipient organization contracting out some or all of the research activities. The first model appears to offer the greatest opportunity for an impact on the consumer research capacity of the organization. In the second model, staff of the organization have less hands-on experience related to the research. However, there can still be impacts through the development of the research application, interactions with the researchers and use of the research results. Learning may also occur through interaction with a methodologist hired to assist in the design stage, to evaluate the final product, or to provide other input.

Key informants who were able to comment on the research capacity change for recipient organizations felt that it had increased through the research projects. A number of organizations identified their ability to take on larger or more complex research assignments with successive projects. Others suggested that being successful on projects each year had allowed them to retain staff they otherwise could not have kept. This stability of trained staff resulted in enhanced research ability for their organization. A number identified improvement that had taken place in their research competence through interactions with the external methodologist required by the Program to review methodology and final reports.

One key informant suggested that universities were starting to acknowledge the research contribution of consumer groups and their role in society. The individual noted that the Université du Québec à Montréal recently gave a prize to the director of a consumer organization in recognition of the group's research activities.

b) Impact on intellectual property/capacity

A generally held view by key informants was that recipient organizations are developing expertise in particular areas through their involvement with Research Project Contributions. They report being invited to conferences or being asked for input related to government initiatives. They receive more media attention related to their expertise. A number of recipient organizations identify their credibility increasing through their involvement with the Research Project Contributions.

The body of work over a number of years for a single organization covered by our case studies is an illustration of the intellectual property that has been vested in the organizations. A minority view was that expertise developed through the Program then made it harder for others to compete against this expertise for future Research Project Contributions. Those groups who are successful in past applications are then more likely to be successful in subsequent applications. This may be an indication of the development of intellectual property in the recipient organization.

Another minority view was that the competence gained through past projects did not translate into success on future projects in the same area, suggesting that no intellectual property had been created.

c) Solicitations for research, analysis, and consultative work

A few examples were put forward by recipient organizations of cases where they had participated in a follow-on activity, including providing policy input, as a result of the expertise they had developed through a research project.

However, this participation was not always funded, particularly if it involved a federal or provincial government. More typical in activities involving a non-profit consumer or volunteer organization and government is for the latter to cover out-of-pocket costs only (typically travel but sometimes time) of the organization. This practice likely evolved when organizations received operating grants from the federal government but no longer fits with the current fiscal reality. Organizations are typically paid for their time and expenses if they are granted intervenor status (for example in a rate hearing of a regulated monopoly).

Some, albeit few, related activities have involved private sector work. Policies of some consumer groups prevent organizations taking on such work due to concern over conflict of interest. Some research activities have involved support from the private sector conducted on a non-directed basis.

In general, organizations have achieved only limited leverage of the expertise they have developed through the Program in paid activities for others.

d) Demand for input by marketplace stakeholders

Demand for input on consumer-related matters by non-profit consumer and volunteer organizations was judged to be high by recipient organizations especially after the release of a project report. Demand has two main sources—government and the media.

Demand occurs in areas even outside organizations' area of expertise. One possible interpretation is that those who need consumer input and are unable to obtain it, subsequently try to obtain it from any group, not just those with expertise in the area.

Of those who commented, organizations appear to be able to respond to such demands in only a small percentage of cases. One identified the fraction at 1 in 10 cases. Organizations report that they are not funded for these activities and to spend time on them ultimately takes away from other activities. They need to be very selective about what demands they meet and to understand that to meet one unpaid demand, they must take away from all other activities.

e) Summary of Impacts on Recipient Organizations

Recipient organizations increase their capacity to do research and may do better research through their involvement with the Research Project Contributions. Many are developing expertise in particular consumer issue areas. Their credibility is also enhanced. However, few recipient organizations appear to have capitalized on these impacts through other funded activities. Although demand for their input on consumer issues is high, consumer groups are able to meet very little of this unfunded demand. To do so would take away from other activities of the organization.

2. Impact on Organizations' Employees

As previously mentioned two models exist related to the funded research. Those groups that conduct the research internally are likely to see the biggest improvement in the human capital/expertise of their employees. However, human capital/expertise increases are also possible for employees of organizations that tend to contract out the research activities although the skills are likely different between the two models.

Recipient organizations report an improvement in the human capital of their employees through activities involving the research projects. As previously noted, some suggested the string of successful projects had provided a continuity in the work of their researchers and subsequent increase in their expertise that the organization would not have been able to achieve on its own.

However, this increase in expertise is a double-edged sword. A number of organizations report that trained staff gets hired away by other employers. Examples cited included a crown corporation, federal government, health research organization, university-based organization, private research, and a consumer organization (Consumers International). Most were said to be still active in the public policy area. As a result, there is some support for the view that the research environment provided by supported consumer organizations acts as an incubator for developing consumer-related researchers who ultimately are hired by outside organizations.

The challenge as reported by one organization was to develop staff but also to retain them. We suspect the issue is one of higher pay scales outside the non-profit consumer and voluntary sector. Staff that remain, place higher value on non-pecuniary aspects of their job (have a missionary spirit according to one organization).

A number of organizations also report that summer/co-op students are exposed to a richer training environment in a broader range of research activities than they would likely experience in any other organizational setting.

3. Impact on Those Outside Supported Organizations

An impact of the supported research may also occur beyond the recipient organization or its employees. This led us to investigate who receives study results and how these results affect the work recipients do or actions they take. In the process, we assessed the timeliness and dissemination of study results. This section discusses findings on these issues.

Projects are typically one year or less in duration. Reports are to be completed by fiscal year end (March 31) to meet requirements of the government funding cycle. However, many are not. Reasons for the lateness of reports are also many. These are grouped within themes and presented by frequency of response, from most to least frequent, below:

• Lack of core depth in organizations/ everyone over-extended.
• Project reliant on input of others and this is sometimes delayed.
• Vagaries of research/ cannot always predict how things will go.
• More time-sensitive issue comes up/ research can be set aside while other issue cannot/ OCA is the least pressing demand on time.
• Groups don't understand the size of the research or have unrealistic timelines.
• Organizations live off cash flow from project and the money has been spent by report time.

Summarizing across these reasons about one-half reflect a lack of capacity in the organizations, about one-third indicate the nature of the work and the balance reflect some lack of experience or competence in the management of the project.

Once reports are produced, the extent and form of dissemination of information varies considerably across organization and likely across projects within an organization. Some identified that some research results turn out not to be worth disseminating.

Most organizations identify posting a summary of the report on their web site and providing the capability to download the report from their site. Most also report holding a press conference or providing a media release. Many develop and send out reports to a list of key stakeholders (including industry) they think should get the information. A few identify making the report available to their membership. A few also mention sending the report to politicians or meeting with politicians to review the study findings.

One organization mentioned doing all of the above but also sending reports to public libraries, universities, and industry, as well as using the information in a weekly television program. One organization sends summaries of their research reports to all non-profit consumer and volunteer organizations.

Some identify doing little with the reports. Some suggested more could be done.

A number identified problems with dissemination that they linked to the Program. Before the research began they found it difficult to identify how much dissemination of results would be required. Given the competitive nature of the application for support and the uncertain surrounding the weight attached to dissemination in the analysis of proposals, organizations may under-allocate resources to dissemination in their application.

The OCA has begun posting summaries of research projects on their website and providing links to the website of the organizations that produced them. However, OCA does not own the report (ownership resides with the organization under a contribution agreement). As a result, OCA's role in distribution is limited although they may be involved in notifying a government department of a research report of potential interest to them.

Key informants interviewed in the study who should have been in positions where they could make use of research reports said they did not receive them routinely or were only able to find them with some difficulty. However, they only knew to look for them if they found a quotation of the study. Some key informants suggested they should, but do not, receive notice of reports produced under the Contributions Program. Recipient organizations identify they should but do not know the outcomes of other organizations' projects.

We have limited information on the use of research results. Organizations do not record contact information for those who obtain their reports, frustrating attempts to follow-up to assess an impact and neither the OCA nor supported organizations systematically record mentions of reports in the media or quotes in peer journals or articles. A few organizations mentioned keeping a clippings file but not compiling any information from them.

Information we have collected on the use of reports is indicative but not definitive:

• A number of studies (automobile retailing and repair) have been featured in television programs. The media has covered issues such as problems with the Drive Clean program in Ontario.
• The Drive Clean study results were discussed at a conference in San Diego.
• Public Interest Advocacy Centre (PIAC) provided useful input through public process and parliamentary committees related to the Personal Information Protection and Electronic Documents Act. Amendments that were put forward by a Member of Parliament were based on briefings by PIAC.
• Research conducted by Option Consommateurs and PIAC on compliance was cited numerous times by other groups.
• Studies on e-commerce were quoted at the 1998 OECD Ministerial Conference on Emerging Market Economy Forum on Electronic Commerce.
• The consumer representative on the Working Group looking into Electronic Funds Transfers is having a dramatic impact on the laws/regulations that will be developed. This consumer involvement would not occur without the past Program support.
• The new Ontario Consumer Protection Act (likely to be passed in 2005) includes a section on disclosure of information on commissions to workers doing repairs on vehicles as a result of the funded research by the Automobile Protection Agency (APA). The APA study was identified as the principal reason for making the commission receive as part of a workers' pay a component of Ontario's proposed regulation.

The variety and diffusion of stakeholder impacts noted above suggest the challenge involved in trying to assess the overall impact of the Contributions Program. Our study was limited to assessing use of research results by stakeholders. Assessing the overall impacts was beyond our evaluation mandate.

C. Success related to Development

Relatively few of the organizations included in key informant interviews had been involved in Development Project Contributions. As a result, our findings are more tentative related to development support. However, the few cases provide a number of examples of positive impact.

1. Impact on Recipient Organization

The following are examples of impact on organizations through the Development Project Contributions:

• An organization was provided funding to translate an educational tool in the energy field that they had developed previously. As a result, the Quebec-based group was able to access the English market with their product. The organization received a contract for 10,000 units that were distributed throughout Canada. Without funding, the English product would not have proceeded. The group would not have received the revenue from this broader distribution and the resulting publicity.
• Funding allowed an organization to further develop a family budgeting software package they expected to market after copyright problems had been overcome.
• Funding was provided to develop a business plan to merge two consumer organizations. As a result, the merger was more harmonious and efficient, leading to a much stronger combined group.
• Through funding one organization developed an on-line virtual member pool of 5,000 individuals. This allows them to do quick consumer surveys of their members. This facility has become an important research tool for them. It has increased their status and recognition with the media and other groups.
• The Canadian Consumer Initiative (supported through contributions to a few member organizations) is a forum for the development of a shared perspective by its members. This initiative has identified opportunities for mutual benefit and for joint action to improve their collective benefit. One such action was to speak to the Minister responsible for Industry Canada.
• The APA has identified class action suits as an opportunity to build on its expertise and to establish a potential revenue stream.

2. Impact on Those Outside Organizations

Development Project Contributions have as their principal goal the strengthening of the consumer organization. The Development Projects, noted above, appear to achieve this goal. Although this may ultimately result in benefits that extend beyond the organization, most of the impact of development support is likely to reside with the organization. The only direct impacts on groups, outside of organizations supported through Development Projects, were the closer more effective link between one organization and its members through the on-line membership initiative and potential awards to consumers as a result of class action suits.

D. Success related to Consumer Movement

In addition to the impacts noted separately for Research Project Contribution and Development Project Contribution, we asked more generally about the impacts of the Contributions Program on the consumer movement in Canada and particularly on the organizational capacity of organizations within the consumer movement.

The most frequent impact identified was the added credibility the Program has brought to non-profit consumer and volunteer organizations. Organizations are now better able to present research-backed arguments that are the equal of arguments by industry. As a result they are more effective lobbyists and better able to protect the interests of consumers.

5. Alternatives

A. Introduction

This chapter explores alternatives to the current Contributions Program. First it discusses recent changes to the Program before presenting suggestions for other possible changes.

B. Recent Changes

The previous evaluation (2001) produced the following recommendations (in bold) with resulting changes made to the Program (in regular text):

• Consultations should be conducted with consumer organisations before the annual research priorities are set for the funded projects. This has been done.
• Consumer organizations should be consulted on any review (e.g. electronic automation) of the proposal process to ensure that it meets their needs. OCA agreed to consult organizations in the event of automation and also noted that automation would not replace the traditional application, quarterly reporting and payment requests methods.
• The challenge surrounding communications of projects results may be addressed by modifying the program's requirements to include a more detailed communications plan, and by ensuring that adequate funds are provided to support these communication activities. Sections of the Program Applicant's Guide dealing with communication requirements were revised to provide more specific requirements. The OCA noted that communication activities were an allowable expenditure.
• The program should encourage the use of seasoned researchers and provide sufficient funds to enable consumer organizations to purchase the adequate expertise to conduct their research. OCA recommended that applicants retain and consult experts on methodology at the project design stage. Organizations are required before making project results public to have methodology, findings and recommendations reviewed by a qualified expert. The cost of such expertise is an allowable expenditure.
• The program should explore alternative ways of helping organizations become more self-sufficient, such as conducting a review of best practices in the area. The OCA paid for organizations to meet with a facilitator to share best practices, explore a more co-ordinated and strategic approach to resolving issues affecting the marketplace, and examine better means of working together.

C. Suggested Improvements

The evaluation explored suggestions for improvement from a number of perspectives including how to make the Program more cost-effective. We discuss suggestions by key informants according to the frequency of mention from those receiving the greatest to the least support.

The most frequently suggested improvement was to increase development support. Many suggested that non-profit consumer and volunteer organizations needed to become stronger. Development support was seen to be the most effective mechanism to achieve this goal, although a few acknowledge that strength could also be gained, albeit over a longer period, through research activities. A number were careful to observe that there needed to be a transition and that research support should not be cut back, at least initially. Most observed that if there was more money it should be devoted to development. Some suggested that development projects could then help organizations wean themselves from their reliance on research support. If cut backs were needed (not all would comment on this possibility) research was the area most often identified for cuts although many commented that this should only be after an injection of new money to development support to help ease this transition.

In support of the view that any reduction in research support should be done slowly, we observe that for the six largest recipients (representing 76% of Research Project Contributions over the period 2000/2001 to 2002/2003), their research contributions accounted for 29% of the average annual revenues for the group. Research Contributions as a per cent of revenue ranged from 11% to 44% across the six organizations. This high proportion of revenues derived from the Research Project Contributions for organizations suggests that such organizations might be significantly and negatively affected by any reduction in the available research support.

The next largest group of comments revolved around suggestions for changes to funding support more generally. There were calls for:

• Moving to a less directed form of support such as core funding. Note that there were as many calls for core funding as there were detractors from moving to core funding.
• More money. Suggestions included some form of voluntary tax (such as a lottery or a contribution made at a point-of-sale) or support through a charitable foundation.
• Create a model similar to a Citizens' Utility Board whereby consumers can add an amount to their (utility) bill and direct that money to a group who would represent their interests in utility matters. (Note this might be extended to a voluntary tax added to the bill of a retailer sympathetic to the consumer movement or wishing the publicity around such a show of support. Some retailers currently direct donations to charitable causes by adding the donation to the total at the check-out counter.)
• Money left in individuals' bank accounts for more than 10 years should be given to the Contributions Program instead of being sent to the Consolidated Revenue Fund. In the view of one key informant this money more closely belongs to consumers and not to all taxpayers.
• Turn over core research activity to academics. Have consumer organizations provide their opinions based on their consumer perspective.
• Provide encouragement or credit for organizations that are able to identify sources of matching funds for their project ideas.
• Provide low-interest, medium term loans for organizations that could be paid back on a recoverable basis (possibly against future revenue streams or future research projects).
• Broaden coverage of Program to be more inclusive.
• Keep and distribute information on expertise of organizations. Update annually. Do not support a project to develop a particular expertise when another organization already has it.
• Seek out and fund projects of general application to all organizations such as effective governance models or how to maintain members. Communicate findings to all organizations.

A number of comments were received that relate to the administration of the project selection:

• Find a more simplified way to screen projects to reduce administration costs.
• Involve consumer groups or grass roots consumer in project selection decisions.
• Provide more specific criteria and a marking scheme to help organizations be more successful and focussed. Give a more specific list of project topics with target amounts attached.
• Do double blind screening of applications. Assign number to proposal, then strip off organization submitting proposal before assessment.
• Encourage partnerships among groups.

A number of comments were also received around ways to improve the dissemination of research findings. Comments included:

• Develop a better and centralised mechanism to distribute the results.
• More money into dissemination of results.
• Adjust parameters of research to include communication.

Finally a few suggestions were received that did not fit within the above categories:

• Should only do research projects of immediate benefit to consumers.
• Should hook into international scene to increase quality and benefit from existing expertise on global issues.
• Hold a conference every second year to share best practices, results of projects among organizations.
• Provide management guidelines on collaborative projects. Accountability around collaborative projects could be better. Someone should monitor to verify input in a collaborative arrangement.

D. Summary of Alternatives

The Program implemented changes recommended in the previous evaluation. In brief these were to improve consultation with organizations, better communicate project results, encourage quality of researchers/research, and explore ways to improve self-sufficiency of organizations.

This evaluation obtained the suggestions of a number of key informants related to further changes. These can be summarized as:

• More and more effective use of development support.
• Expansion of total support through innovative measures.
• Modify project selection process and guidelines.
• Improve dissemination of results.

Evaluation of the Contributions Program for Non-Profit Consumer and Voluntary Organizations

February 2005

Background

This evaluation study was undertaken by TNS Canadian Facts on behalf of the Audit and Evaluation Branch of Industry Canada. Evaluation issues of the study were grouped thematically reflecting the Program's:

• Relevance (rationale, need, and avoidance of duplication).
• Success overall and for separate research and development components.
• Alternatives considering cost-effectiveness, lessons learned and suggestions for change.

The Contributions Program for Non-Profit Consumer and Voluntary Organizations is intended to create a more effective consumer voice and thereby encourage a fair, efficient and competitive marketplace.

The evaluation study confirmed that the objectives of the Program remain relevant. Evidence suggests that the need is expanding and there is some evidence that the Program is not keeping pace with this need. More generally, the Contributions Program was felt to add credibility to non-profit consumer and volunteer organizations. Organizations are now better able to present research-backed arguments that are the equal of arguments by industry. As a result, they are more effective consumer representatives and better able to protect the interests of consumers.

This evaluation study will be very helpful in making recommendations to guide decisions for the future of the Program.

Program management is committed to addressing each of the recommendations found within the evaluation study as set out in the detailed Management Response below. The management of the Contributions Program accepts and will act upon other suggestions for change listed in "Other Items of Note" in the evaluation study.

Management Response to the Recommendations

1. Find a way to increase the available funds

Management Response

Agreed. Program management strongly supports this recommendation.

Demands for funds for useful research activities exceeds available contributions. The Program only supports two out of every five applications and provides, on average, one out of every three dollars of support requested.

Program management plans to build a strong business case in the coming year in support of additional funds for the Contributions Program. However, increased funding will depend upon IC and federal government strategic priorities for the coming years.

Management Response

Agreed. Program management strongly supports this recommendation. Some of the supported research activities benefit government organizations such as those with a mandate to consult consumers.

OCA will discuss with Treasury Board Secretariat how best to support this recommendation through existing TBS policies.

2. Improve Dissemination of Information and Institute Tracking of the Impacts of Information

Management Response

Agreed. OCA recognizes that project results are of little value unless decision makers and interested stakeholders are made aware of them.

Major recipient organizations currently house on their respective websites research briefs, and/or executive summaries, and/or the whole report of projects funded under the program. OCA will strongly encourage all recipients to post reports on their websites.

In June 2004, the OCA launched the Consumer Policy Research Database accessible through its website Consumer Connection. The database contains short summaries and hyperlinks to project reports produced by recipient organizations under the Contributions Program, and to consumer research documents undertaken by university researchers, and governments. OCA will continue to post and actively promote this database.

Management Response

Agreed. The degree of communication of project results by recipient organizations ranges from a little to a lot. No organizations follow-up to assess the impact of project results. Organizations could do more to disseminate information and to help assess the value of that information.

Recipient organizations will be required to provide with the final project report a communications report on dissemination activities of project results, list of stakeholders distributed copies of the report or executive summary, website access of project report, press conference, press release, media coverage of findings, etc. The communication report will be reviewed by the OCA, and any omission of interested stakeholders will be reported to the organization for remedy. This information will be used for performance measurement and evaluation purposes.

Management Response

Agreed. While there are currently provisions in the Program for applicants to claim the cost of communication activities during a project, there are some project results of particular value which warrant increased dissemination efforts in the year following project completion, and for which the organization has no financial support to undertake.

The OCA will examine funding mechanisms that make it possible to support these activities. The OCA will favour a type of funding that is not cumbersome for applicants or an administrative burden to OCA. If additional funding were provided, OCA may also designate a certain proportion of the Program budget for this type of funding.

Management Response

Agreed. As mentioned above, recipient organizations will be required to provide with the final report a communications report on dissemination activities of project results. Stakeholders who received copies of reports could be interviewed during the next program evaluation scheduled in 2010.

In addition, recipient organizations will be asked to provide OCA with any feedback they may receive from stakeholders on project results. The OCA will also take note of any response received directly by stakeholders. These stakeholders will be singled out as contacts for the next program evaluation.

Management Response

Agreed. These activities will be reflected in the Results Based Accountability Framework.

3. Institute General-Application Development Projects and Communicate their Findings Broadly

Management Response

Note that projects dealing with issues aiming to strengthen the consumer movement, such as research on the effectiveness of Citizen Utility Boards (CUB), or an assessment of developmental factors for consumer associations, are funded under Research Project Contributions. Research findings for this type of funding are public and shared with all organizations.

Organizational development contributions aim to strengthen an organization's capacity to diversify revenue sources and increase its level of self-sufficiency. While organizations willingly work together to develop common policy positions and action plans to address consumer issues of national interest, they are, however, very reluctant to share results of their development activities with organizations they view as direct competitors when it comes to the limited resources available to them.

The OCA analyses development project results to further the evolution of the consumer movement in Canada, and to shape the Program. The OCA will examine the possibility of sharing the results of this analysis with all consumer organizations without revealing privileged proprietary information or jeopardizing organizational confidentiality.

4. Increase Transparency of Application Approval Process

Management Response

Currently, all administrative changes to the program are reported in the "What's New This Year" section of the Applicant's Guide. Changes are also reviewed with applicant organizations at bilateral meetings or telephone calls to discuss project ideas for the coming funding year. The application and evaluation processes are explained in detailed in the Guide, and organizations are invited before applying to discuss proposals or funding decisions with representatives of the OCA.

Nevertheless, in order to further assist organizations, the OCA plans to organize this Fall a seminar on consumer priority issues for the coming funding year. The OCA will also take this opportunity to present to attending organizations changes to the Program and review the assessment process. Depending on the success of the event, it may be repeated on a yearly basis.

Management Response

Agreed. Eligible points assigned to each question of the proposal form will be included in the Program Applicant's Guide (2006-07).

Management Response

Organizations are invited to inquire on the assessment of their project proposal, and the OCA provides detailed reports of assessments, including the proposal's strengths and weaknesses, and identifying areas for improvement if the organization plans to resubmit the proposal the following year.

A detailed assessment report contains information on the number of proposals received for the priority areas, the highest and the lowest scores allocated to proposals in that area, and the score the proposal received. Organizations are provided the scores per assessment sections such as: Relevance, Importance and Timeliness of the Issue (based on 30 points); Methodology, Capability, and Cost (based on 50 points); Intended Uses (based on 15 points); and Communications Plan (based on 5 points).

5. Encourage Greater Self-Reliance through Development Projects

Management Response

Agree. Over the last five years, less than 40% of the amount requested for research projects was awarded. During the same period, 31% of the amount requested in development contributions was funded.

One in two research project proposals submitted received funding compared to only one in three for development project proposals. Both types of funding would benefit from additional funds. However, a significant proportion of the funds would be allocated to development projects, should the Program receive increased funding.

Appendix A—Steering Committee Members

Committee Members:

Radek Bandzierz—Chair
Aboriginal Business Canada
Industry Canada

Karen Bron
Analysis and Integration Directorate
Justice Canada

David Hoye
Consumer Information and Coordination
Industry Canada

Pippa Lawson
Canadian Internet Policy and Public Interest Clinic
University of Ottawa, Faculty of Law

Genevieve Reed
Option consommateurs

Industry Canada Secretariat:

Peter Zoutis
Audit and Evaluation Branch
Industry Canada

Observer:

Jean-Baptiste Renaud
Consumer Information and Coordination
Industry Canada

Appendix B—Case Studies

Automobile Retailing

Introduction and Background

This case study focuses on a number of related research projects conducted by the Automobile Protection Association (APA). Funding for these projects was through the Contributions Program for Non-Profit Consumer and Voluntary Organizations of the Office of Consumer Affairs, Industry Canada.

Links to other Support

Total contributions to projects related to automobile retailing were $272,000 in the last six years. This represents slightly less than one-half of the support to APA through the Program in this period.

Contributions by the Program represent 11% of APA revenues in the last four years. Other principal sources of revenue are annual memberships, services not covered in membership fees (Dealer Invoice Cost Service, New Vehicle Buying Service, and Used Car Buying Service (in Toronto only)), cost recovery through class action suits, and publications (Lemon Aid a new car buyers guide, and The Complete Canadian Used Car Guide).

The APA is not aware of any other federal programs of support targeting nonprofit consumer and voluntary organizations. They have participated in job creation programs and in student placement programs.

APA also receives support through a research program of the Office de la protection du consommateur in Quebec. This program is similar to the Contributions Program but with an annual budget of approximately $80 thousand.

Project Impacts

The APA studies involve researchers going to new and used (depending on the study) automobile dealers posing as "would-be" buyers. They follow a script or interview protocol and gather information on a number of characteristics of the sales experience. Information gathered is used later to rate the dealer/sales experience as "pass" or "fail". In the studies covered by this case, locations were selected purposively. (The sampling technique will be the subject of a later discussion.)

The projects are conducted in partnership with CTV. The network devotes about 20 minutes to each study on its current affairs program, W-FIVE. The program airs twice—on Saturday (prime time) and on Sunday. Each story is also widely picked up by news services. This results in a broad and cost-effective dissemination of the findings of the research to an audience likely in excess of two million people at no cost to the Program.

In addition to coverage by the media, the APA web site provides summaries of some of the projects covered in this case study—new (2000 and 2002) and used (2004) but does not provide a link to download the reports. The Consumer Connection website of OCA provides a summary of the 2001 retail audit (new vehicles) and suggests that a copy of the report can be obtained by contacting the APA.

The investigative reporting technique of the APA studies, although not traditional research, was seen to be very effective. It is the only publicly available assessment of automobile retail practices across Canada. It is relatively inexpensive, especially considering its potential impact through broad dissemination of study findings. In large measure this is through the original W-FIVE coverage and subsequent media attention. Consumers pay attention to the findings and, in turn, governments are encouraged to take action.

An association representing automobile dealers, however, was critical of the methodology employed and the subsequent reporting of the results around the 1999/2000 study. They commissioned a report critiquing the methodology. Principal concerns were: measurement validity (concern that the measures used could not be applied consistently), reliability (concern that measurement would not be reliable due to a lack of rigour and documentation), and representativeness (concern that the samples used by the study were not representative of the population of retailers). They also produced an article (Canadian Auto World, (no date)) quoting Carl Compton, registrar of the Ontario Motor Vehicle Industry Council (OMVIC) as saying; "A media story, however, which advises its viewers that an entire retail sector composed of several thousand businesses can be characterized as dishonest on the basis of a sample of 45 businesses, 62 percent of which 'failed' a subjective undisclosed test, is outrageous and itself deceitful."

The studies were said to have increased knowledge of automobile retail practices in the marketplace. APA has been able to use this knowledge to add value in some of its products and services available to members and non-members. The automobile industry has been able to use this knowledge to develop training tools to improve the sales experience of its customers.

The attention brought to bear on questionable business practices through the reporting may act as a deterrent. Salespeople and dealerships do not want to be identified negatively in an APA investigation. This likely discourages others from adopting similar practices. Reporting of positive experiences observed through the studies also likely rewards such salespeople and dealerships.

One view, expressed in our data collection, was that automobile retailing had undergone a fundamental change in its operations (full disclosure, more simplified contracts) possibly affecting 30% of the cars purchased in Canada through the studies. Another view was that the studies have had no such overall effect.

In Ontario we were told that the reports were very influential and were being used in a comparative way. If Toronto did poorer than Montreal in the study, the study would generate a lot of attention. The responsible Minister would want an explanation and the Ontario Motor Vehicle Industry Council (OMVIC, the self-regulator in Ontario) would be asked for its view on the reasons for the poor showing and corrective actions. (The use of study results to make regional comparisons is discussed in a later section.)

Funding provided through the Contributions Program has allowed the APA to conduct more extensive research than would otherwise be possible. More recent studies have included investigations in five cities across Canada. A small organization (located in Montreal and Toronto), APA suggested it could not mount such a broad undertaking without this external support. The media attention has increased the awareness and credibility of the organization across Canada.

Other evidence of the impact of APA studies (funded by the Program but not all covered in this case study), which might support the previous views of the retailing studies' impacts, included:

• Consent Prohibition Order issued by the Federal Court of Canada requiring Toyota to amend its sales, promotion, training and monitoring practices for the Access Toyota Program.
• Charges laid against two used vehicle dealers based on enforcement activities by the Quebec Consumer Protection Office.
• Requirement for disclosure of the internal commission structure for repair-persons required under Ontario's new Consumer Protection Act.
• Full disclosure leasing for new vehicles.
• National motor vehicle repair chain upgrading its training practices and monitoring activities through its own mystery shoppers in Quebec.
• Deferral of a decision on extended warranties by the Board responsible for OMVIC until an APA study was released.

Cost-effectiveness Considerations

Media partnering reaches a large audience with no extra Program expenditure

Partnering with the CTV's W-FIVE has resulted in wide dissemination of study results among consumers with no additional expenditure by the Program.

Commitment to multi-year funding would provide stability for the organization

The fixed annual nature of the application process constrains the way the APA must operate. APA also finds it challenging to identify a change to the core study that makes it sufficiently novel to qualify for support each year. However, staff of the OCA were said to be very accommodating and understanding of the schedules imposed by their media partner.

Ideally the APA would like to see a commitment for multi-year funding to conduct annual assessments of retailing practices. Annual descriptions of how the funds would be spent and annual audits could be retained. But the stability provided through a multi-year research commitment would be valuable to the APA and might reduce the need for different themes for the project for each funding year.

Lessons Learned and Suggestions for Change

Reporting requires an understanding of research limitations

The APA studies identify abuses in the retail market for automobiles—a market where consumers are likely to be at a disadvantage as they participate infrequently, typically have less information than the seller, and risk a sizeable share of their income in a single transaction. This is important information with important lessons for the consumer. And it is highly newsworthy.

But great care should be applied to the release of any information, especially that supported by the Federal Government through initiatives such as the Contributions Program. Like all research, there are limitations, based on the methods used. These need to be fully understood and communicated to avoid misuse of the research findings.

We focus on the 1999/2000 and 2001/2002 Auto Retailing Audits. To be clear, the method used by the APA is an acceptable research approach and the findings presented in the APA reports are appropriate to that method. However, the description of the sample as random¹ and the extrapolation of findings to the population of dealers² as if the sample was random in the W-FIVE presentation of 2000 were not technically correct.

The debate about methods may have overshadowed the importance of the findings of the study for consumers. The statement made by APA in the 2000 W-FIVE segment is supported by the research method: "It is a huge industry. It covers the whole range. But when it is bad it's very bad." We suspect this message may have been lost.

The reporting of the 2001/2002 Auto Retailing Audit by W-FIVE did not go beyond the limits imposed by the study methodology. Representativeness concerns by industry identified related to the 1999/2000 study had been addressed in the later study.

It should be noted that the Program now requires external review, by a suitably qualified external methodologist, of study methods, conclusions and recommendations before project results are made public. The methodologist's comments must be considered in the redraft of the final report. The Program also recommends that a methodologist be retained and consulted throughout the research project. These efforts by the Program are, in part, to deal with earlier deficiencies.

Improper use of research findings is a potential danger in any research. Our data collection activities identified cases where interpretations of the study findings exceeded interpretations that were appropriate based on the study design. This is at the core of the concerns expressed to us by the association of automobile dealers.

To help prevent such misuse in the future, research reports and any release of findings from them should contain a clear statement of the limitations of the study methodology and guide the reader toward proper interpretation of the results.

There are risks in research that may be hard for a non-profit group to bear

In its 2000 audit, the APA made an error in its calculation of leasing costs. Based on this error, APA gave a failing grade to the dealership in its assessment. This was reported in the Toronto Star. The APA and the Toronto Star issued a retraction and an apology within the week for the error made.

The dealer is seeking restitution through the courts. The APA may be held liable in this matter and could face a financial penalty. The APA does not have a fund to cover this potential liability. As a non-profit organization, with a limited budget, any award might adversely affect the group.

In the future the program might consider requiring the organizations to obtain liability insurance coverage and to include the premiums as an allowable expense.

¹ "How do they choose the dealers that they visit? Well it is random selection. They look in the paper, see a deal that looks too good to be true, and go out and see if it is." (Return to reference 1)

² "Final result, a total failure rate of 60%. Put another way you have about a one in three chance of going to a car dealership in this country and being a totally satisfied customer." (Return to reference 2)

Appendix B—Case Studies

Privacy

Introduction and Background

This case study focuses on a number of related research projects conducted by Public Interest Advocacy Centre (PIAC). Funding for these projects was through the Contributions Program for Non-Profit Consumer and Voluntary Organizations of the Office of Consumer Affairs, Industry Canada.

PIAC began in 1976 as a public interest law firm "representing the unrepresented" in a broad range of environmental, social, regulatory, and human rights issues. In the 1990's, faced with cut-backs in support for advocacy by government, PIAC chose to narrow its coverage to areas of particular strengths such as telecommunications, the Information Highway, energy, privacy and competition law while maintaining its focus on low income Canadian consumers. Its work related to regulated industries led to activities in the banking, financial services and electronic commerce areas. The projects, which are the subject of this case study, touch on all of these areas but have the common focus of consumer privacy.

Links to other Support

Total support through the Contribution Program has represented about one-third of PIAC's total budget. The privacy-related projects make up 17% of these contributions or $309,000 in the last six years. Cost awards, when granted convenor status at proceedings such as for the Canadian Radio and Telephone Commission or Ontario Energy Board, represent the principal source of funding for PIAC. Other minor sources of support come through one-off projects for other groups within Industry Canada or other federal departments. Although PIAC explored support through the Volunteer Sector Initiative it abandoned the pursuit "befuddled" by the Initiative's focus on process and not outcome. Most other federal sources of support are felt to be peripheral to the organization's need to support its capacity to make a product.

Project Impacts

The Contributions Program is critical to the advocacy role of PIAC. It allows the group to extend the work they are able to do in an area of competency as well as to build new bases of knowledge that the group is then able to apply elsewhere. A contribution in an area that is totally new is very unusual according to PIAC staff. Instead the group extends its knowledge base through contributions to projects that are reasonable extrapolations of their expertise and experience.

The "Identity Theft" project is a case in point. It built on prior experience by PIAC in e-commerce, security matters, and consumer credit. One of the project's strengths was its timing. Identity theft had just started to be an issue in the popular press and was beginning to be looked at within government circles when the project began. When the study was released the issue had become more central. The report was "at the right time and place", satisfying an eager audience for such information on its release. The study dealt with the legal aspects of the issue and helped inform the consumer protection debate.

Although the Contributions Program is not the largest component of PIAC's overall budget it provides a significant and consistent part of the group's annual revenue. Contributions are spread over a number of areas, with privacy representing only one-fifth of the total support through the Program. These funds provide a core of support for PIAC to maintain its capacity to affect change related to Canadian consumers, particularly low-income consumers.

PIAC relies on press releases and the direct forwarding of research reports to parties contacted as part of the study or who show an interest in receiving the results. However, this ad hoc approach may miss some target audiences. A policy staff member at a provincial information and privacy commission interviewed related to this case study was unable to recall receiving or seeing any of the privacy-related reports.

Staff also note putting completed reports on the PIAC web site beginning last year. PIAC staff send the report to OCA but are uncertain if OCA distributes the reports further. We found two of the six completed reports on the Consumer Connection web site. Content of the PIAC reports also likely find application on Privacytown web pages in Industry Canada's Strategis web site.

The involvement of PIAC, supported through earlier privacy-related research funded under the program, was said to be instrumental in a number of amendments to the Personal Information Protection and Electronic Documents Act (PIPEDA) that was passed in 2001. One area credited to their input was the "reasonable purpose test" whereby: "An organization may collect, use or disclose personal information only for purposes that a reasonable person would consider are appropriate in the circumstances." Without this clause it is unlikely that the legislation would have satisfied a European directive related to the privacy of its citizens in e-commerce transactions. PIACs involvement may be credited with ensuring a place for Canadian companies in e-commerce.

Other impacts identified through our data collection relate to other projects that pre-date those that are the focus of this case study. Most of these impacts, as would likely be true for the privacy projects of this case study, involved some action by government (a task force, regulation or code). This may suggest that there can be a long gestation period before research results produce impacts that are discernable in the marketplace.

Although we identify few impacts through the privacy-related projects of this case study, there are indications that any such impacts may take a longer period to achieve than provided by our case study window. The PIAC studies try to influence the policy debate through credible research. The exact timing from research to promulgation is unknown and the link between research findings and final policy is a tenuous one. Although the lack of concrete evidence of impact at this time may be understandable, concern exists that the limited dissemination of research results may reduce impacts below what would have been possible otherwise.

Cost-Effectiveness Considerations

Achieving cost-effective follow-on activities

Once the research project ends, further follow-on work is unfunded. The "free" research creates expectations among government departments with an interest in the issue. Departments expect PIAC to be involved in further development of the issue (through participation of conferences, development of handbooks or guides, for example) but also appear unwilling to pay for that involvement (possibly as they did not pay for the first involvement).

PIAC finds itself in an untenable position once a research report is released. Any resources that it devotes to furthering the first issue after the project ends must come at the expense of a project budget for a second issue or from some other area of activity. PIAC has to be very selective in terms of its non-funded involvements. Although staff of PIAC may become involved in follow-on activities related to some past projects the involvement is limited by necessity. Constraints on further involvement may limit the positive benefit of the original research.

PIAC would like to conduct follow-on research of some of its past studies. Other departments responsible for the subject matter of the research may encourage PIAC to do so. But these line departments are unwilling to pay, claiming consumer research is not within their mandate. A perception also exists (shared with other organizations) that OCA is unlikely to fund a similar study to follow-up on earlier work. As a result, a lot of potentially valuable research that could capitalize on previous research supported under the Program does not proceed through the lack of a willing funding source. Follow-on activities may be highly cost-effective as they can build on the previous research methodology and base to confirm or help modify previous findings.

Lessons Learned and Suggestions for Change

Timing to maximize responsiveness to emerging issues

Appropriate timing helped guarantee impacts through the "Identity Theft" project. However, how to choose a research topic that will be significant at the release date of a project report is uncertain. Vagaries around the timing and movement of the political process as well as rigidities in the timing of the application process and relatively fixed research length may limit proactive targeting of "hot" topics to happenstance.

Given the knowledge base and expertise that has developed within PIAC it is likely that they would become aware of any emerging privacy-related issue. PIAC uses any such knowledge, its ability to leverage other experience or activities and understanding of what are "saleable" issues to determine the applications for funding it will advance in any one year.

However, the fixed timing of the application process and research schedules which conform to fiscal year funding realities, limit the ability to be responsive to an emerging issue. Potentially OCA could consider a second, more limited, call for applications mid-fiscal year. These would have shorter time frames and focus on emerging issues only.

Distribution and follow-up of reports

PIAC sends completed reports to people who ask for them or who provided input to the report. However, they have no mechanism to identify others who should receive the report. Nor do they have a mechanism to follow-up with those who receive reports to see what if anything has happened as a result. Although these further dissemination and follow-up activities may be valuable, the group has no internal resources to conduct them and no funding mechanism to pay for them. Staff are already doing a number of activities (research, advocacy, legal) making it difficult to take on one more. Potentially these activities might be better conducted if done through some central or umbrella organization given a mandate to disseminate and to assess the impacts of reports produced under the Contributions Program.

PIAC is one of five consumer groups involved in the Canadian Consumer Initiative (CCI), to promote mutual benefit through an over-aching Development Project. Potentially CCI or a similar organization could provide this dissemination/assessment function if given the mandate.

Appendix B—Case Studies

Weights and Measures

Introduction and Background

This case study focuses on a number of weights and measures related research projects conducted by Option consommateurs (OC). All but one study deal with the Trade Sector Reviews (TSR) of Measurement Canada, an agency of Industry Canada. The single study deals with practices by grocery stores involving selective use of metric and non-metric units in their advertisements. This project is recently completed, has yet to yield impacts and is significantly different than the TSR projects. As a result, this case study focuses on the projects related to Measurement Canada's Trade Sector Reviews.

Measurement Canada is responsible for ensuring confidence in the marketplace involving measurement transactions. It evaluates and certifies the accuracy of measuring equipment, investigates complaints received from businesses and consumers who feel they have not received accurate measurement, and certifies measurement standards.

To ensure service coverage and quality it periodically conducts in-depth consultation with stakeholders in the marketplace sectors in which measurement forms the basis of the financial transaction. These reviews involve extensive consultation with stakeholders to obtain their views and ideas on the role of Measurement Canada, manufacturers, service providers, business and consumers on how measurement accuracy is achieved and monitored in the sector. Measurement Canada uses the results of the TSR to implement changes to its programs and services.

Measurement Canada has identified 39 sectors where weights and measures are used to facilitate the transaction. It has listed these in order of importance for review. Key sectors in order of importance were: electricity, water, natural gas, food and retail gasoline. Consumers are significant in all key sectors.

Industry has a vested interest in participation in the TSR processes. However, Measurement Canada also requires the input of consumers. This consumer input was arranged through the Contributions Program for Non-Profit Consumer and Voluntary Organizations of the Office of Consumer Affairs, Industry Canada.

Option consommateurs research studies provided consumer input related to the need and the nature of the need for involvement by Measurement Canada in the five priority sectors. Study methods included a literature review, orientation and review of Measurement Canada's proposals for new regulatory frameworks, review of regulatory frameworks in other jurisdictions, and (subcontracted) surveys and focus groups across Canada to gauge consumers' views.

Links to other Support

Total contributions to support projects related to the review of weights and measures were $209,000 or 19% of the support through the Program for OC over the three-year period. Support by the Program accounted for 49% of revenue by Option consommateurs in the same period.

Option consommateurs received a small increment to their research budget from Measurement Canada to add more focus groups and more questions to the national survey to enhance the methodology. Measurement Canada also paid time and expenses for a representative of OC to attend TSR meetings.

Project Impacts

Meetings for each TSR were held in selected cities across Canada. The results of the OC research, specific to the Trade Sector, were presented at each of these meetings. Other interveners also made presentations. (The Consumers Association of Canada was an intervener at a few meetings.) After presentations were made there were discussions among the parties with the goal to achieve a consensus on the need for and the form of intervention by Measurement Canada in the marketplace.

Research reports prepared by OC on each Trade Sector are available on the OC and Measurement Canada web sites. Copies were also provided to participants in TSR. Given the nature of the work as part of the TSR process and subject matter, OC did not publish findings of the Trade Sector research in its magazine or hold a press conference as would be typical for other research projects by the OC.

Project impacts are of three general types:

• Option consommateurs benefited as the projects enhance their credibility as a national consumer group. They were asked to represent the views of consumers across Canada and their research was focussed on providing that national perspective. Measurement has become an area of expertise for OC. They are the only consumer group represented at the annual Canadian Forum on Trade and Measurement (CFTM). As a result, they are asked to provide the consumer perspective on issues dealt with by the group.
• Measurement Canada benefited, as without the input of consumers, the TSR consensual approach would be undermined. Consumers play a significant role in the five trade sectors researched by OC. If the views of consumers were not presented there could be no consensus achieved. As a secondary benefit, support through the Contribution Program may have reduced tensions around the provision of the consumer input that otherwise may have lessened the chances for consensus. Measurement Canada as regulator might have been in a potential conflict of interest if it also tried to represent the consumer interest. It might have paid directly for consumer research through some consulting arrangement. However it again perceived a potential conflict, as manager of the consulting contract. Even if Measurement Canada had contracted directly with a group such as Option consommateurs to provide this consumer viewpoint it felt conflicted. Why should participation be supported for the residential customer compared to the commercial customer, or the owner of the device, or the seller of the product being transacted with the device?
• Each trade sector also benefited through the OC input. In marketplaces that are dependent on measured transactions, the volume of transitions, in fact, the markets themselves, rely on the integrity of the measurement system. Some level of intervention is needed to ensure that integrity but there are costs to both too little and too much intervention. The input of Option consommateurs helped determine the correct balance. Given the heavy consumer orientation of the five sectors, and the relative strength (vulnerability) of the consumer relative to other market players, the views of consumers were given significant weight. One individual we interviewed as part of the case study felt that the recommendations of OC were followed in 95% of the cases. These recommendations then affect how the sectors operate and the nature of the intervention by Measurement Canada.

Cost-Effectiveness Considerations

Contributions Program may help broker needs for consumer input

Measurement Canada needed input by consumers. Without it, its consensual approach to TSR would have been undermined in sectors where consumers make up a significant share of the sector volume of transactions. However, Measurement Canada was in a real or perceived conflict if it followed approaches to get that consumer input that did not involve the Contributions Program. Measurement Canada also displayed its willingness-to-pay for consumer input when it topped-up the research support and found other funds to pay for the participation of Option consommateurs at its TSR meetings.

We suspect that Measurement Canada would have been willing to cost-share the Research Project Contributions, possibly up to 100% of the research costs. Any such cost sharing effectively increases the annual budget for the Contributions Program and provides more funds to support consumer-oriented research in non-profit consumer groups and voluntary organizations. If Measurement Canada transferred money to the OCA to manage under the Contributions Program it would avoid conflict of interest concerns.

As a result there may be a role for OCA to help broker cost sharing arrangements when a department or group directly benefits from the consumer research. Any cost sharing would add to the Contribution Program funds available for distribution.

A more flexible funding mechanism might be needed to facilitate cost sharing

Measurement Canada found the fixed schedule of the application process an inconvenience. It created some logistic headaches for them and their reviews and a longer planning horizon than may be reasonable. Potentially moving to two application windows for the Contributions Program in a fiscal year might help reduce the lags created by the current application cycle.

Lessons Learned and Suggestions for Change

Partnering may help avoid coverage limitations

The methodology and presentation style of Option consommateurs were given high praise. However, one criticism levelled at OC was the lack of understanding of the regional differences in some Trade Sectors, particularly natural gas. Residential customers have vastly different market shares in different regions of the country. This was not factored in to the analysis presented by OC in its earliest presentations for the Natural Gas Trade Sector but was acknowledged by the Vancouver meeting. This error was said to have diminished the strength of their findings and resulted in some industry groups discounting their input.

Experience might have suggested some weighting of the data to reflect regional differences would overcome this concern. Alternatively, partnering with another consumer organization from another region may have helped point out the marked regional difference and suggested an alternative analysis strategy.

Appendix C—Logic Model and Study Issues

Exhibit 1: Logic Model for Contributions Program for Non-Profit Consumer and Voluntary Organizations

Appendix C—Logic Model and Study Issues

Appendix D—Documents and Literature Reviewed

Documents and Literature Reviewed

Goss Gilroy Inc., Evaluation of the Contributions Program for Non-Profit Consumer and Voluntary Organizations: Final Report, March 30, 2001.

Goss Gilroy Inc., Review Strategy for the Contributions Program: Fourth Draft, March 31, 2004.

Hadfield, R. Howse and M. Trebilcock, Rethinking Consumer Protection Policy, University of Toronto, 1996.

Industry Canada, Office of Consumer Affairs, Consumer Trends Report, June 2004.

Industry Canada, Office of Consumer Affairs, Contributions: A Practical Guide to Project Contribution Management, September 2003.

Industry Canada, Office of Consumer Affairs, Applicant's Guide, Contributions Program for Non-Profit Consumer and Voluntary Organizations, for the Fiscal Year 2004-05 (and 2005-06).

Industry Canada, Office of Consumer Affairs, Results-Based Management and Accountability Framework: Contributions Program for Non-Profit Consumer and Voluntary Organizations, November 22, 2000, Revised November 19,  2002.

Mancur Olson, The Logic of Collective Action: Public Goods and the Theory of Groups, Cambridge, Harvard University Press, 1971.

Appendix E—List Of Interviewees

A. Key Informant Interviews

1. Office of Consumer Affairs

Michael Jenkin, Director General, Office of Consumer Affairs

David Hoye, Director, Consumer Information, and Coordination

Maryanne Murphy, Director, Consumer Policy

Jean-Baptiste Renaud, Senior Analyst

B. Stakeholders

1. Non Profit Consumer and Voluntary Organizations

a) Long time recipients awarded funding in 2004-2005

Automobile Consumer Coalition (ACC)
Executive Director: Mr. Mohamed Bouchama

Consumers' Association of Canada (CAC)
President: Mr. Bruce Cran

Consumers Council of Canada (CCC)
Executive Director: Michael Lio

Union des consommateurs (UC)
Coordonnatrice: Charles Tanguay

b) Past Recipient Organizations

Service d'aide au consommateur (SAC) de Shawinigan
Directrice: Mme Julie Plamondon

The Canadian Toy Testing Council
Executive Director, Mrs. Leigh A. Poirier

Pollution Probe Foundation (Pollution Probe)
Manager, Communications & Senior Project Manager, Mrs. Elizabeth Everhardus

c) No previous financial support.

Infant Feeding Action Initiative (INFACT Canada)
Executive Director, Mrs. Elisabeth Sterken

Coalition des associations de consommateurs du Québec
President, Gilles-André Paquin

d) Other

Canadian Consumer Initiative
Peter Bleyer

2. Academia

Dr. Robert Kerton
Professor of Economics and Dean of the Faculty of Arts
University of Waterloo

Jean-Pierre Beaud, Ph. D
Département de science politique
Université du Québec à Montréal

C. Industry Sector Representatives

Chair, Dr. James Savary
Canadian Motor Vehicle Arbitration Plan (CAMVAP)

Mrs. Amanda Maltby
Canadian Marketing Association

D. Other Government Representatives

Mr. Rob Harper and John Mitsopulos
Ministry of Consumer and Business Services

Mr. Gilles P. Vinet
Vice President, Program Development Directorate
Measurement Canada

E. Case Studies

1. Automobile Retailing

M. George Iny
President
Automobile Protection Association (APA)

Mr. Huw Williams
Executive Director
Canadian Automobile Dealers Association

Marc Migneault
Direction des affaires juridiques et des pratiques commerciales
Office de la protection du consommateur

2. Privacy

Michael Janigan, Executive Director, and John Lawford and Sue Lott
Public Interest Advocacy Centre (PIAC)

Mrs. Stephanie Perrin
President
Digital Discretion Company Inc. | Société discrétion digitale Inc.

Brian Beamish
Director of Policy
Information and Privacy Commissioner

3. Review of Weight and Measures

Madame Geneviève Reed
Responsable
Service de recherche et de représentation
Option consommateurs

Mr. Gilles P. Vinet
Vice President
Program Development Directorate
Measurement Canada

Mike Tarr
MB Tarr & Associates Consulting Ltd.

Appendix F—Interview Guides

Case Study Interview Guide—Funded Organization

We are under contract to Industry Canada to conduct an evaluation of its Contributions Program for Non-Profit Consumer and Voluntary Organizations. Support under the program related to (Name project area) has been selected as the topic of a case study. I would like to ask you some questions about this support and its impacts. Your comments and those of others will be used to prepare a brief case study for the evaluation. When completed you will be given an opportunity to review the case study for completeness and accuracy. May I continue?

Thank you for your time

Case Study Interview Guide — Other

We are under contract to Industry Canada to conduct an evaluation of its Contributions Program for Non-Profit Consumer and Voluntary Organizations. Support under the program to (Name organization) related to (Name project area) has been selected as the topic of a case study. I would like to ask you some questions in support of this case. Your comments and those of others will be used to prepare a brief case study in support of the evaluation. When completed you will be given an opportunity to review the case study for completeness and accuracy. May I continue?

Thank you for your time

Key Informant Interview Guide

We are under contract to Industry Canada to conduct an evaluation of its Contributions Program for Non-Profit Consumer and Voluntary Organizations. I would like to ask your views on some issues in the evaluation. Your comments and those of others will be used in our assessment of these issues. We will not attribute any individual responses to individuals or to organizations in information we provide on the evaluation. May I continue?

Thank you for your time