Decoding the New Code How the revised MRIA Code of Conduct affects your organization and what you need to do to comply David W. Stark, CIPP Vice-President

Decoding the New Code

How the revised MRIA Code of Conduct affectsyour organization and what you need to do to comply

David W. Stark, CIPP

Vice-President &Standards Portfolio Chair, [email protected]

February 2008

mailto:[email protected]

Background

In 2007, an MRIA Standards Review Task Force set out to update the association’s ethical code of conduct. It was last revised in 2001, which was a bygone era in some respects, considering the nascent state of Internet-based research in Canada.

The Task Force conducted two rounds of consultations with members and received thoughtful contributions from a total of 16 individual members. Collective comments were received from Community of Practice Leaders in the Government of Canada, as well as from the MRIA’s QRD, B2B and RAC Boards.

On November 26, 2007, the MRIA National Board of Directors unanimously approved the proposed revisions to the association’s Code of Conduct and Good Practice.

Purpose

The purpose of this document is twofold:

• to summarize the Code’s substantive revisions; and

• to explain what MRIA members need to do to comply with selected clauses in the revised Code

Members will notice that the Internet research section has been completely revamped and now includes several clauses designed to protect respondent privacy.

No section of the Code was left untouched. A number of new clauses were added to the general rules of conduct, responsibilities to the public, responsibilities of researchers to clients, responsibilities of clients to researchers, rules specific to qualitative research, guidelines for interviewing children and young people, and the Code’s disciplinary procedures.

High-level view of changes to the Code

Section A: Introduction

• The Code’s new name is Code of Conduct and Good Practice. Previously it was Rules of Conduct and Good Practice.

• Members must uphold the MRIA Charter of Respondent Rights, a document that was created in 2006, and which forms part of the revised Code (Section F, Appendix 2).

• New terms were inserted in the Definitions section: Focus Group, Moderator, Recruiter. The definitions used for these terms are based on those found in the ISO Standard for Market, Opinion and Social Research (20252).

• A new term, Researcher, was inserted and given the same definition as Practitioner.

• Children are now defined as under 13 (changed from under 12). Parental/adult consent is required when interviewing children, and recommended when interviewing young people (13-17), especially when the subject matter is sensitive. For qualitative research, the age at which adult consent is required remains unchanged at under 16.

• Reasons for changing the definition of child from under 12 to under 13 include:

• Under 12 was low compared to other MR associations (e.g. ESOMAR = under 14; Market Research Society = under 16)

• Canadian Marketing Association defines child as under 13


• Overly long questionnaire is now defined for Internet-based interviews—over 30 minutes—which is the same as for telephone and mall intercept interviews.


• Even though respondents can complete online surveys at their convenience, by stopping interviews to perform some other task and then resuming at a later time, the reality is most respondents prefer to complete online surveys in one go.

• When online questionnaires are too long, respondent fatigue may result, which affects data quality. Also, there may be a negative impact on panel rapport, with some panelists electing to withdraw from the panel altogether.

Section B: General Rules of Conduct

Ten Core Principles

• The Ten Core Principles, which summarize the ideas in the Code, were moved to this section from the Introduction.

Professional Responsibilities

• Clause 9 in this section, Integrity of Reporting, has been strengthened:

• Members must disclose relevant potential sources of error, both sampling and non-sampling.

• Members must be cautious and explicit about the assumptions made about data accuracy when employing quota or stratification methods with probability samples.

• Margins of sampling error must not be quoted when non-probability samples are used.


Responsibilities to the Public

• Clause 11 makes reference to the MRIA Charter of Respondent Rights that members must uphold.

• Clause 13 states that overly long questionnaires should be avoided at all costs “except where permission has been obtained in advance from the respondent and/or special arrangements have been made.”

• Clauses 14-17 address important points that were previously buried in a footnote:

• Respondents are entitled to know the approximate duration of interviews, upon request, and must always be told when the questionnaire is overly long.

• Appointments for B2B interviews should be made in advance.



• Clauses 14-17 (continued):

• Subsequent interviews with a respondent in a study should only be carried out when permission to re-contact was obtained during the initial interview.

• Special care should be employed when contacting respondents who initially refused to participate in a study.



• Clause 18a now permits sharing a respondent’s personally-identifiable responses with a client for customer service purposes, in addition to research purposes. In both cases, the respondent must give permission for their identifiable data to be disclosed, after having been informed to whom the information would be supplied and the purpose for which it would be used.

• In customer satisfaction studies, customers may express particular concerns about a client’s product/service and expect remedial action quickly. This is the intent behind permitting disclosures of respondents’ personal information for customer service purposes.



• Clause 18b requires informing respondents before an interview commences whether observation techniques or recording equipment will be used. Added to this existing requirement is an obligation for researchers to erase recordings, where possible (e.g. individual interviews), if a respondent withdraws from an interview and requests that the recording be erased.

• Clause 20 addresses respondent safety when carrying out research. 20c is a new sub-clause that requires researchers to comply with an ASTM Standard whenever they conduct product evaluations involving the consumption of alcoholic beverages.


Responsibilities of Researchers to Clients

• Clause 26 outlines specific information that clients are entitled to receive for any market research project that they commission. These report details were previously contained in Notes to the ESOMAR/ICC Code, which was an appendix to the 2001 version of the MRIA Code (it is no longer an appendix in the updated version).

• Clause 27a specifies safeguards that researchers must have in place when conducting checks on the quality of fieldwork and data preparation. These requirements were previously outlined in the Notes to the ESOMAR/ICC Code.



• Clause 27b provides clarity on monitoring and verification requirements. Previously, the Code required that 10% of each interviewer’s completed interviews be monitored. There were divergent practices among members, however, as to how much of a call should be monitored, ranging from “the whole call” to “half the call” to “3 to 4 minutes.” As a result, the Code now requires monitoring a minimum of 5% of each interviewer’s completed interviews, and 75% of the whole interview must be monitored to count towards the 5% monitoring requirement.

• 27c acknowledges that in exceptional cases, it may be impossible to carry out monitoring to the required level, in which case project records must explain why this is the case and what other steps have been taken to validate data collection.



• Clause 29 specifies records that belong to the client, which the researcher must not disclose to any third party without the client’s permission. Client’s property includes:

• Marketing research briefs, specifications, technical approaches and ideas and other information provided by the client

• Research data and findings from a project (except syndicated studies)

• Client-supplied lists – they must not be used for any other projects or for adding names to the researcher’s database. Client-supplied lists should be returned to the client or destroyed upon project completion.



• Clause 32 requires the researcher to re-perform work that is defective or incomplete. The researcher is not liable for any business losses of the client, or for any other indirect, incidental, special or consequential damages, whether attributable to defective or incomplete work or otherwise. The researcher’s maximum monetary liability in connection with the performance of the work is to refund any sums of money the client previously paid to the researcher for the project out of which the client’s claim arose.


Responsibilities of Clients to Researchers

• Clause 37 addresses publishing research results to a wider audience. If clients wish to publish findings from a research project, they must consult the research supplier that conducted the project. The researcher’s agreement on the form and content of the proposed publication (e.g. a news release) is necessary. If a client does not consult with the researcher in advance and the client makes misleading statements about the research and its findings, the researcher has the right to correct the misleading statements publicly.



• Clause 38 addresses an apparent inconsistency with clause 29 (records that constitute a client’s intellectual property). Since non-syndicated research data and analysis paid for by a client belongs to the client, it is a bit of a stretch to say that the researcher’s organization retains exclusive authorship rights, and yet this is what was stated in the previous version of the Code. An improved clause 37, though, makes it clear that the researcher is to be consulted and acknowledged when findings are disseminated internally or externally.



• Clause 39 notes that any client who asks for competitive bids from two or more practitioners should tell prospective bidders the number of proposals or cost estimates that have been requested. If this information is not given voluntarily, then clients should provide it to prospective bidders upon request.

• Also in this clause is a suggestion that no more than four competitive bids or proposals should be sought on any single project. New text within this clause, though, acknowledges that these limits do not apply to certain public sector contracts which may require an open bidding process under NAFTA and other trade agreements.

Section C: Rules Specific to Qualitative Research

• Throughout this section, references to analogue media (e.g. audio and video tapes) were replaced with terms that are inclusive of both analogue and digital means of recording interviews (e.g. audio and video recordings).

• Clauses 1 and 3iv were amended to permit releasing the identities of qualitative research participants, with their permission, to third parties. These clauses are now consistent with the general rules for releasing respondent-identifiable data found in Section B (clause 18a).


• Clause 7 lists a number of recruiting specifications that are assumed to be in operation unless changes to any or all of the rules have been discussed and agreed to by all the parties involved in the research study. For the sake of clarity, we added the following text: “Parties may include, but are not limited to, the following: researcher, moderator, client, facility operator, recruiter, sub-contractor and respondent.”

• Clause 7f now states that no respondent may be recruited who has attended five or more focus groups or in-depth interviews in the past five years. This replaced an extremely difficult standard to meet, which was that no respondents could be recruited who had attended five or more focus groups ever, unless they had not attended a group within the past five years.


• Clause 7g previously required that at least half of the respondents recruited for a group/study must never have attended a group discussion or in-depth interview before. This is another example of an extremely difficult standard to meet, so we relaxed the requirement from one-half to one-third.

• Clause 7j forbids recruiters from using advertising to recruit respondents for a particular project without authorization from the moderator or client. This clause has been beefed up by requiring the moderator or client’s approval of both the ad copy and the selection of the medium/media in which the proposed advertisment would run.


• Clause 10b is a new statement specifying that clients and/or third parties who request recordings must provide written confirmation that they will use the recordings only for the intended purpose of the research and for internal use only.

• Clause 14 places an obligation on facility operators to maintain client confi-dentiality, including whenever separate qualitative sessions are being conducted for different clients simultaneously or are closely consecutive. Any computerized messaging produced by a client must be erased after the event so as to prevent access to any of it by a subsequent client.


• Clause 18 notes that recruiters are responsible for ensuring that respondents are qualified for inclusion in studies. Recruiters must also communicate what is expected of respondents before a study commences. A number of examples are listed. We expanded the list to include whether respondents may be asked to complete pre-session exercises before the session and/or upon arrival for the session.

• Clause 18 is also improved by requiring that respondents are made aware of ingredient lists and product information if they are being asked to taste food or beverages. Respondents must sign a waiver form. For the consumption of alcoholic beverages, appropriate notification and arrangements for transportation must be made.


• Clause 20 references the Qualitative Research Registry (QRR) and notes that recruiters should provide accurate data to the Registry on a consistent basis and check all respondents against the QRR. We added an additional statement:

• Any respondents recruited from a third party list, such as a client’s customer list, should have their names submitted to the QRR unless requested otherwise, in writing, by the list owner.

Section D: Rules Specific to Internet Research

This section of the Code completely replaces what very little was said about Internet-based research in the previous version. As such, we recommend that that you read Section D carefully in its entirety. Key highlights include the following:

• For customer database surveys, the identity of the client must be revealed.

• Where lists are used for sample selection, the source of the list must be disclosed.

• Lists must be permission-based for research purposes.


• The use of unsolicited email for consumer research is prohibited.

• Unsolicited survey invitation emails may be sent to B2B respondents if researchers comply with the anti-spam policies of their Internet service providers and email service providers.

• All survey invitation emails—consumer and B2B—must contain a functioning opt-out link or mechanism.

• Researchers must use SSL encryption on panel registration pages and in surveys that collect sensitive personal information.

• Harvesting email addresses is prohibited.


• Message header information must not be false or misleading. The from labels of email messages, for example, must not be spoofed.

• Members should post their privacy policies on their websites, with hyperlinks to the policy available on every page.

• The Office of the Privacy Commissioner of Canada has ruled that workplace email addresses assigned to employees of organizations are considered personal information. As such, our Code states that “a respondent’s email address is personal information and must be protected in the same way as other identifiers.”

• The use of cookies, web log files or software must be disclosed to respondents.

Section E: Interviewing Children and Young People

Very few changes were made to this section of the MRIA Code of Conduct:

• For the purposes of the Code, the age of a child was changed to under 13 (it was under 12 previously).

• The age of a young person was changed to “aged 13-17” (from “aged 12-17”).

• For qualitative projects, parental or legal guardian consent remains necessary for participants under 16.

• We removed “teacher” as someone who could give consent for a child to be interviewed. In this day and age, no teacher or school board would allow researchers into classrooms to interview children without parental or legal guardian consent.

Section F: Appendices

Previously, this section consisted of the following appendices:

• Notes to the ICC/ESOMAR Code

• A description of the MRIA’s AdviceLine service for standards inquiries

• Disciplinary Procedures

We incorporated pertinent contents from the Notes into Section B of our Code, such that the Notes were no longer necessary as an appendix.

All parenthetical references throughout our Code to broadly equivalent clauses in the ESOMAR Code were also removed.

Section F: Appendices

Now our Appendices section comprises:

• AdviceLine service for standards inquiries

• Charter of Respondent Rights

• Disciplinary Procedures

• Example of an Internet survey privacy statement

We improved the Disciplinary Procedures to reflect the fact that the MRIA now has full-time staff. Prior to the 2005 merger of the MRIA’s three predecessor associations, we did not have a full-time Executive Director. The Disciplinary Procedures authorize the Executive Director to handle most standards complaints. This means that the Standards Chair and other industry volunteers are not unnecessarily exposed to confidential information about their competitors.

Enforcement

Formal complaints about the practices of MRIA members may be lodged with the MRIA’s Executive Director and will be handled in accordance with our Code’s Disciplinary Procedures. These procedures reflect the commitment of Association members to industry self-regulation.

As a broad strategic priority initiative, MRIA is striving—as many members have been strongly advocating through strategic plan input to the Association—to be vigilant, proactive, and outspoken on matters of standards and the image and reputation of the industry. Thus, your Association is participating in an international initiative called Project Solon, spearheaded by ESOMAR and supported by many marketing research industry Associations around the world. The Solon initiative aims to bring about effective and internationally harmonized self-regulation in marketing and opinion research.

Enforcement

As a Solon participant, MRIA is being proactive in communicating the industry’s standards and is “speaking out” whenever instances of faulty or misleading reporting on survey research or the industry appears in the media, through such means as Letters to the Editor, Op-Ed pieces, Editorial Board visits, interviews, etc.

One of the precepts behind the Solon initiative is effective publicity, a must if industry self-regulation is to gain and maintain public trust and confidence. Publicity encourages more professional behaviour by everyone, including non-members of marketing research associations and journalists.

Related to this initiative, the MRIA is currently developing a plan to communicate our updated Code to editors, reporters, journalists and other members of the media, with a view to elevating the media’s standards for reporting polls and other types of research.

Complying

with Key New

Clauses in

the MRIA

Code

Margin of Sampling Error

The MRIA’s updated Code of Conduct explicitly states that members must refrain from making statements about margins of sampling error on population estimates when probability samples are not used.

In October 2006, the American Association of Public Opinion Research (AAPOR) published a bulletin entitled, “Opt-in Surveys and Margin of Error.” Here is an excerpt:

When we draw a sample at random—that is, when every member of the target population has a known probability of being selected—we can use the sample to make projective, quantitative estimates about the population. A sample selected at random has known mathematical properties that allow for the computation of sampling error.

Surveys based on self-selected volunteers do not have that sort of known relationship to the target population and are subject to unknown, non-measurable biases. Even if opt-in surveys are based on probability samples drawn from very large pools of volunteers, their results still suffer from unknown biases stemming from the fact that the pool has no knowable relationships with the full target population.

http://www.aapor.org/whenrespondentsselectthemselves


The following is an excerpt from The New York Times polling standards:

Statements about sampling error and statistical significance can be used only if the survey is based on a probability sample. Probability samples include "random digit dialing" (RDD) sampling, which is generally used for telephone surveys, and cluster sampling, used in face-to-face interviewing and exit polls.

Non-probability samples are commonly used in Internet polls, call-in polls, intercept polls, blast e-mail polls, etc. Regardless of what a press release may say, there is no valid margin of sampling error for surveys based on non-probability samples.

http://publiceditor.blogs.nytimes.com/2006/08/29/the-times-updates-standards-for-use-of-poll-results/?scp=3-b&sq=polling+standards&st=nyt

http://publiceditor.blogs.nytimes.com/2006/08/29/the-times-updates-standards-for-use-of-poll-results/?scp=3-b&sq=polling+standards&st=nyt


When reporting survey results in which the findings were derived from a non-probability sample, MRIA members must refrain from making statements about margins of sampling error on population estimates.

AAPOR suggests that firms use the following statement:

Respondents for this survey were selected from among those who have [volunteered to participate/registered to participate in (company name) online surveys and polls]. The data (have been/have not been) weighted to reflect the demographic composition of (target population). Because the sample is based on those who initially self-selected for participation [in the panel] rather than a probability sample, no estimates of sampling error can be calculated. All sample surveys and polls may be subject to multiple sources of error, including, but not limited to sampling error, coverage error, and measurement error.


By way of another example, Harris Interactive in the U.S. uses the following boilerplate in its press releases:

All sample surveys and polls, whether or not they use probability sampling, are subject to multiple sources of error which are most often not possible to quantify or estimate, including sampling error, coverage error, error associated with non-response, error associated with question wording and response options, and post-survey weighting and adjustments. Therefore, Harris Interactive avoids the words "margin of error" as they are misleading. All that can be calculated are different possible sampling errors with different probabilities for pure, unweighted, random samples with 100% response rates. These are only theoretical because no published polls come close to this ideal.

Encryption

Rules Specific to Internet Research: Encryption

The new Internet research section in the MRIA Code of Conduct requires members to use encryption when handling respondents’ sensitive personal information. Panel registration forms typically collect sensitive data (e.g. date of birth, household income, etc.) which must be safeguarded. Online surveys may also collect sensitive personal information, such as medical conditions/ailments and prescriptions.

To protect such data from being intercepted over the public Internet, it is essential that Secure Socket Layer (SSL) encryption or other similar technologies be used. Check whether your online survey software provider includes SSL encryption as a feature. Encrypting your panel registration website can be done through third-party certificate authorities. The cost of encrypting a website is modest, ranging from $100 to $1,000 per year. Some certificate authorities include: DigiCert, Entrust, Geotrust, Thawte and Verisign.

If your panel registration website is not currently encrypted, chances are that some respondents planned to join but ultimately decided not to register. Increasingly, consumers are becoming aware of the need to look for a padlock icon in the lower right-hand corner of their browser or “https:” in the website address bar (the s means secure).

http://www.digicert.com/ssl-certificate.htm

http://www.entrust.net/

http://www.geotrust.com/

http://www.thawte.com/

http://www.verisign.com/

Interviewing Children

Interviewing Children

The MRIA recognizes that the change in the definition of children from “under 12” to “under 13” has cost implications for companies that recruit children for research projects or access panels.

As a result, any 12-year old children who, without parental consent, were recruited in 2007 for ongoing research projects are grandparented. Members are not required to obtain parental/adult consent retroactively. Effective July 1st, 2008, parental/adult consent is required for all children 12 and under recruited for ad hoc or panel-based research.

Keep in mind that the MRIA Code continues to require adult consent for all qualitative research participants under the age of 16 so the grandparenting rule above does not apply for this type of research.

Obtaining parental consent online presents unique challenges given the medium. Researchers cannot see the parent giving consent or hear an adult voice over the telephone granting permission. Please consult the Internet research section of the Code for advice on how to seek parental consent online.

www.mria-arim.ca

http://www.mria-arim.ca/

Documents

Decoding the New Code How the revised MRIA Code of Conduct affects your organization and what you need to do to comply David W. Stark, CIPP Vice-President