Transcript Slide 1

Decoding the New Code
How the revised MRIA Code of Conduct affects
your organization and what you need to do to comply
David W. Stark, CIPP
Vice-President &
Standards Portfolio Chair, MRIA
[email protected]
February 2008
Background
In 2007, an MRIA Standards Review Task Force set
out to update the association’s ethical code of
conduct. It was last revised in 2001, which was a
bygone era in some respects, considering the
nascent state of Internet-based research in Canada.
The Task Force conducted two rounds of
consultations with members and received thoughtful
contributions from a total of 16 individual members.
Collective
comments
were
received
from
Community of Practice Leaders in the Government
of Canada, as well as from the MRIA’s QRD, B2B
and RAC Boards.
On November 26, 2007, the MRIA National Board of
Directors unanimously approved the proposed
revisions to the association’s Code of Conduct and
Good Practice.
Purpose
The purpose of this document is twofold:
• to summarize the Code’s substantive
revisions; and
• to explain what MRIA members need to
do to comply with selected clauses in the
revised Code
Members will notice that the Internet research
section has been completely revamped and now
includes several clauses designed to protect
respondent privacy.
No section of the Code was left untouched. A
number of new clauses were added to the general
rules of conduct, responsibilities to the public,
responsibilities
of
researchers
to
clients,
responsibilities of clients to researchers, rules
specific to qualitative research, guidelines for
interviewing children and young people, and the
Code’s disciplinary procedures.
High-level view of changes to the Code
Section A: Introduction
• The Code’s new name is Code of Conduct and
Good Practice. Previously it was Rules of
Conduct and Good Practice.
• Members must uphold the MRIA Charter of
Respondent Rights, a document that was
created in 2006, and which forms part of the
revised Code (Section F, Appendix 2).
• New terms were inserted in the Definitions
section: Focus Group, Moderator, Recruiter.
The definitions used for these terms are based
on those found in the ISO Standard for Market,
Opinion and Social Research (20252).
• A new term, Researcher, was inserted and
given the same definition as Practitioner.
Section A: Introduction
• Children are now defined as under 13
(changed from under 12).
Parental/adult
consent is required when interviewing children,
and recommended when interviewing young
people (13-17), especially when the subject
matter is sensitive. For qualitative research,
the age at which adult consent is required
remains unchanged at under 16.
• Reasons for changing the definition of child
from under 12 to under 13 include:
• Under 12 was low compared to other MR
associations (e.g. ESOMAR = under 14;
Market Research Society = under 16)
• Canadian Marketing Association defines
child as under 13
Section A: Introduction
• Overly long questionnaire is now defined for Internet-based interviews—over 30
minutes—which is the same as for telephone and mall intercept interviews.
• Even though respondents can
complete online surveys at
their convenience, by stopping
interviews to perform some
other task and then resuming
at a later time, the reality is
most respondents prefer to
complete online surveys in one
go.
• When online questionnaires are too long, respondent fatigue may result, which
affects data quality. Also, there may be a negative impact on panel rapport, with
some panelists electing to withdraw from the panel altogether.
Section B: General Rules of Conduct
Ten Core Principles
• The Ten Core Principles, which summarize the
ideas in the Code, were moved to this section
from the Introduction.
Professional Responsibilities
• Clause 9 in this section, Integrity of Reporting,
has been strengthened:
• Members must disclose relevant potential
sources of error, both sampling and nonsampling.
• Members must be cautious and explicit
about the assumptions made about data
accuracy when employing quota or
stratification methods with probability
samples.
• Margins of sampling error must not be
quoted when non-probability samples are
used.
Section B: General Rules of Conduct
Responsibilities to the Public
• Clause 11 makes reference to the MRIA
Charter of Respondent Rights that members
must uphold.
• Clause
13
states
that
overly
long
questionnaires should be avoided at all costs
“except where permission has been obtained in
advance from the respondent and/or special
arrangements have been made.”
• Clauses 14-17 address important points that
were previously buried in a footnote:
• Respondents are entitled to know the
approximate duration of interviews, upon
request, and must always be told when
the questionnaire is overly long.
• Appointments for B2B interviews should
be made in advance.
Section B: General Rules of Conduct
Responsibilities to the Public
• Clauses 14-17 (continued):
• Subsequent interviews with a respondent
in a study should only be carried out
when permission to re-contact was
obtained during the initial interview.
• Special care should be employed when
contacting respondents who initially
refused to participate in a study.
Section B: General Rules of Conduct
Responsibilities to the Public
• Clause 18a now permits sharing a respondent’s
personally-identifiable responses with a client
for customer service purposes, in addition to
research purposes.
In both cases, the
respondent must give permission for their
identifiable data to be disclosed, after having
been informed to whom the information would
be supplied and the purpose for which it would
be used.
• In customer satisfaction studies, customers
may express particular concerns about a
client’s product/service and expect remedial
action quickly.
This is the intent behind
permitting disclosures of respondents’ personal
information for customer service purposes.
Section B: General Rules of Conduct
Responsibilities to the Public
• Clause 18b requires informing respondents
before an interview commences whether
observation techniques or recording equipment
will be used.
Added to this existing
requirement is an obligation for researchers to
erase recordings, where possible (e.g.
individual interviews), if a respondent
withdraws from an interview and requests that
the recording be erased.
• Clause 20 addresses respondent safety when
carrying out research. 20c is a new sub-clause
that requires researchers to comply with an
ASTM Standard whenever they conduct
product evaluations involving the consumption
of alcoholic beverages.
Section B: General Rules of Conduct
Responsibilities of Researchers to Clients
• Clause 26 outlines specific information that
clients are entitled to receive for any market
research project that they commission. These
report details were previously contained in
Notes to the ESOMAR/ICC Code, which was
an appendix to the 2001 version of the MRIA
Code (it is no longer an appendix in the
updated version).
• Clause 27a specifies safeguards that
researchers must have in place when
conducting checks on the quality of fieldwork
and data preparation. These requirements
were previously outlined in the Notes to the
ESOMAR/ICC Code.
Section B: General Rules of Conduct
Responsibilities of Researchers to Clients
• Clause 27b provides clarity on monitoring and
verification requirements. Previously, the Code
required that 10% of each interviewer’s
completed interviews be monitored. There
were divergent practices among members,
however, as to how much of a call should be
monitored, ranging from “the whole call” to “half
the call” to “3 to 4 minutes.” As a result, the
Code now requires monitoring a minimum of
5% of each interviewer’s completed interviews,
and 75% of the whole interview must be
monitored to count towards the 5% monitoring
requirement.
• 27c acknowledges that in exceptional cases, it
may be impossible to carry out monitoring to
the required level, in which case project
records must explain why this is the case and
what other steps have been taken to validate
data collection.
Section B: General Rules of Conduct
Responsibilities of Researchers to Clients
• Clause 29 specifies records that belong to the
client, which the researcher must not disclose
to any third party without the client’s
permission. Client’s property includes:
• Marketing research briefs, specifications,
technical approaches and ideas and
other information provided by the client
• Research data and findings from a
project (except syndicated studies)
• Client-supplied lists – they must not be
used for any other projects or for adding
names to the researcher’s database.
Client-supplied lists should be returned to
the client or destroyed upon project
completion.
Section B: General Rules of Conduct
Responsibilities of Researchers to Clients
• Clause 32 requires the researcher to reperform work that is defective or incomplete.
The researcher is not liable for any business
losses of the client, or for any other indirect,
incidental, special or consequential damages,
whether attributable to defective or incomplete
work or otherwise. The researcher’s maximum
monetary liability in connection with the
performance of the work is to refund any sums
of money the client previously paid to the
researcher for the project out of which the
client’s claim arose.
Section B: General Rules of Conduct
Responsibilities of Clients to Researchers
• Clause 37 addresses publishing research results to a wider audience. If clients
wish to publish findings from a research project, they must consult the research
supplier that conducted the project. The researcher’s agreement on the form and
content of the proposed publication (e.g. a news release) is necessary. If a client
does not consult with the researcher in advance and the client makes misleading
statements about the research and its findings, the researcher has the right to
correct the misleading statements publicly.
Section B: General Rules of Conduct
Responsibilities of Clients to Researchers
• Clause
38
addresses
an
apparent
inconsistency with clause 29 (records that
constitute a client’s intellectual property). Since
non-syndicated research data and analysis
paid for by a client belongs to the client, it is a
bit of a stretch to say that the researcher’s
organization retains exclusive authorship rights,
and yet this is what was stated in the previous
version of the Code. An improved clause 37,
though, makes it clear that the researcher is to
be consulted and acknowledged when findings
are disseminated internally or externally.
Section B: General Rules of Conduct
Responsibilities of Clients to Researchers
• Clause 39 notes that any client who asks
for competitive bids from two or more
practitioners should tell prospective
bidders the number of proposals or cost
estimates that have been requested. If
this information is not given voluntarily,
then clients should provide it to
prospective bidders upon request.
• Also in this clause is a suggestion that no more than four competitive bids or
proposals should be sought on any single project. New text within this clause,
though, acknowledges that these limits do not apply to certain public sector contracts
which may require an open bidding process under NAFTA and other trade
agreements.
Section C: Rules Specific to Qualitative Research
• Throughout this section, references to
analogue media (e.g. audio and video
tapes) were replaced with terms that are
inclusive of both analogue and digital
means of recording interviews (e.g. audio
and video recordings).
• Clauses 1 and 3iv were amended to
permit releasing the identities of
qualitative research participants, with
their permission, to third parties. These
clauses are now consistent with the
general rules for releasing respondentidentifiable data found in Section B
(clause 18a).
Section C: Rules Specific to Qualitative Research
• Clause 7 lists a number of
recruiting specifications that are
assumed to be in operation unless
changes to any or all of the rules
have been discussed and agreed
to by all the parties involved in the
research study. For the sake of
clarity, we added the following text:
“Parties may include, but are not
limited
to,
the
following:
researcher,
moderator,
client,
facility operator, recruiter, subcontractor and respondent.”
• Clause 7f now states that no respondent may be recruited who has attended five or
more focus groups or in-depth interviews in the past five years. This replaced an
extremely difficult standard to meet, which was that no respondents could be recruited
who had attended five or more focus groups ever, unless they had not attended a
group within the past five years.
Section C: Rules Specific to Qualitative Research
• Clause 7g previously required that at
least half of the respondents recruited for
a group/study must never have attended
a group discussion or in-depth interview
before. This is another example of an
extremely difficult standard to meet, so
we relaxed the requirement from one-half
to one-third.
• Clause 7j forbids recruiters from using
advertising to recruit respondents for a
particular project without authorization
from the moderator or client. This clause
has been beefed up by requiring the
moderator or client’s approval of both the
ad copy and the selection of the
medium/media in which the proposed
advertisment would run.
Section C: Rules Specific to Qualitative Research
• Clause 10b is a new statement specifying
that clients and/or third parties who
request recordings must provide written
confirmation that they will use the
recordings only for the intended purpose
of the research and for internal use only.
• Clause 14 places an obligation on facility
operators to maintain client confidentiality, including whenever separate
qualitative sessions are being conducted
for different clients simultaneously or are
closely consecutive. Any computerized
messaging produced by a client must be
erased after the event so as to prevent
access to any of it by a subsequent
client.
Section C: Rules Specific to Qualitative Research
• Clause 18 notes that recruiters are responsible
for ensuring that respondents are qualified for
inclusion in studies.
Recruiters must also
communicate what is expected of respondents
before a study commences.
A number of
examples are listed. We expanded the list to
include whether respondents may be asked to
complete pre-session exercises before the session
and/or upon arrival for the session.
• Clause 18 is also improved by requiring that
respondents are made aware of ingredient lists
and product information if they are being asked to
taste food or beverages. Respondents must sign
a waiver form. For the consumption of alcoholic
beverages,
appropriate
notification
and
arrangements for transportation must be made.
Section C: Rules Specific to Qualitative Research
• Clause 20 references the Qualitative Research Registry (QRR) and notes that
recruiters should provide accurate data to the Registry on a consistent basis and
check all respondents against the QRR. We added an additional statement:
• Any respondents recruited from a third party list, such as a client’s
customer list, should have their names submitted to the QRR unless
requested otherwise, in writing, by the list owner.
Section D: Rules Specific to Internet Research
This section of the Code completely replaces
what very little was said about Internet-based
research in the previous version. As such, we
recommend that that you read Section D
carefully in its entirety. Key highlights include
the following:
• For customer database surveys, the
identity of the client must be revealed.
• Where lists are used for sample selection,
the source of the list must be disclosed.
• Lists must be permission-based
research purposes.
for
Section D: Rules Specific to Internet Research
• The use of unsolicited email for consumer
research is prohibited.
• Unsolicited survey invitation emails may be
sent to B2B respondents if researchers
comply with the anti-spam policies of their
Internet service providers and email service
providers.
• All survey invitation emails—consumer and
B2B—must contain a functioning opt-out link
or mechanism.
• Researchers must use SSL encryption on
panel registration pages and in surveys that
collect sensitive personal information.
• Harvesting email addresses is prohibited.
Section D: Rules Specific to Internet Research
• Message header information must not be
false or misleading. The from labels of email
messages, for example, must not be
spoofed.
• Members should post their privacy policies
on their websites, with hyperlinks to the
policy available on every page.
• The Office of the Privacy Commissioner of
Canada has ruled that workplace email
addresses assigned to employees of
organizations are considered personal
information. As such, our Code states that “a
respondent’s email address is personal
information and must be protected in the
same way as other identifiers.”
• The use of cookies, web log files or software
must be disclosed to respondents.
Section E: Interviewing Children and Young People
Very few changes were made to this section of
the MRIA Code of Conduct:
• For the purposes of the Code, the age of a
child was changed to under 13 (it was under
12 previously).
• The age of a young person was changed to
“aged 13-17” (from “aged 12-17”).
• For qualitative projects, parental or legal
guardian consent remains necessary for
participants under 16.
• We removed “teacher” as someone who
could give consent for a child to be
interviewed. In this day and age, no teacher
or school board would allow researchers into
classrooms to interview children without
parental or legal guardian consent.
Section F: Appendices
Previously, this section consisted of the following
appendices:
•
Notes to the ICC/ESOMAR Code
•
A description of the MRIA’s AdviceLine
service for standards inquiries
•
Disciplinary Procedures
We incorporated pertinent contents from the
Notes into Section B of our Code, such that the
Notes were no longer necessary as an appendix.
All parenthetical references throughout our Code
to broadly equivalent clauses in the ESOMAR
Code were also removed.
Section F: Appendices
Now our Appendices section comprises:
•
AdviceLine service for standards inquiries
•
Charter of Respondent Rights
•
Disciplinary Procedures
•
Example of an Internet survey privacy
statement
We improved the Disciplinary Procedures to
reflect the fact that the MRIA now has full-time
staff. Prior to the 2005 merger of the MRIA’s
three predecessor associations, we did not have
a full-time Executive Director. The Disciplinary
Procedures authorize the Executive Director to
handle most standards complaints. This means
that the Standards Chair and other industry
volunteers are not unnecessarily exposed to
confidential information about their competitors.
Enforcement
Formal complaints about the practices of MRIA
members may be lodged with the MRIA’s Executive
Director and will be handled in accordance with our
Code’s Disciplinary Procedures. These procedures
reflect the commitment of Association members to
industry self-regulation.
As a broad strategic priority initiative, MRIA is
striving—as many members have been strongly
advocating through strategic plan input to the
Association—to be vigilant, proactive, and outspoken
on matters of standards and the image and reputation
of the industry. Thus, your Association is participating
in an international initiative called Project Solon,
spearheaded by ESOMAR and supported by many
marketing research industry Associations around the
world.
The Solon initiative aims to bring about
effective and internationally harmonized self-regulation
in marketing and opinion research.
Enforcement
As a Solon participant, MRIA is being proactive in
communicating the industry’s standards and is
“speaking out” whenever instances of faulty or
misleading reporting on survey research or the industry
appears in the media, through such means as Letters to
the Editor, Op-Ed pieces, Editorial Board visits,
interviews, etc.
One of the precepts behind the Solon initiative is
effective publicity, a must if industry self-regulation is to
gain and maintain public trust and confidence. Publicity
encourages more professional behaviour by everyone,
including non-members of marketing research
associations and journalists.
Related to this initiative, the MRIA is currently
developing a plan to communicate our updated Code to
editors, reporters, journalists and other members of the
media, with a view to elevating the media’s standards
for reporting polls and other types of research.
Complying
with Key New
Clauses in
the MRIA
Code
Margin of Sampling Error
The MRIA’s updated Code of Conduct explicitly states that members must refrain from
making statements about margins of sampling error on population estimates when
probability samples are not used.
In October 2006, the American Association of Public Opinion Research (AAPOR) published
a bulletin entitled, “Opt-in Surveys and Margin of Error.” Here is an excerpt:
When we draw a sample at random—that is, when every member of the target population
has a known probability of being selected—we can use the sample to make projective,
quantitative estimates about the population. A sample selected at random has known
mathematical properties that allow for the computation of sampling error.
Surveys based on self-selected volunteers do not have that sort of known relationship to the
target population and are subject to unknown, non-measurable biases. Even if opt-in
surveys are based on probability samples drawn from very large pools of volunteers, their
results still suffer from unknown biases stemming from the fact that the pool has no
knowable relationships with the full target population.
Margin of Sampling Error
The following is an excerpt from The New York Times polling standards:
Statements about sampling error and statistical significance can be used only if the survey is
based on a probability sample. Probability samples include "random digit dialing" (RDD)
sampling, which is generally used for telephone surveys, and cluster sampling, used in faceto-face interviewing and exit polls.
Non-probability samples are commonly used in Internet polls, call-in polls, intercept polls,
blast e-mail polls, etc. Regardless of what a press release may say, there is no valid margin
of sampling error for surveys based on non-probability samples.
Margin of Sampling Error
When reporting survey results in which the findings were derived from a non-probability
sample, MRIA members must refrain from making statements about margins of sampling
error on population estimates.
AAPOR suggests that firms use the following statement:
Respondents for this survey were selected from among those who have [volunteered to
participate/registered to participate in (company name) online surveys and polls]. The data
(have been/have not been) weighted to reflect the demographic composition of (target
population). Because the sample is based on those who initially self-selected for
participation [in the panel] rather than a probability sample, no estimates of sampling error
can be calculated. All sample surveys and polls may be subject to multiple sources of error,
including, but not limited to sampling error, coverage error, and measurement error.
Margin of Sampling Error
By way of another example, Harris Interactive in the U.S. uses the following boilerplate in its
press releases:
All sample surveys and polls, whether or not they use probability sampling, are subject to
multiple sources of error which are most often not possible to quantify or estimate, including
sampling error, coverage error, error associated with non-response, error associated with
question wording and response options, and post-survey weighting and adjustments.
Therefore, Harris Interactive avoids the words "margin of error" as they are misleading. All
that can be calculated are different possible sampling errors with different probabilities for
pure, unweighted, random samples with 100% response rates. These are only theoretical
because no published polls come close to this ideal.
Encryption
Rules Specific to Internet Research: Encryption
The new Internet research section in the MRIA Code of Conduct requires members to use
encryption when handling respondents’ sensitive personal information. Panel registration
forms typically collect sensitive data (e.g. date of birth, household income, etc.) which must
be safeguarded. Online surveys may also collect sensitive personal information, such as
medical conditions/ailments and prescriptions.
To protect such data from being intercepted over the public Internet, it is essential that
Secure Socket Layer (SSL) encryption or other similar technologies be used. Check
whether your online survey software provider includes SSL encryption as a feature.
Encrypting your panel registration website can be done through third-party certificate
authorities. The cost of encrypting a website is modest, ranging from $100 to $1,000 per
year. Some certificate authorities include: DigiCert, Entrust, Geotrust, Thawte and Verisign.
If your panel registration website is not currently encrypted, chances are that some
respondents planned to join but ultimately decided not to register. Increasingly, consumers
are becoming aware of the need to look for a padlock icon in the lower right-hand corner of
their browser or “https:” in the website address bar (the s means secure).
Interviewing Children
Interviewing Children
The MRIA recognizes that the change in the definition of children from “under 12” to “under
13” has cost implications for companies that recruit children for research projects or access
panels.
As a result, any 12-year old children who, without parental consent, were recruited in 2007
for ongoing research projects are grandparented. Members are not required to obtain
parental/adult consent retroactively. Effective July 1st, 2008, parental/adult consent is
required for all children 12 and under recruited for ad hoc or panel-based research.
Keep in mind that the MRIA Code continues to require adult consent for all qualitative
research participants under the age of 16 so the grandparenting rule above does not apply
for this type of research.
Obtaining parental consent online presents unique challenges given the medium.
Researchers cannot see the parent giving consent or hear an adult voice over the telephone
granting permission. Please consult the Internet research section of the Code for advice on
how to seek parental consent online.
www.mria-arim.ca