6 MRA’s Alert! Magazine – September 2012

MRA’s Alert! Magazine – September 2012 7

Building Effective Online Research Panels:Qualitative Respondents, Quantitative Researchby Laura Livers

You’ve defined your problem, chosen a research design, and are ready to recruit respondents.

8 MRA’s Alert! Magazine – September 2012

With so many panel providers, choosing the right type to recruit from can be a daunting task: What makes a good panel? How are they built? Should I use a qualitative or quantitative panel for a mixed-mode study? How about for online studies?

Even with the most careful planning, a study can fizzle out before it starts because of a low-quality recruitment panel. Effective panels are built using varied techniques that verify and re-verify a respondent’s interest in marketing research, define expectations for participation, and clearly identify incentives. Study results can change considerably depending on the panel used because of the subtle differences in the way they’re built.

Strong panels share common attributes that revolve around a key element: treating respondents with respect. Recently, many of our colleagues in the research industry have called to stop the commoditizing of panels and instead asked for a push to treat respondents like people – not “assets,” “sources,” or “completes.”

When respondents feel that they’re being treated fairly and that their needs are met, they are all the more willing to answer questions honestly, thoroughly, and regularly. Effective panel providers make sure their respondents are responsive and engaged by keeping the focus on these three elements of mutual respect.

Clear CommunicationThe messaging used by a panel provider during the first touch with a potential respondent defines how that respondent will treat their experience on the panel. If the respondent is told to expect to receive up to five research invitations each week, for example, and then receives double that amount, the messaging used to recruit that panelist was deceiving.

“ Because many respondents are constantly ‘wired’ to their handheld devices...panel providers have more opportunities to reach respondents where they are, and therefore more chances to recruit willing participants. When panelists are able to view messaging more often and in all different settings, it is essential that providers keep their messaging fresh, compelling, and concise.”

Fair CompensationIncentives for marketing research respondents run the gamut – some providers award respondents points, while others offer dollars for survey or research study completion. In the end, both structures work, but it’s important that the panel provider answers this question when evaluating each research study and its reward: Has an incentive been assigned that makes completing the research a worthwhile activity?

Appropriate InvitationsWhen a panel provider asks new respondents to complete precise demographic information, the respondent has a reasonable expectation to receive research invitations that closely match their profile. Sending invitations that deviate from information included in a respondent’s profile wastes a lot of time; the panel provider drives up costs for the client because of erroneous invitations, and the respondent is forced to re-answer questions they’ve already completed.

All things considered, when building panels, providers must be careful to ensure the language they use in recruitment materials accurately reflects their panel’s invitation, research study, and incentive procedures to mitigate respondent burn-out. This is all the more important because of the 24/7 nature of panel building today.

Because many respondents are constantly “wired” to their handheld devices (like phones and tablets), panel providers have more opportunities to reach respondents where they are, and therefore more chances to recruit willing participants. When panelists are able to view messaging more often and in

all different settings, it is essential that providers keep their messaging fresh, compelling, and concise.

For panel providers, this takes continuous time and effort; a strong panel is built over a long period of time and using a variety of methods. Qualitative panel providers use a variety of methods to gain new respondent registrations, all of which revolve around an opt-in process. By clearly choosing to enter their personal information, the respondent is agreeing to

participate in the research process.

Qualitative research involves asking broader, more open-ended questions that are hard to quantify; therefore, respondents that join panels built exclusively for this type of research are vetted thoroughly up-front. In this day of rapidly advancing technology, coupled with

people’s desire to be “connected” online, researchers are challenged with keeping up and utilizing these trends to encourage potential respondents to join panels. Qualitative recruitment techniques include:

Sign Up WebsiteRespondents that actively search for marketing research opportunities often sign-up via a recruitment portal or sign-up website.

Targeted, Online AdvertisementsPanel providers sometimes identify demographics that would enhance the depth of their panel and target those selects through online advertisements on search engines, social networks, or websites.

Telephone RecruitmentWhen Web screeners come up short and email addresses have been exhausted, panel providers pull a statistically relevant sample to dial on the phones.

Physical RecruitmentTo recruit respondents that don’t notice or respond to online advertisements, panel providers hit the streets to hand out flyers asking for their participation.

MRA’s Alert! Magazine – September 2012 9

“ Using a qualitative panel for quantitative online research gives the ability to target invitations specifically, based on the respondent’s demographic profile and personal preferences.”

Social Media RecruitmentAs social media continues to gain in popularity, panel providers have the opportunity to build their panels based on word-of-mouth referrals spread through social media giants Facebook, Twitter, Myspace, and Google+, as well as ever-evolving smaller venues.

Unlike the qualitative recruitment methods, entry into a quantitative panel does not always require an explicit opt-in. Many times, respondents are added to these panels as the result of a one-time survey or poll on the assumption that they will participate in future research opportunities.

Because the focus in quantitative research is on numbers, larger numbers of respondents are needed for accurate results. These respondents answer only a few base-line demographic questions before proceeding to the study. Quantitative panel providers utilize several recruitment methods:

Purchased SampleBuying sample is especially popular for very difficult to target and low-incidence respondents, like healthcare and business professionals. Panel providers pay a fee per respondent to access their name, email address, and phone number.

Client ListFor a business with a very specific research goal and established user base, utilizing the client’s existing lists for research purposes allows panels to recruit targeted members at a low-cost.

Third Party SharingSimilar to a client list, third-party respondent sharing is when a panel provider rents, buys, or trades a list of respondent names, email addresses, and phone numbers to invite them to a research study.

Just like building these panels takes repeated effort and more than one method, ensuring that the panel is well managed is a 24/7 job for providers. A managed panel has high response rates, authenticated respondents, and new registrations each day.

For many qualitative panels,

management means added touch points to verify that the respondents are real, viable people. This includes asking them to update their profiles, running reports on low-response profile sections, and regularly calling them on the phone to recruit them to studies. For respondents participating in business and healthcare studies, credentials can be affirmed through searches on search engines and social networks such as LinkedIn.

For quantitative and qualitative panels, management often includes a technical, computer-driven clean-up process. This includes cleaning data,

de-duping profiles, checking for numeric errors, and deleting records based on pre-defined criteria. Whether an outside firm is used for panel management or if it’s handled in-house, keeping a research panel current is an ongoing process and essential to maintaining an active, engaged population of respondents.

With so many elements in play when it comes to what makes a panel strong, deciding which type to use for an online research study (or any research study, for that matter) can seem like a daunting task. Before choosing a panel provider, it’s important the researcher has a clear definition of what needs to be accomplished by the study.

Using a quantitative panel for online research, for instance, is appropriate for very large studies, where a qualitative panel could not provide enough sample due to any number of reasons, including lack of geographic requirements. For global online projects, the sheer number of respondents in the panel offers a competitive advantage.

That said, there are many more advantages to utilizing a qualitative panel for quantitative projects. Using a qualitative panel for quantitative online research gives the ability to target invitations specifically, based on the respondent’s demographic profile and personal preferences. Better initial vetting

means the researcher sees a more accurate initial response.

Respondents that enter the marketing research world through a qualitative panel are used to longer form research, like multi-hour focus groups, with a higher incentive. Because they are compensated well for all of their research participation, qualitative respondents are more likely to answer invitations. Even though they may be compensated differently for an online survey, they are more willing to participate in these studies as part of a “total research experience” over time, which includes in-person and telephone interviews.

Additionally, qualitative panels can be used more effectively for mixed-mode studies because respondents are familiar with different research types and therefore are more versatile. If a quantitative

survey has a qualitative element, for instance, a subset of respondents can easily be pulled from the quantitative study into a qualitative situation, like a telephone interview or focus group.

Similarly, with data collected during the screening phase of an online study using a qualitative panel, the researcher gains access to “exhaust” data, or information entered by respondents that didn’t qualify for the research but may be useful in gaining overall insights.

Whether the panel you choose for online research is qualitative or quantitative, it’s important that the panel provider has a comprehensive panel building and maintenance plan to ensure respondents are not only fully qualified and validated, but also interested in and quick to respond to the invitations they receive.

Though quantitative panels can give researchers access to the hard numbers needed for precise results, the depth of qualitative panels enables them to be highly effective in any research situation.

Laura Livers is the President of Focus Pointe Global. As previous President of Shop’n Chek Worldwide, Laura has the proven ability to run a successful customer-focused organization that delivers high quality services and continued value through effective and efficient systems and practices.