Disrupting and Innovating with Respondent Experience, Technology and Data Collection

By Arundati Dandapani, CMRP

Kara Mitchelmore, CEO of the Marketing Research and Intelligence Association (MRIA), opened the annual Net Gain conference on marketing research’s innovators and disruptors in Toronto’s Hotel Novotel, North York this year. Scott Megginson, President of Kantar Millward Brown introduced the opening keynote panel of Paul Neto (Kantar), Joseph Chen (ZappiStore) and Luke Stringer (Facebook) with a vision of bringing together thought leadership from around the world, adding that partnerships were a “key theme” at this conference, stating that, “nobody knows enough about these. As we share more data, we need to know what to do with this data to elevate our industry with meaningful partnerships and new digital technologies.”

KEYNOTE – DEBUNKING THE MYTH ABOUT DIGITAL COPY TESTING: SHOULD I BE WORRIED ABOUT NOT TESTING DIGITAL ADS?

Speakers: Joseph Chen, Vice President/Country Manager – Canada, ZappiStore, Luke Stringer, Brand Measurement and Research Lead, Facebook Canada, Paul Neto, Vice President, Digital and Media – Toronto, Kantar Millward Brown

 “Proxy metrics – likes, shares, retweets – don’t drive the ideal business value and should not be confused with actual measurement.”

– Luke Stringer, Facebook Canada

Yes, you should be worried about not testing digital ads.

Some of the best ads break all the rules. Despite one’s best plans, if ads are not optimized they can be a big monetary drain; 75% of a campaign’s effectiveness is determined by the creative. You can’t afford to not pay attention to creative when the way in which consumers experience it is rapidly changing.

Non-Linear Media Consumption in a Mobile First World

Non-linear media consumption has disrupted otherwise linear worlds. With the rise of mobile-first media and so many digital platforms and channels, there are about 300 million daily active Instagram stories alone. Understanding and studying ad recall is important in a world where audience fragmentation is high and consumers have more choice over what they pay attention to. This causes consumers to quickly flick between devices, feeds and channels during commercial breaks. Facebook validated this theory by testing user/viewer activity during the commercial breaks of a major cable television program.

Luke Stringer of Facebook predicts, “Mobile video is core to the evolution of mobile and there will be a 9 times increase in mobile video by 2020.” Cisco also estimates that video will increase from 50% to 75% of all mobile traffic by 2020. People engage with mobile much differently than other channels—audience engagement and intensity vary across media— and people spend less time on mobile feeds than on desktop devices (1.7s vs. 2.9s). Visual and ephemeral forms are growing (eg. Instagram and Facebook Stories) translating to impact on brand metrics, noted Stringer.

Consumer Behaviour on Mobile

Mobile behaviour is more visual and faster, and provides diverse and different contexts. While it is true that the longer the time people spend with an ad, the more impact it has, it’s even truer that ad recall and purchase intent – i.e., the actual impact of an ad – happens in its first three seconds. So while longer views help, ad-impact happens fast on Facebook. Fast behavior drives brand metrics but also sales. Stringer warns, “don’t be fooled by proxy metrics like views, view-throughs, clicks, and likes that don’t give accurate insights into brand impact; it’s not like they are entirely useless, just that they’re not effective measures of business value.”

Stringer shared that mobile is different from other screens, because there’s more distraction in television viewing versus a more attentive and positive relationship with mobile in intensity and engagement. Around 75% of smartphone users choose to customize their apps on their home screen. Moreover, the same ad assumes two completely different contexts on Facebook versus Instagram and the context of an ad fundamentally drives impact. The challenge is that even though people consume on mobile differently, mobile ads are still being based on best practices and the rules of engagement that TV and print ads have been premised on.

How to Optimize Your Ads for Mobile

Joseph Chen of ZappiStore, explained the methodology for ad copy-testing, you can use Zappi’s insight engine to harness understanding with macro-analysis across all Facebook ads. Zappi considered on brand differentiation, brand breakthrough, brand associations, motivation and decided a mobile-focused study would be the best suited to this purpose.

They created a Zappi-Pro account, selecting twenty Facebook ads across sectors and three verticals (various ad lengths, ad characteristics and varying level of in-market performance), focusing on the media target of each creative. They were able to tag different ads and then passed on all the data to Millward Brown for the analysis.

Paul Neto, of Kantar Millward Brown also shared his insights on copy testing in this mobile-first world. Very little copy testing is happening because of compressed timelines, he said. The optimization in market being done in-field moves from direct response (clicks, views) to building brands and engagement. He warned against putting your investment into risk with poor creative as poor performers not only don’t contribute to ROI but also damage the brand.

Mobile Ad Test Results

Strong branding is a strong indicator to brand lift metrics. With the series of media channels proliferating, one must think of how the mindset/environment of different channels (e.g., mission-oriented viewing on YouTube, more distracted viewing on Facebook) impacts how you deliver the message.

Creative diagnostics have no simple or direct formula. Twenty different elements were analyzed by Neto’s team and the types of ads were very diverse. The four themes across their analyses were: branding, ad constructs, elements of engagement, and messaging techniques.

Here are some of the takeaways:

• Prominent branding happens in the first three seconds, and has 2X lift in ad recall. TV ads often have a ‘reveal’ format – i.e., they tell the story then reveal the product/service. It takes only a few frames to make an impact, so one has stand out at the start. For example: fire has thumb stopping power.
• The absence of sounds forces a different creative approach, enforcing a strong brand presence in every single frame. Strong design elements help make ads stand out. Shorter ads are better suited for mobile newsfeeds, and the design elements all contribute to a strong interface.
• Compelling content leads to higher levels of engagement. Engaging elements in an ad can make up for weak branding. Making people feel something improves the ad lift.
• Slow-motion creates interest; the use of humour, voice-over, and a human presence are important as minds gravitate to the human face in ads.
• If you add sound, it should be added only for delight. In general, sound should be minimized for fast-moving mobile media/channels.
• Engagement is spurred by surprise and affinity.
• None of the elements of design exist in isolation; a bunch of different techniques rather than one formula to good copy is the most useful advice.
• Communication type and timing matter, and understanding that digital is not TV, but a new format, helps.

Successful Mobile Ad Commonality

Finally, what did top ads have in common? Copy test results found that ads with similar creative qualities have higher levels of brand breakthrough. Strong branding and an ad’s ability to show something unique or interesting were top differentiators in ad performance. When building ads for mobile, one must consider a) nailing the first three seconds of attention-grasp b) making ads that work within context (depending on platform/channel) c) creating ads that can stand alone without sound and d) executing ads with crisp messaging. Ad testers need to test out nimble and agile techniques because digital environments are not static. Neto concluded by inviting brands to relook and rethink about their storytelling to incorporate into the different platforms, including the Internet of Things, and what copy or copy testing is going to look like in that world.

“Let’s relook and rethink storytelling to incorporate into the different platforms; we haven’t even started thinking about the Internet of Things, and what copy or copy testing is going to look like in that world.” – Paul Neto, Kantar Millward Brown

HOW CROWD-SOURCING DATA PROVIDES CRITICAL MARKET INSIGHTS NOW

Speaker: Christie Christelis, President, Technology Strategies International Inc.

If you’ve heard of HoodMaps, you’re not a stranger to crowdsourcing.  HoodMaps allows users to classify their neighbourhood into a profile type based on residents’ occupations and socioeconomic status. The cool thing with crowdsourcing data is that it’s a disruptive technology born out of necessity and “zero” budget. But can data quality be relied on 100% with crowdsourcing and what are the privacy measures needed?

Disruptive technologies and ideas are yet to find scale but start as innovative concepts with potential or immediate business value filling a gap in the market, said Christie Christelis, President, Technology Strategies International Inc., who presented a disruptive mobile app called Anagog, which provides geo-location based market analytics by leveraging unique competitive intelligence capabilities from crowdsourced data.

Anagog is a company of location specialists – their mobile application offers a complete profile of the mobility status of the owner of a phone using only 1% of battery power per day. Context awareness geo-fencing technology is at the heart of this software. An AI-engine is embedded in the phone; a lot of encryption is taking place on the phone that gives you routing information and location history. There is no cost to mobile developers for using this tool; in fact revenue is shared with mobile developers.  As a Business Intelligence tool, this app also gives a live demo of adwords.

Anagog could provide users like restaurant owners a comparative analysis of Chinese restaurants with data about client retention, in-store traffic (e.g., number of visitors per month), origin of visitors, etc. It could also give users critical information, like the location history of clients at your restaurant, store, mall or business, offering business intelligence stats and analytics to improve foot traffic, retention or trigger repeat visits.

“The aim is to offer strategic and tactical insights at a low cost, and not to compete with MR applications yet.” – Christie Christelis, Technology Strategies International Inc.

Understanding the preferences of store customers and segmenting customers based on mobility behavior are key functions of this app. The foot traffic of customers and behaviours of an individual are recorded. There are two versions of this application: the closed application and the free and open source one. The aim is to offer “strategic and tactical insights at a low cost”. Christelis reminded the audience that disruptive innovation is something that’s not embracing the sophisticated clients, and with Anagog, the data crowdsourced is not sophisticated. The benefit to this type of technology is that business questions can be solved for a low amount of money over a mass market.

45,000 people in Canada use this Anagog app, and within the next 2-3 years they could get upto the 200-300,000 point, but at the moment it’s not really competing with the MR industry and is not yet an MR app, just a tool that takes the mobility status of users to drive business value in some way, with no representative data/sample.

Christelis noted that just like how today 70% of Canadians have smartphones, and in five years this will become 90%, in the future, the crowdsourced data sample might become representative. But at the moment, this app is not addressing the needs of the Market Research Community. With respect to privacy, there are safeguards and no identifying information is kept on servers. Moreover, as with all other mobile/online services, people will weigh benefits and tradeoff privacy – 35 to 40% people will give up their privacy if there’s a perceived benefit to it– there’s a lot of valuable information available for a platform like this.

CONNECTING A FAST FASHION COMPANY WITH ITS CUSTOMERS – THE CUSTOMER INSIGHTS JOURNEY AT GROUPE DYNAMITE INC.

SPEAKER: MELIZA MATOS, SENIOR MARKET RESEARCH SPECIALIST, GROUPE DYNAMITE INC.

Meliza Matos, a strategist and marketer of research, presented the story of how she implemented an all-new marketing research function in the fashion company Groupe Dynamite from scratch across its two brands, Garage and Dynamite, since August 2015. Matos’s mandate as the new research specialist at Group Dynamite Inc., was to put the voice of the customer first, and help a product-centric company become customer-centric. She began by explaining to employees what research was about and trying to understand what they knew and didn’t know about their customers. To her delight, it became apparent that what was unknown from customers was extensive. However, when interviewing colleagues from different departments about new market research projects or needs, their response was that research was not needed and that they already had ways in place to make business decisions.

Determined to do a diagnosis of the situation, she continued to have intensive meetings with the internal teams, but this time, she was more internally focused and asked them “What are your challenges and projects, how are your process working, and how do we do research now?”

Their existing research function was limited to a focus group in Montreal, along with a basic CSAT survey to the company’s Facebook fans on Survey Monkey with the incentive of 15% off their next purchase. But for an expansive brand with 300 stores throughout Canada and the US, they needed to have more real representation! So Matos built a marketing research model that GDI could rely on to make better business decisions for its current markets at a corporate level as their short to mid-term goal, and they decided to position marketing research as a strategic partner in the company across all tools and markets as their long-term goal.

“Building an insights community is like building a house.” – Meliza Matos, Groupe Dynamite Inc.

The methodology for fast fashion with zero budget would best be served with in-house online insight communities that spurred fast and deep conversations with customers, across a spectrum and mix of behaviours and experiences, cutting across many audiences of interest over time. Building an insight community is like building a house. The business case was to work with different teams and test each insight community for three months with three objectives for each pilot: customer feedback will improve the business, the team will use it, and customers will engage with the brands. The old ways of research wouldn’t open new doors. They received a new budget, and were able to organize pre-launch, launch, and post-launch plans. They proposed a calendar of research, analysis and instruments, and launched the insight communities on a pilot test.  They discovered that their customers were very engaged and wanted to continue in the panel.

It’s been 2+ years now since the Garage Online Insight Community was launched, and one year since the Dynamite Online Insight Community was launched. Both have been successfully inserted in the business process, so much so that the research team is now a team of two and might grow to three members soon. 4,000 customers in the Garage community are directly invited through the email blast. The new strategic goal is to provide deep customer and market understanding that allows a competitive advantage both for the company, and its current and new clientele. Their vision is to master strategic market intelligence and engage customers. They are able to gather top-down and bottom-up support through quarterly calendar meetings, delivering with a service disposition, and summarizing findings though the newsletter to relevant stakeholders. More recently, the owner congratulated Matos and expressed excitement in wanting to leverage these insight communities.

PERCEPTIONS OF ARTIFICIAL INTELLIGENCE AMONG CANADIANS: EITHER YOU’RE WITH IT OR IT’S AGAINST YOU

Speaker: Chasson Gracie, Director of Management Consulting, Sklar Wilton & Associates

Canada is a leader in Artificial Intelligence and prides itself in being so. At MRIA’s 2017 annual Net Gain conference, Chasson Gracie dissected Canadian’s perceptions of AI with the unveiling of Sklar Wilton’s annual AI Tracker, the result of a 15-minute online survey of 1,001 interviews with adult Canadians, in-field from July 31^st to August 7^th 2017, nationwide.

Already one of the leaders, Canada’s vision is to be the top AI innovator beyond technologies like voice recognition. Our country is a vast reservoir of current and future talent, and Gracie emphasized the need to understand how “everyday people” perceive and use AI in Canada, including how they define AI, a term which remains ambiguous to many.

Canadian perceptions of AI revealed the following trends:

1. Canadian Humility: Canadians are humble when it comes to saying they understand AI but, in fact, they are highly knowledgeable of realistic AI applications. The majority of people know what AI means; they know what AI is and isn’t. And, even though only 45% of Canadians say they understand AI, 72% know it can recognize speech, 68% know it can translate languages, and 58% know it can recognize faces.
2. There is a new future coming and it’s coming fast. Six in ten Canadians agreed that “AI will completely change the way we live and work.” Barely under a quarter believed AI would have no impact on their work or lives or future.
3. Emotions matter. Despite the mystery surrounding AI, Canadians are curious and positive. About 71% were curious about AI, 46% saw it as mystery/unknown, and 32% saw it as science fiction.
4. Canadians are curious about AI. Canadians look at AI with much curiosity (71%), followed by positivity (47%), and lastly negativity (40%). There was a healthy level concern towards AI (34%) and only 18% associated AI with “anxiety/fear.” When it comes to negative effects of AI, the top two worries of Canadians were decreasing levels of privacy (75%) and increasing levels of job loss (72%).
5. Canadians are eager for AI. Those who feel they understand AI’s current development (45%) are also more likely to have an immediate positive reaction to AI, see AI as a tool rather than a threat, be excited by the development of new AI tech, and be willing to try out new products and services that use AI technologies.
6. Canadians want a genuine connection. Current AI does not always meet consumer expectations in both hardware and design. To create this ideal AI in the form of a personal assistant, Canadians would want it to have a sense of humour (71%), have a name (68%), have high ethical standards (68%), reflect their values (62%), and refuse a request that could harm them (56%). Canadians want an AI that is intelligent and humanistic. Or more simply put, Canadians want AI to be a typical Canadian!
7. The Consumer Decision Journey (CDJ) will change. The “new consumer decision journey” is being shaped by the world of digital assistants. Marketing will change as people’s demands of their digital “shopping” assistants evolve.
8. The demographic divide is a myth. It’s a myth that women and people aged 55+ are any less interested than young men in AI. In fact, more than half of all Canadians (54%) plan to use AI-enhanced tools in the next five years, and immigrants and visible minorities in particular believe that AI is going to have a broader impact on society as it creates new knowledge, employment opportunities, and improves political and economic decision making by governments.
9. Canadians are rooting for Canada when it comes to AI. Consumers have chosen engagement. Sure, we have questions about privacy and the government’s role, and whether people will lose their jobs. We worry that innovation isn’t transparent and maybe it’s not always truly beneficial for consumers. Despite those concerns, 84% of Canadians support government investment in AI.

When probed about when or how AI could turn against you, in the context of its overall inevitability, Gracie answered, “AI in one form or another is already touching every Canadian’s life even if they’re not aware of it, and it will continue. Sklar Wilton’s study zoned in on Canadian attitudes and perceptions towards AI, which could be used for various segments of society – government, market research, marketing, etc. – to be less afraid of AI and embrace it, with the end goal of building a better society – there are certainly some ‘watch outs’ that we discovered in the study.”

Some watch-outs include the alarming 42% who incorrectly indicated that AI was a self-aware computer program rather than “self-learning algorithms designed to do a specific task.” Gracie added that market research is filled with examples of when AI could turn against you, with “companies that have mapped behavioural paths to purchase using public data, not requiring marketers to conduct custom studies or with ZappiStore moving from automation into AI and creating systems that learn and adapt, or using chatbots to conduct studies or bots to impersonate segments from a segmentation study and questioning if there’s a need for qual research or not in the future!”

Canada could lead the ethical revolution in AI, concluded Gracie, while indicating that understanding Canadians’ perceptions towards AI is crucial to consumers, citizens, businesses, workplaces, and governments wanting to understand how to lead the way or succeed in Canada’s AI revolution.

“AI in one form or another is touching every Canadian’s life, even if they are not aware of it, and it will continue.” – Chasson Gracie, Sklar Wilton & Associates

Follow Chasson Gracie on Twitter @GracieMgt and enter the conversation at #SklarWiltonAI. Thefull research report can be downloaded here.

View Chasson Gracie’s slides here 

THE FUTURE OF FEEDBACK: NEW WAYS OF RESPONDING TO SURVEYS

Speaker: Andrew Grenville, Chief Research Officer, Maru/Matchbox

Andrew Grenville introduced himself as being from the largest research company you never knew about, Maru/Matchbox, the former research division of Vision Critical.  Grenville presented his research on research on the respondent experience, a critical component of research design, sharing important knowledge on how people feel about feedback models below.

As researchers, we’re all in the communications business. We ask questions and get answers, so how we ask questions is important. How we are communicating, who with, what is the medium: our interactions keep evolving, from drums, through smoke signals, the MORSE code, the internet, instant messaging, texting. We need to evolve how we do research too. In tumultuous and disruptive times like these, research has been a consistent laggard. Artificial Intelligence will change not only the way we market but also how we do research. This evolution has been long, and both gradual and fast in relation to time and technological advancement, from face-to-face interviewing, to telephone, online, mobile, and now the brave new world of the internet of things—it is changing the way we work.

Today 30% of all surveys are done on a mobile device. We need to be where people are, in an age where e-mail is already considered “for old people”. The story of tech adoption is a generational story, as different generations embrace different communications methods. Texting is not for everyone – there are differences with age. Willingness to receive text invites to respond to surveys was highest among millennials, for example, in Canada and the US.

“Embrace Multi-modality, because change is coming fast…we are at the slowest point of change and it is only going to get faster”

 – Andrew Grenville, Maru/Matchbox

Brazil’s Ministério da Saúde (i.e. Ministry of Health), in conjunction with Ideia Big Data, conducted a public health research study to combat Zika in Brazil. It was a study with over 1 million people. Photos were geotagged. Through their initial e-mail blast they got 3,000 people to respond. But their most successful feedback collection mechanism was through WhatsApp, a highly popular mobile social network, particularly in the BRIC nations. This allowed respondents in Brazil to upload videos and reached around 750,000 people. The key learning here was to “embrace multi-modality”.

However, when setting up an app-centered community, two things must be known: First, communities are a powerful tool if you can set up a good representative group. Second, the impact of the app on participation must be known. It is tough to be an app today—most people have 27 apps on their phone. The willingness to download any given app and use is low. We could thus set up a community that’s not representative because the respondents were forced to use a certain methodology, warned Andrew.

In Maru/Matchbox’s surveys, it was revealed that respondents preferred texting over voice. Of the 43% who successfully used voice-to-text, 69% preferred typing. There’s no qualitative difference between the type of data offered between voice-to-text and answers that were typed/written (not necessarily texted). 22% of respondents are willing to respond via open-ended video. Video is a good tool but it may also be skewing towards the young and the extroverted. And even so, that’s not a bad thing, so long as it’s acknowledged who it skews towards and what it represents and taken in context.

AI powered qualitative is liked in North America and is already being used widely around the world. Why interrogate if there are virtual assistants (VAs)? Not a majority of people like it, but they are open to it. 52% of people in North America have a virtual assistant either on their phone (think Siri) and/or they have a speaker with a virtual assistant (think Alexa or Google home), 38% have used a virtual assistant, and only 25% would try surveys on it. Among North American millennials, this proportion is higher, with about 73% having VAs, 54% having used VAs, and about 40% being willing to try surveys on it. With virtual assistants, no two people respond the same way.

A lot of training of the AI is something companies have to enable, advised Grenville. Apple is on the cusp of releasing its speaker. Change is coming so fast we all need to be prepared to get information from respondents where they are and how it works for them. The future will be fragmented, and we need to deal with it.

With user experience, Grenville shared that we need to realize long lists do not work on mobile devices as rank questions, and this changes how people answer. We need device agnostic design across all devices, or else we introduce noise, and these are high stakes. If we keep doing trackers the way we’ve been doing all this while, we will lose quality and depth in data. We need to adapt quickly to the times or we’ll fall away into oblivion and irrelevance. We are at the slowest point of change and it’s only going to get faster.

View Andrew Grenville’s slides here

THE LEGAL AND REGULATORY ENVIRONMENT FOR RESEARCH IN CANADA—WHAT YOU NEED TO KNOW

Speaker: Greg Jodouin, President, PACE Public Affairs & Community Engagement, MRIA Government Relations Consultant

If you have had sleepless nights worrying about advocacy and whether you’re on the right side of the law as it pertains to market research, Greg Jodouin had the perfect pill for you in the form of his presentation about the laws and regulations governing marketing research practice today and what the MRIA is doing to protect the industry. Greg’s job as a lawyer lobbyist–as opposed to your personal counsel–is to ensure that federal regulations create a positive environment for our marketing research industry in Canada.

Why does government care so much? Mostly because it wants interactions with Canadians to be legitimate and not a nuisance; i.e., not deceptive marketing, aggressive sales, or solicitationmasked as research. The government is not so concerned with Marketing and Survey Researchers (MSR) as it is with consumer and respondent rights, privacy protection and good government. The MRIA’s key messages to the government include:

• societal benefit (to build a strong social use case for MR),
• that marketing research never solicits (our ability to distinguish ourselves from those who solicit and fundraise),
• effective job self-regulation and when government can prove we don’t do this well, it steps in,
• being able to tell the government that its intervention in MR is not justifiable,
• creating jobs and boosting the Canadian economy, and
• assessing how a certain policy or proposal may impact jobs.

MRIA’s milestones in advocacy and outcomes achieved so far have included the following:

1. Mugging and sugging (deceptive telemarketing) is now illegal
2. Marketing and Survey Researchers don’t have to comply with the Do-Not-Call (DNC) rules as the law now excludes surveys from its application. As such, researchers can call even those numbers that are on the national DNC registry
3. Clarification from the government that CASL does not apply to online research
4. Restoration of the long form census in 2015
5. Closure of a Corrections Canada program that had inmates in prison doing surveys

Key federal laws and regulations related to marketing research include the Criminal Code and the privacy act, called PIPEDA. The Telecommunications Act, which is enforced by the CRTC, applies to all calls, and establishes the Do Not Call List and the Unsolicited Telecommunications Rules. There’s also the federal anti-spam act, commonly known as CASL, the Canada Elections Act and theCompetition Act. There was some discussion around whether incentives could be seen by the government as a solicitation and whether they could trigger the laws and regulations that govern marketing, specifically CASL. Greg clarified that incentives offered on call were no longer a gray area in that MRIA was successful in convincing the CRTC that incentives offered for the sole purposeof collecting information from the public was not a marketing activity. Incentives offered electronically, however, are still a grey area although MRIA believes that the “sole purpose” test that applies to incentives offered on a call should also apply to incentives offered by electronic means (email, text, etc.). This was an exciting refresher of standards and an important session especially for those who have been so engaged with other pillars of member value and engagement, that advocacy is just a given.

View Greg Jodouin’s slides here   

ELEVATING THE INSIGHTS TEAM: MARRYING CHOICE MODELS WITH BUSINESS DATA

Speakers: Steve Olsen, Consumer Insights Consultant, McDonald’s Canada, Cheryl Hung, Director, Dig Insights

The supplier and client team of Cheryl Hung from Dig Insights and Steve Olsen from McDonald’s Canada presented a fascinating study of how McDonald’s used a combination of real-world sales data and data from simulated menu ordering to identify pricing and menu mix opportunities. They created four different simulators (reflecting different channels and dayparts). Across those simulators, virtual menu orders from 9,500 respondents were modeled to project the effect of changes to pricing and menu mix on the business. This approach proved more powerful than previously-used TURF modeling.

So what were the key drivers of business on the McDonald’s menu? And what was a fact-based way to arrive at what items to eliminate or re-cost, and why, in order to maximize revenues? With around 250 menu items, price change didn’t happen in isolation, but led to a series of contextual implications; i.e., every menu item had a relationship with every other menu item.

Hung’s team needed a robust quantitative methodology, reflective of real choices people make, because in commerce, all choices are tradeoffs. They used discrete choice modelling and an observed behavioural approach, observing how people ordered, allowing respondents to react to the thousands of possible iterations of the menu in which item cost, meal upcharge cost, and menu mix changed.

Respondents were able to upgrade to side salads, breakfast or lunch. Rules of engagement included keeping globally protected menu items. The research team prioritized respondent experience; users could customize specialty drinks to their liking in the main menu, and when they clicked on the meal bag they could add or subtract burgers.

Behind the scenes, all the respondent data was being churned using genetic algorithms where each run evolved from the previous analysis. This approach explored thousands of possible menu scenarios, identifying the scenarios that had the strongest positive revenue implications.

At the end of the study, the client’s business questions were answered. They were able to maximize revenue and answer ad-hoc questions like what would happen if they removed a cup size or an item, and were able to leverage granular data for precise projections.

How did the model hold up, in terms of projections vs. sales volumes? When they benchmarked their model with sales results from several promotional launches, the projected volume was accurate to 98% of the observed values.

Olsen’s team excitedly approached the menu simulator, and used it to help with implementation of important business initiatives like the launch of All Day Breakfast Selections and McTasters.

They also used the menu simulator to understand the pricing relationship between different menu items. Specifically, they used to tool as a hypothesis generator to identify scenarios to run on in-market tests.

Understanding the impact of launching something new and removing a few items from the menu was key to their recommendations. Maybe consumers liked a certain item a lot, but they substituted it for another. It was important to find products consumers liked that did not cannibalize another’s sales. The incremental likes and changes allowed for a more inclusive, revenue-effective menu.

The supplier’s key learnings were that research tools give an opportunity to push strategic conversations earlier on, driving forward custom solutions. For the client, customer education was considered important and must be slipped into promotional awareness at the right time. Olsen also emphasized the importance of strategic partnership, understanding the operations and sales supply chain, and maintaining brand synergy.

From a marketing research traditional viewpoint, Dig Insights was very happy that they were able to scale their menu simulator to a version 2.0 and answer ongoing questions, using this concept and model iteratively.

“With around 250 menu items, you can’t just use the cookie-cutter approach.” — Cheryl Hung, Dig Insights

APPLICATION OF TECHNOLOGY: INNOVATIVE APPROACHES TO DATA COLLECTION

Speaker: Steve Mast, President and Chief Innovation Officer, Delvinia

Delvinia’s Steve Mast was introduced by Julianne Ng of element54, which recently launched an ad-tracking product – AdTrack Express – on Methodify, Delvinia’s research automation platform.

A former video game designer and developer, Mast leads the innovation team at the firm.  His presentation shared how brands are applying technologies in innovative ways to tap into their customers and solve today’s business challenges. He also shared ways organizations can apply these technologies for market research purposes, including executing surveys through voice and in chat form.

Real-time feedback

A self-confessed heavy metal fan, Mast kicked off the presentation by recalling the first time he heard about a feedback collection system to test real time engagement among heavy metal fans. The system used cameras to test engagement with heavy metal music. The longer fans head-banged to a band, the higher their engagement. Mast noted that implicit and real-time feedback collection is becoming increasingly pivotal in marketing research.

Conversational data collection

Conversational data collection and the rise of screenless experiences are other notable trends that have led to virtual assistants like Amazon’s Alexa (launched officially in Canada this December) becoming an important facet of global households. Virtual assistance is nothing new, it has shifted the norm in participant engagement. Dominoes (you can substitute this for virtually nearly any other company today), for example, is no longer just a pizza company, it’s now a tech company. Its application DOM is IVR on steroids and scale; it is not only learning based on transactions of pizzas but is capturing your voice and can detect your mood, giving you a discount to cheer you up on a bad day!

Voice user interfaces

Twenty per cent of searches in the Google Android App are now being done by voice, according to a Google Canada study conducted in July 2017. Mast’s team have been running some pilots with Alexa and have programmed a few surveys using the interface.

Augmented reality

Apple and Google are rapidly moving into the augmented reality space as virtual reality continues to be cost-prohibitive. IKEA’s “Place” app is a great example of augmented reality and immersive data collection allowing users to shift and move objects/products to design and create their dream homes/living rooms/kitchens/offices/studios etc. to scale. Users can also capture photos through their app and buy the products online through the store’s website. Embedding interactions inside 3D environments is more engaging to users/respondents.

Wearables

Wearable technology is another innovation to watch out for. The Apple watch haptic suit is an example of a successful tech wearable with the core principle of enabling a motion-sensing feedback system through virtual reality, allowing you to “feel” and “touch” (or send a virtual reality “heart” to) the recipient from a distance. Wearable tech is going to be big for athletic companies and will help to measure athletes’ performance levels. As Mast said, “It’s all about embrace the machine or die.”

Home robots

Home robots like Kuri retail for approximately $300 and can listen to your commands and perform basic and even sophisticated functions (e.g., tell a bedtime story, navigate chairs, talk in robo language, pick things up etc.) at home. However, rolling robots that substitute humans are still a long way off.

Artificial Intelligence

The more immediate opportunity and advancement is with AI, Mast said. Instant concept testing—new ad or product concepts—is one of the crucial functions of AI. Nowadays data collection is not simply about asking questions. It’s about implicit testing/implicit behavior. For example, Delvinia recently used a mixed mode chat-bot survey with the millennial market and got a 45% respondent rate, exceeding the industry average of 20% among millennials (no matter what the mode), indicating success in engaging the toughest segment.

“Embrace the Machine or Die.” —Steve Mast, Delvinia

View Steve Mast’s slides here

NEXT GENERATION IDEA SCREENING

Speaker: Frank Beirne, Vice President of Technology and Data Science, Dig Insights

“If you’ve not heard of Tinder, you’re lying,” said Frank Beirne, VP of Dig Insights, as he took the stage to explain some of the mobile research innovations his company is leading.

It has been said that every year since the iPhone launch will be the “year of mobile”, but Beirne believes that it is only now that the market research industry is starting to see the impact. Mobile tech has evolved too quickly for most of us to keep up; in 2005, the PalmPilot and Blackberry were style icons, and in 2007, the iPhone was a game-changer that optimized the mobile ecosystem, at a time when zooming in was a novelty. In the short time since, we’ve evolved to better screens, higher pixel densities, and we now have supercomputers in our pockets! Apple and Google have made great strides in enforcing design standards for a more consistent user experience.

Beirne stated that bad design is the leading reason for user dissatisfaction, and that taking inspiration from emerging technologies and design standards is the key to developing methodologies that are both smart and user-friendly. He shared examples of Dig’s mobile innovation using discrete choice interfaces which are made to look like real airline booking sites; payment diary apps that utilize APIs, like Google Maps, for a familiar user experience; and survey chatbots with neural networks for brains.

With all the recent advances in mobile technology, he said, “it’s Tinder that has had the single biggest impact on how we interact with our devices – yes, even if you don’t use Tinder.” Dig Insights has built on this learned behaviour with their new concept screening app, Upsiide, which allows users to swipe to indicate preferences, and pitches similar concepts against each other in a tournament-style trade-off exercise, like a MaxDiff. While the data collected is simple and easy for users to provide, it allows for sophisticated modelling.

Dig Insights has built their Upsiide application on web, iOS, and Android, and has integrated it with other popular apps as well. Upsiide’s modern dashboard allows clients to log in a view results in real-time, most notably the proprietary Upsiide Score, which “is the secret sauce of Upsiide. It combines the likes, tradeoffs, and a little bit of Bayesian modelling.” Dig’s rigorous testing has proven the Upsiide Score is highly correlated (0.8) with in-market performance. Additional analytics are also updated automatically in the dashboard, including network mapping and TURF (made possible by a super-fast TURF algorithm developed by Dig Insights).

“Many of our clients, and their ideas, have already made Upsiide their concept testing home,” said Beirne. CPG concepts, TV shows, menu items, and even text-only claims have all found success of the platform. Most recently, licensed marijuana producers are looking to Upsiide to test strain ideas and

“Girl Scout Cookies”[*] currently tests highest on their flavours wishlist.

In 2018, Dig Insights plans to push Upsiide even further through integration of additional geo-location and push notification features. Beirne concluded, “It’s an exciting time. We have more tech available to us than ever before. We just have to want it, learn it, and build with it. We now have to forget what we know about online research and rethink how we’re interacting with consumers.”

View Frank Beirne’s slides here

VIRTUAL REALITY WITH EYE TRACKING: STEP INTO THE FUTURE

Speaker: Anne Stephenson, Founding Partner, Explorer Research

Anne Stephenson, a partner at Explorer Research revealed some trends in Virtual Reality, handing out chocolates to those who knew a thing or two about stepping into the future! Take a stab at some tech-trivia:

• How many VR headsets are there projected to be by 2020? 200 million
• When was the iPhone launched? 2007
• How many employees does Amazon have? 97,000
• What % of millennials prefer to shop online? 67%
• When was the first time you could deposit your cheque with your phone? 2013

And the pace of change will continue to be rapid. Stephenson’s presentation discussed how Virtual Reality combined with eye tracking could be leveraged for powerful behavioral insights in the retail environment. Inundated with choice, and about 60,000 items in a regular grocery store, the average shopper walks 2 miles on a stock-up trip and still only buys 20-30 items.

With a human shopper’s vision as wide as 120 degrees, and our ability to focus on just 2% of all that we are catching in a glance, much of what is viewed in-store is observed through peripheral vision. For any given retail environment, about 65-85% of the shelf gets ignored. And what shoppers say they do doesn’t always reflect their actual behaviour.  Shopping is not linear; shoppers use shortcuts, they rely on System 1 (intuitive impulses) or brand bias and social norms, and their decisions are quick. It is thus important to understand consumers’ triggers and barriers, to determine their influence behavior.

There is usually disparity between what shoppers say and how they buy; for example, the long and logical decision tree for toilet paper with the following steps of: 1) price 2) roll count 3) ply level 4) brand 5) softness 6) strength 7) environmental factors —is all nullified by System 1 forces in real time.

“Context is everything. Being in the actual environment triggers a physiological reaction.” – Anne Stephenson, Explorer Research

Context is everything, Stephenson explained, where being in the actual environment triggers a physiological reaction. In order to measure behavior, therefore, you need to test in the situation. The following are particularly immersive and impactful environments: in-store, Virtual Reality 3D, in-lab staging, in-lab 2D, and Online 2D. All allow for deep and impactful insights when combined with eye tracking.

The benefits of Virtual Reality with eye tracking is that it measures behavior with life-size immersion. Virtual Reality with eye tracking can be used to measure the broader context of a total physical location, such as a store or bank branch. Virtual Reality with eye tracking also allows the flexibility to test in any market or to customize to any retailer environment.

VR with eye tracking helps you understand optimal brand placement based on scan patterns and visual fixations. Virtual Reality helps companies innovate through iterative testing, quick turnaround, testing nudges (or theories), and dynamic content vs. static images to conceive and evolve the aisle of the future.

Explorer research undertook an aisle reinvention project using virtual reality. The framework included aisle flow, section adjacencies, space allocation, and innovation placement. They used 150 in-lab interviews. A few critical factors played into the aisle renovation, including: scan patterns or detecting how people scan the aisles, stopping and closing power to identify the right aisles to place products on, and findability times or the time it takes to find a product on the aisle (and this varies a lot).  Innovation placement can thus be measured by detecting where innovations were performing most strongly on the aisle.

To zero in on the recommended aisle, VR with eye tracking allowed Stephenson’s research team to test different parameters to optimize the aisle, delivering 5% growth in-market. Despite so much information and output power, VR is still in its infancy and has its challenges. It’s costly from an initial set-up standpoint but then becomes very cost effective if the assets are being used for different studies. For some categories where tactile interaction with products is very important there is still a controller that limits this interaction.

However, the top five positives for using virtual reality with eye tracking in your research include: iterative testing (nudges); life-size testing to measure real behavior; contextualizing the decision process, customizing to different environments; ease of testing in different markets, measuring what is seen; and leveraging real-time emotions and triggers.

Closing thoughts

Mark Wood, Chair of MRIA’s National Board of Directors, closed the conference by thanking all its sponsors at what he described was his favourite conference about innovation and optimizing data collection and expertise across the breadth of our industry.

This year’s Net Gain Conference was another milestone showcase of the challenges and opportunities in designing respondent-centric research today and the brimming potential to partner across technologies, sectors and talents, as we collect, share and harness data for a richer, meaningful future.

Arundati Dandapani, CMRP, MLITT, has worked for leading global media and research brands including most recently with the Rotman School of Management at U of T, and is also on Twitter @itadnura .