Realism in Research
Innovative Use of 3D Animation for Qualitative & Quantitative Research Methodologies by Michael Richarme, Ph.D. and John Colias, Ph.D.
Overview
T
o a very large degree, people experience and interact with their surroundings from a visual perspective. An often-cited advantage of mail- and Internet-based surveys over telephone surveys is the ability to present information (such as concepts, scales, and question wording) in a visual manner, easing the cognitive burden on respondents and allowing for richer, more meaningful responses.
3D Animation can provide a realistic interpretation of the store experience for the respondent.
Yet all surveys inherently contain a potentially deadly flaw—the act of completing a survey does not mimic the true decisionmaking process utilized by consumers. The use of 3D animation helps to mitigate this limitation of surveys, giving respondents a more realistic decision-making context. This article will demonstrate one 3D animation model and will discuss the incorporation of 3D animation into most choice-modeling research tasks. An empirical study was conducted to compare the use of a traditional grid-based choice-modeling approach to a 3D animation choice-modeling approach. The sample consisted of over 600 nationally representative adults in the United States. The respondents were randomly assigned to one of the two modeling approaches. Analysis of the data shows encouraging support that the use of 3D animation in choice-modeling projects can result in a more accurate estimate of future purchases
Strategic Research Analytics Modeling Optimization
Traditional Internet-based choice- modeling is text oriented, sometimes with pictures for cognitive aid.
1.817.640.6166 or 1.800. ANALYSIS • www.decisionanalyst.com
as compared to traditional text-based choicemodeling methodologies. The respondents tend to be more engaged in the research and tend to report higher levels of satisfaction with the 3D animation approach. Respondent cognitive stress is reduced, and the level of frustration with the repetitive choice-modeling task is reduced. Finally, the use of 3D animation reduces the price elasticities of the stimulus product. The 3D animation lends an increasing amount of realism to the consumer decisionmaking process that is lacking in the traditional text-based, choice-modeling approaches.
Though the 3D animation methodology was utilized in this experiment with a single consumer packaged good category, it is expected that the methodology could be similarly utilized in a wide variety of choice-modeling situations, where store-shelf comparisons are used by consumers to make purchase decisions. The 3D animation lends an increasing amount of realism to the consumer decision-making process that is lacking in the traditional text-based, choice-modeling approaches.
Background The use of realism in advertising has been increasing (Warlaumont, 1998). This is partly due to an increased level of both short-term and long-term brand recall when realistic ads are placed in a consumer’s stream of consciousness (Nelson, 2002). As an empirical finding, this has great practical value to practitioners. Of considerable importance, however, is the need to understand the underlying theory of why the use of reality causes this increased brand recall, and whether or not the use of reality stimulates purchase intent. To do this, it helps to understand how people learn about products. The more information and understanding that people have about products, the more likely the firm is to exhibit long-term profit performance (Wernerfelt, 1996). In terms of presentation format, Jiang and Benbasat (2007) examined four different
presentation formats for online consumers, attempting to determine which of the formats had the most efficiency and salience for building better product understanding. The four formats were presentation of a static picture, presentation of a video without narration, presentation of a video with narration, and a virtual product experience. In this empirical study, they found that the virtual product experience provided consumers with a higher
Decision Analyst
level of product understanding than the other three formats. Berneburg (2007) conducted a laboratory experiment in Germany with a single consumer product to examine whether or not virtual reality could be utilized in a choice-based conjoint (CBC) study. Her conclusions were that the use of virtual reality (3D simulation) did not cause respondents to focus on the cool technique at the cost of enhanced accuracy, and in fact it increased the accuracy of the part-worth estimation and resulting purchase intentions.
Methodology This empirical research utilized both qualitative and quantitative methodologies to examine the efficacy of 3D animation for choicemodeling tasks versus traditional grid-based, choice-modeling tasks. Online depth interviews were conducted for a small number of respondents, in which the respondents were connected to an audio- and text-conferencing service, which allowed the researchers to converse with the respondent via the respondent’s computer monitor being mirrored on the researcher’s desktop. When the survey was launched by the respondent, the researcher was able to talk with the respondent about the survey as it was being conducted. The second methodology was to conduct an Internet survey with random assignment of respondents to two different treatments. The first treatment utilized a traditional gridbased, choice-modeling task, and the second treatment utilized a 3D animation-based, choice-modeling task. The survey and incorporated choice-modeling tasks were exactly identical except for the use of traditional gridbased, or 3D animation-based, choice-modeling formats.
Sample
A nationally representative sample of 600 adults in the United States, aged 18-65, was selected from Decision Analyst’s 7 million member global consumer panel, American Consumer Opinion® Panel. Using Icion®, a proprietary multivariate sampling tool developed by Decision Analyst, population characteristics of age, gender, ethnicity, household income, education, and marital status were simultaneously considered for the sample. Respondents were subsequently screened to be food and beverage shoppers for the household, and had to have purchased the stimulus item within the past three months to qualify for the study. In addition, the respondents had to have purchased the stimulus item most
often at a grocery, drug, mass market, or club store. For the purposes of the choice-modeling task, the respondents were randomly divided into three blocks of eight scenarios, creating an experimental design of 24 total scenarios. Each respondent viewed eight scenarios (eight shopping trips) for the choice-modeling task. In addition, because respondents were each provided a cash incentive of $5 to complete the survey, a security question involving a brief video was utilized to eliminate computer-generated responses and to ensure that the respondent’s computer equipment was capable of handling the survey. At the end of the survey, respondents were asked another quality-control question to ensure that the answers were not computer-generated.
Microwave popcorn was selected as the stimulus because it is common, purchased by all segments of the population, and is sold under few brand names.
The choice-modeling stimulus utilized in the survey was microwave popcorn. This stimulus was selected for a variety of reasons. The researchers wanted a stimulus that was relatively common in Realism in Research
usage, that was purchased by all segments of the population, and that had relatively few numbers of brands from which to choose. A moderate level of stimulus importance was desired to allow for brand switching and a manageable consideration set. The stimulus also had to be available across the entire United States. Using Decision Analyst’s Health and Nutrition Strategist™ (a large scale, longitudinal, searchable syndicated database) microwave popcorn was determined to meet the stimulus criteria. From that database microwave popcorn was found to have been utilized by roughly half of the adult population in the past 30 days, with little difference in gender or income. Families with children had a slightly higher microwave popcorn usage than families without children (56% vs. 47%, respectively), and popcorn with health claims (lower calories, low fat, etc.) tends to have a slightly higher consumption skew toward higher-income households.
Some comments from the qualitative research component relative to 3D animation are as follows: “The graphics actually worked. It was interesting to see things placed side by side so I could visually choose.” “It was like being in a real store with the same choices.” “It actually showed me how I tend to choose my popcorn item. I tend to like the butter flavor but price has a lot of meaning.” “You get to see all the products in front of you as though you were looking at a store shelf. I think having pictures with questions is helpful.” When assessing the traditional methodology, respondents were not favorably impressed with the typical choice-model card. Some comments from the qualitative research component of the project are as follows:
Microwave popcorn had a moderate number of nationally available stock keeping units (SKUs), which allowed for a manageable choice task. A total of 47 SKUs were collected for the choicemodeling task.
“Pics too small and my eyes straining; didn’t want to take so much extra time to click on each one to make them bigger.”
Analysis and Results Qualitative Assessment
“Too many screens of the same information.”
In addition to the depth interviews conducted with several respondents, all survey respondents were asked a series of open-ended questions at the completion of the survey regarding their survey experience. When assessing the 3D animation methodology, respondents indicated that the methodology was not unfamiliar, given the prevalence of animated movies and computer games with this type of technology. In addition, the similarity to their real-world purchase environment provided a better ability to simulate actual decision-making processes.
With online or in-person qualitative depth interviews, it appears that the use of 3D animation triggers better recall of actual purchasing memories, thus leading to more cogent and useful discussion with the moderators about purchase intentions and behaviors.
Question 1: Purchase Volume There was a significant difference between the traditional choice-model technique and the 3D animation choicemodel technique, with the latter providing a reduced
Decision Analyst
stated purchase intention that was much more aligned with resulting purchase behavior. Thus, there is support for the hypothesis that the 3D animation methodology reduces stated purchase volume in survey data. While the absolute accuracy of the reduced purchase volume relative to actual market data is the subject of a future study, the finding of reduced stated purchase volume suggests that the 3D animation choicemodeling approach more accurately predicts actual purchase behavior than does the traditional grid-based choice methodology.
Question 2: Satisfaction with Methodology
The 3D animation methodology was perceived by respondents to be a more realistic, stimulating, unique, and effective way of presenting information. Only when respondents were asked about complexity and level of boredom did the traditional methodology have a statisti-
More Favorable Attitudes to 3D Animation Approach There were two natural groups of attitudes about the survey experience (based on factor analysis): Attitude about the Survey Approach Survey approach was boring
Another element of investigation was whether or not the respondents viewed the 3D methodology as more satisfying than the traditional choice model approach. A between-groups comparison revealed evidence that the 3D choice-modeling methodology was viewed as more interesting, engaging, and stimulating than the traditional choice-modeling methodology. Other than selecting the same brand and selecting the same price, all the other items in this scale were significantly different between the two groups of respondents.
Decrease in Stated Purchase Volume
Survey approach was stimulating I’d like to do this type of survey more often Creative way of presenting information Choices were too complex Survey approach was unique
Attitude about the Choice Task The way I made the choices is similar to the way I make choices in the store
Adequate amount of information for each choice Similar to my current decision-process Helped me explore different choices
4
The survey approach was realistic
Mean Ratings Among Attributes Within Factor
2
5
When asked to decide how many items would be purchased within one month, 3D respondents indicated a significantly lower purchase volume.
Realism in Research
1
Survey Approach
Traditional
2
Choice Scenario
3D
3
Traditional
4
3D
0
3D
1
Traditional
Average Units
Approach helped me form decisions 3
Choice Task
cally significant higher difference. Thus, there is support for this hypothesis that the use of 3D animation in a choice-modeling environment will lead to a more engaged respondent.
Question 3: Mental Labor to Respondents
Question 4: Price Elasticity
Additionally, a cognitive strain task was provided to the respondents immediately upon completing the choicemodeling tasks. The respondents were asked to go through eight iterations of the choice-modeling task. The variables were taken from a validated cognitive stress scale (the NASA Task Load Index). This monadic approach to the evaluation of the methodology allowed for a direct comparison of the two groups of respondents. The traditional choice-modeling task is perceived by the respondents to be more mentally demanding, and the choice task is perceived by the respondents to result in less success in accomplishing the task. In addition, the amount of work and level of frustration are both higher for the traditional approach than the 3D animation approach.
3D Animation Reduced Choice Model Price Sensitivity Act II Movie Theatre Butter (0g Trans Fat) (6 bags in a box)
Product Unit Demand Per 1000 Shoppers
18.2
Tr a d
12.8
itio
nal
3.2
$3.00
3 D A n i m a ti o n $3.50
$4.00
$4.50
$5.00
Price Per Box Simulated demand curve for Act II Movie Theater Butter Popcorn Lower purchase volume Lesser price sensitivity
We tested the hypothesis that price elasticity is reduced when estimated from 3D animation choice data vs. traditional grid-based choice data. We applied a between-subjects statistical test to measure the opposite of our fourth hypothesis—that the averages of price coefficients were equal. We rejected the hypothesis of equality, suggesting that less negative price coefficient (i.e., lower price sensitivity) was significant. (As shown on page 7 graph.)
Conclusions and Managerial Recommendations It is clear from the data that the utilization of 3D animation is preferred over traditional choice-modeling techniques in terms of respondent satisfaction and engagement with the methodology, and in terms of presenting a more realistic survey environment. The data suggests a greater accuracy in measurement of purchase intention; however, comparison to in-market data must be done to be more conclusive. Though still not as realistic as a physical store shelf set, this methodology presents the opportunity to conduct choice-modeling tasks with nationally representative samples and using the full power of part-worth estimation (something that is more difficult, time-consuming, and expensive in physical store shelf-set projects). The research focused on one limited category of a consumer packaged good, popcorn. Replication across a variety of consumer packaged goods would examine the degree of applicability across segments.
9.1 3.9
The four items discussed in this section are significantly different between groups. Thus, there is support for the hypothesis that the cognitive stress level is higher for the traditional choice-modeling methodology relative to the 3D animation choice-modeling methodology.
2.6 $5.50
In addition, this research was conducted with a sample of adults who are representative of the U.S. population. Conducting similar research in other countries with high Internet usage would help to examine whether or not this technique is useful on a global scale.
References 1. Berneburg, Alma (2007), “Interactive 3-D Simulations in Measuring Consumer Preferences: Friend or Foe to Test Results?” Journal of Interactive Advertising, 8:1, 1-37.
Decision Analyst
3D Animation Is Less Cognitively Demanding 10
8
6
3D Traditional
3D Traditional
3D Traditional
Survey process Survey process mentally physically demanding demanding
3D Trad.
3D Trad.
0
3D
2
Traditional
4
Pace of survey process hurried
Successful in accomplishing tasks
Worked hard to complete survey
Frustrated during survey process
2. Jiang, Zhenhui (Jack), and Izak Benbasat (2007), “The Effects of Presentation Formats and Task Complexity on Online Consumers’ Product Understanding,” MIS Quarterly, 31:3, 475-500. 3. Nelson, Michelle R. (2002), “Recall of Brand Placements in Computer/Video Games,” Journal of Advertising Research, 42:2, 80-92.
A study conducted in 2007 showed that the use of virtual reality increases the accuracy of purchase intentions.
4. Warlaumont, Hazel G. (1998), “Realism in Magazine Advertising: Meaning, Prevalence, and Intention,” Journal of Current Issues in Advertising, 20:2, 19-30. 5. Wernerfelt, B. (1996), “Efficient Marketing Communication: Helping the Customer Learn,” Journal of Marketing Research, 33, 239-246.
About the Authors Michael Richarme, Ph.D. is a Senior Vice President at Decision Analyst. (
[email protected]) John Colias, Ph.D. is a Senior Vice President at Decision Analyst. (
[email protected]) Decision Analyst is a leading international marketing research and analytical consulting firm. The company specializes in advertising testing, strategy research, new product ideation, new product research, and advanced modeling for marketing-decision optimization. The authors may be reached at 800.262.5974 or 1.817.640.6166.
Strategic Research Analytics Modeling Optimization
© 2009 Decision Analyst
604 Avenue H East • Arlington, TX 76011-3100, USA 1.817.640.6166 or 1.800. ANALYSIS • www.decisionanalyst.com
Realism in Research
