And the winner of Superbowl XLIV is…Google

The winner of the advertising Superbowl that took place on Sunday, February 2, that is.  This is not just my opinion.  Comments captured from the digital ether by Alterian SM2 give the Sunday night victory to Google’s “Parisian Love” spot that ran at the end of the third quarter (mashable.com has a summary of the results).  Alterian SM2 looked at three measures for each of the 44 advertisers who aired commercials during the 2010 Superbowl:  total mentions, reach, and sentiment.  Google was the leader in mentions by a wide margin (almost 7,000 mentions, compared to 2,100 for the next highest ad–the Tim Tebow ad from Focus on the Family–and an average of  just over 500 mentions for all advertisers).  Google also came out ahead on Alterian’s Social Engagement Index (SEI), which weights the conversations by the popularity of the source.  The SEI for Google’s spot was 1,703 (versus an average of 100 for all ads).  Finally, Alterian weighted the SEI by sentiment to create a second index.   Google came in second on this measure, behind Doritos (SSEI of 673 and 941, respectively, against an average SSEI of 100).  It’s probably worth noting that Doritos ran three different ads during the telecast, against Google’s one spot, and these results do not separate out specific commercials.

Of course, not everyone who has expressed an opinion about the commercials aired during Superbowl XLIV put Google’s ad at the top.  The spot was not, for example, among the “top 10″ Superbowl commercials listed at Fanhouse.  But in it’s way, Google’s ad may be the best example of what advertising is supposed to do.  Google’s dominant position in online search (and the revenues that search advertising generates) is under attack from Microsoft’s Bing, and Microsoft has been running ads showing how easy it is to use Bing to do things like find a dimly lit restaurant (apparently a plus for hungry vamps, if we take a recent ad literally).

 

Social Networking, as seen by The Economist

The current issue of The Economist (January 30 -February 5 2010) features a 15-page special report on social networking.  Typically thorough, the report covers history, the differences between major players (Facebook, Twitter, and MySpace), benefits for small businesses, potential sources of profit for social networking sites, and some of the “peripheral” issues–such as the impact on office productivity and privacy concerns.  For any marketers who’ve been caught by surprise by the emergence of social media and social networking as marketing forces or been watching out of the corner of their eye, this special report might be especially informative.

 

Seeing the Story in the Data

The New York Times is one of the more interesting innovators when it comes to using data visualization to tell a story or make a point.  In particular, the Business section employs a variety of chart forms to reveal what is happening in financial markets.  The Weather Report uses “small multiples” to show 10-day temperature trend for major U.S. Cities.

Even more interesting are the occasional illustrations that appear under the heading of “Op-Chart.”  For a few years now the Times periodically presents on the Op-Ed page a comparative table that tracks “progress” in Iraq on a number of measures such as electric power generation.

Another impressive chart appeared in “Sunday Opinion” on January 10, 2010.  Titled “A Year in Iraq and Afghanistan,” this full page illustration provides a detailed look at the 489 American and allied deaths that occurred in Afghanistan and the 141 deaths in Iraq.  At first glance, the chart resembles the Periodic Table of Elements.  Deaths in Iraq take up the top one-fourth or so of the chart (along with the legend); deaths in Afghanistan occupy the bulk of the illustration.

Each death is represented by a figure, and each figure appears in a box representing the date which the death occurred. One figure shape represents American forces, and a slightly different shape signifies a member of the coalition forces.  For coalition forces, the color of the figure indicates nationality.  A small symbol indicates the cause of each death (homemade bomb, mortar, hostile fire, bomb, suicide bomb, or non-combat related).  Multiple deaths from the same event or cause on a date occupy the same box.

Most dates have only a single death, but a few days standout as particularly tragic:  seven U.S. troops dying due to a non-combat related cause in Afghanistan on October 26; eight killed by hostile fire on October 3rd; seven killed by a homemade bomb on October 27; six Italians killed by a homemade bomb on September 17; five Americans killed by a suicide bomber in Mosul, Iraq, on April 10.

The deaths are linked to specific locations on maps of Iraq and Afghanistan.  Helmand Province was the deadliest place, with 79 of the 489 deaths in Afghanistan.  In Iraq, Baghdad was the most dangerous place, accounting for 42 of the 141 deaths in that country.  While Americans are the largest number, 112 of the dead in Afghanistan were British troops.

There is a wealth of information in this chart with four pieces of information on every death, but in some ways there is too much detail.  To get at the numbers I provided above, I had to manually count the pictures.  There are no summary statistics.  The picture grabs our attention, and immediately conveys the magnitude of the price the U.S. and our allies are paying in Afghanistan.   But if we want to act on data, we need a little more than just a very clever visual display.  Summaries of the numbers would help, here.  It’s useful to know, for example, that 65 of the 141 deaths in Iraq (46%) were due to non-combat related causes, compared to 48 (10%) of the deaths in Afghanistan.  Eighty percent of the fatalities in deadly Helmand province were due to hostile fire; 57% in other parts of Afghanistan were caused by homemade bombs (in Iraq there were 19 deaths, or 13% of the total, from homemade bombs).

Two of the creators of this chart, Adriana Lins de Albuquerque (a doctoral student in political science at Columbia) and Alicia Cheng of mgmt.design, produced a slightly different version of this chart summarizing the death toll in Iraq for 2007 (click here).  That earlier version did not have as much detail about each individual death (location information is not included, for example) but includes some additional causes, like torture and beheading that, thankfully, appear to have disappeared.

The advantage to displaying data in this fashion lies in the ability of our brains to form patterns quickly.  The use of color to designate coalition members makes the contributions of our allies apparent in a way that a simple tally might not.  Even without a year-to-year comparison, we can see that Iraq has become, at least for US troops and our allies, a much safer place than Afghanistan.  Additionally, this one chart presents data that, in other forms, might require several PowerPoint slides to communicate: deaths by date, deaths by city or province, deaths by nationality, causes of death, number killed per incident, and cause of death.

Any complex visual display of data requires making trade-offs.  In this case, for example, the creators arranged the deaths chronologically (oldest first) within each geographic block.  That means that patterns in other variables, such as cause of death or nationality of troops, may be harder to detect on first glance.  The chronological ordering has layout implications, since on some dates there were multiple casualties.

All in all, it’s a great piece of data visualization that to my mind would be even better with the addition of a few summary statistics.

A disclaimer–I counted twice to get each of the numbers I provide above, but I offer no guarantee that I am not off by one or two deaths in any of those numbers.

Copyright 2010 by David G. Bakken.  All rights reserved.

 

D-I-Y Market Research: Worse than no research at all?

I just completed an online survey at the invitation of a company I’ve purchased from in the past.  It was obvious that the survey was an example of what the market research industry calls “D-I-Y” research.  If the quality of the questionnaire had not given this away, there was the “Powered by [name of enterprise feedback software vendor]” at the bottom of the screen.  I was asked to look at two different print ads for one of the products this company sells and answer a few questions that bore some slight resemblance to the questions you might find in an ad test conducted by one of the MR firms that specialize in that type of work.

One can only assume that the results of this survey are meant to drive a decision of which ad to run (there may be other candidates that I didn’t see).  If that’s true, then I think this may be a case where D-I-Y will turn out to be worse than no research at all.  The acid test for any market research is whether or not the decisions made on the basis of that research are “better” than the decision that would have been made without the research.

 

What’s ahead for market research in 2010?

Looking back over the last year in market research offers an opportunity to consider just which transformations, new ideas, industry trends, and emerging techniques might shape MR over the next few years.  Here’s a list of eight topics I’ve been following, with thoughts on the potential impact each might have on MR over the next two or three years.

 

Tweets are to Customer Knowledge as….?

…spontaneous complaints and complements are to customer loyalty management.  Like these forms of customer experience feedback, tweets are unsystematic, unorganized, and representative of who knows what underlying sentiments in the broader universe of individual experiences.

 

Online survey research–how accurate?

The debate over the accuracy–and quality–of survey research conducted online is flaring at the moment, at least partly in response to a paper by Yeager, Krosnick, Chang, Javitz. Levendusky, Simpson and Wang: “Comparing the accuracy of RDD telephone surveys and Internet surveys conducted with probability and non-probability samples.”  Gary Langer, director of polling at ABC News, wrote about the paper in his blog “The Numbers” on September 1. In a nutshell, the paper compares survey results obtained via random-digit dialing (RDD) with those from an Internet panel where panelists were recruited originally by means of RDD and from a number of “opt-in” Internet panels where panelists were “sourced” in a variety of ways.   The results produced by the probability sampling methods are, according to the authors, more accurate than those obtained from the non-probability Internet samples.  You can find a response from Doug Rivers, CEO of YouGov/Polimetrix (and Professor of Political Science at Stanford) at “The Numbers,” as well as some other comments.

The analysis presented in the paper is based on surveys conducted in 2004/5.  In recent years the coverage of the RDD sampling frame has deteriorated as the number of cellphone-only users has increased (to 20% currently).  In response to concerns of several major advertisers about the quality of online panel data, the Advertising Research Foundation (ARF) established an Online Research Quality Council and just this past year conducted new research comparing online panels with RDD telephone samples.  Joel Rubinson, Chief Research Office of The ARF, has summarized some of the key findings in a blog post. According to Rubinson, this study reveals no clear pattern of greater accuracy for the RDD sample.  There are, of course, differences in the two studies, both in purpose and method, but it seems that we can no longer assume that RDD samples represent the best benchmark against which to compare all other samples.

 

Who knows what happiness lurks in the hearts of men? Facebook knows.

Have you heard about the Facebook Gross National Happiness Index?  On Monday, October 12, the Times ran an article (by Noam Cohen) reporting some of the findings based on analysis of two years’ worth of Facebook status updates from 100 million users in the U.S.  The index was created by Adam D. I. Kramer, a doctoral candidate in social psychology at the University of Oregon, and is based on counts of positive and negative words in status updates.  According to the article, classification of words as positive or negative is based on the Linguistic Inquiry and Word Count dictionary.

Among the researchers’ conclusions:  we’re happier on Fridays than on Mondays; holidays also make Americans happy.  The premature death of a celebrity may make us sad.  According to a post by Mr. Kramer on the Facebook blog, the two “saddest” days–days with the highest numbers of negative words–were the days on which actor Heath Ledger and pop icon Michael Jackson died.  Mr. Kramer points out that, coincidentally, Mr. Ledger died on the day of the Asian stock market crash, which might have contributed to the degree of negativity.

We’re going to see a lot more of this kind of thing as researchers delve into the rich trove of information generated by users of search engines and web-enabled social networking.  The happiness index, based as it is on simple frequency analysis of words, is the tip of the iceberg.  At the moment, “social media”–I’m not exactly sure what that label means–is getting incredible attention in the marketing and marketing research community.  The question that has yet to be posed, let alone answered, is, “what exactly do we learn from all this information?”

 

The Challenge of Predicting Consumer Response to Innovation

In the October 4, 2009 edition of The NY Times “Sunday Business” section (“It’s Brand New, but Make It Sound Familiar“), Mary Tripsas, an associate professor at the Harvard Business School, writes about the challenge of finding the right consumer reference points for innovations.  In a nutshell, consumers have a hard time figuring out innovation unless they can compare it to something that is more familiar.  One example offered in the column  comes from Arthur Markham, a professor of psychology at the University of Texas in Austin:  the less than blockbuster introduction of the Segway motorized personal transport device.  In a similar vein, Dan Ariely (Predictably Irrational) argues that comparison is a fundamental process in consumer decision making.

Estimating demand for really new innovations may just be the most difficult endeavor in market research.  A decade ago Robert Veryzer, Jr. identified six factors that make it difficult for consumers to react to innovation (“Key Factors Affecting Customer Evaluation of Discontinuous New Products,” Journal of Product Innovation Management, 1998, 15, 136-150) .  The first factor listed is “lack of familiarity with the product, with the way in which the product is used, or with the underlying technology.”  And one way consumers try to understand a discontinuous product is by comparison with things they already know about.

By and large, I think marketers and market researchers underestimate the fundamental role of comparison and contrast in the way we make judgments about products.  As Professor Tripsas makes clear, humans (consumers included) rely on categorization to understand the world.  Looking at a new, discontinuous product, we’re likely to ask, is it this or that?

 

Recommended reading: The Psychology of Survey Response

The Psychology of Survey Response by Roger Tourangeau, Lance J. Rips, and Kenneth Raskinski (Cambridge University Press, 2000) will change the way you think about the “craft” of survey design.  While there are other, well-regarded books on survey question construction (such as Asking Questions by Norman Bradburn, Seymour Sudman, and Brian Wansink, Jossey-Bass, 2004) and tons of individual research papers and articles on various aspects of survey design, measurement scales, question construction and the like, this is the first book I’ve encountered that presents a practical conceptual framework for understanding the cognitive processes that produce a response to a given question.  Moreover, the authors review a lot of relevant research to support their framework.