Social Media ROI: Advocacy for BMW

April 11, 2011

This is the real “one number you need to grow”.  I’m sure most of you are familiar with Net Promoter Score.  This is the MotiveQuest version of NPS; observed, organic, real.

We measure advocacy, which is the number of people recommending a brand above any others, and have seen strong relationships between the metric and sales in a number of categories including automotive, cellular, CPG, pharma and others.  The underlying concept is similar to NPS – namely that if people are recommending your brand to others, that you will be better off than if they are NOT recommending your brand.

Here is where it is different from NPS.  We are observing naturally occurring recommendations.  There is no suggestion, no survey, no artifice.  People are either recommending your brand to others or they aren’t.  We have tested this extensively (working with a team at Northwestern’s Kellogg School) to validate the correlations between change in advocacy and change in sales or market share.

Example below from the luxury car category showing the correlation between changes in advocacy and changes in share for BMW.

While the scales are different we can see the positive correlation in advocacy and luxury market share for BMW. Looking at the scatter plot and doing the regression confirms the relationship and it’s statistical significance with a p-value of 5% (we’re 95% confident that there is a positive correlation between advocacy and luxury market share). 

Given the complexity of the automotive category and the economic conditions in this time frame it’s amazing that a single factor model can so nicely line up.  With an understanding of incentives to dealers/buyers, advertising spend and supply issues I’m certain the model could be improved but at the end of the day we’re validating that social media does matter. 

Social media clearly has an impact on sales in this category and others.  Many times the relationship will not be this clear or the category may be so complex that a single variable model cannot tease out the results or there could be errors in the metric (source spam/problems, bad language model, etc…)  but if you believe in the 3 tenets, it’s not surprising and quite intuitive that there should be a link between a social media metric and sales.

Of course, to change advocacy you need to know what drives it – but we do that part too!  We have an underlying analytic model to understand in detail what drives advocacy for any category, brand or competitive set.

@tomob

Advertisements

What if Social Media Doesn’t Matter?

April 7, 2011

Guest Post from MotiveQuest CTO Brook Miller (@brookmiller)

As I’ve worked the last 7 years in building metrics and tools to help companies understand the reasons why people do what they do, 3 facts have become evident:

1) Pretty much everyone is online

2) Online communities are reflective of or an enhancement to traditional social relationships

3) People I know are the best source of product recommendations

The digital trails consumer conversations leave online are a good approximation for the world at large.  The latest pew Internet research report show online population reaching  79% of all Americans.  Furthermore, rather than asking people questions about what they think, we can just observe what they say.  We don’t have to ask people if they do or don’t do something we can observe them.

Online communities are as real and vibrant as any other and are built around long term relationships.  Whether it’s a community like Facebook that looks like friendships I’ve developed throughout life or LinkedIn, the connections I’ve made in business, or on a forum like Rennlist, my boss’s favorite Porsche aficionado site, the community is strong and vibrant, with lots of on-going relationships that grow and fade with new people coming into ask questions from experts or lurkers that keep up with the community without contributing or the occasional visitor that just wants to see what the experts think.  These communities are strong, vibrant and have their analogs in social situations in the offline world.

The impact of person to person recommendations is stronger than advertising or other methods of company sponsored communications, when you want to know what car to buy you ask a car nut, when you want to know how to get the best deals on frequent flyer miles you ask a friend with a passion for travel.  Increasingly, though you’d find your friend saying “go look on flyertalk” there’s a thread for that (can I trademark “there’s a thread for that?”).  I was recently browsing flyertalk to figure out international fares for a 2 year old ( I’ve never booked a child fare before and didn’t want to get screwed) while I was there I saw a new miles promo on United it had been posted and had hundreds of comments before it even reached my inbox from United later that day.

Given that people online are pretty much everyone, the communities represent real social relationships and people I know give the most trusted recommendations we should clearly expect to see that a social media metrics measuring peoples’ recommendations to each other would be reflected in sales.

We measure advocacy, which is the number of people recommending a brand above any others, and have seen strong relationships between the metric and sales in a number of categories. (I’ll post more in the future specific to the nuts and bolts of it)

While the scales are different we can see the positive correlation in advocacy and luxury market share for BMW. Looking at the scatter plot and doing the regression confirms the relationship and it’s statistical significance with a p-value of 5% (we’re 95% confident that there is a positive correlation between advocacy and luxury market share).

Given the complexity of the automotive category and the economic conditions in this time frame it’s amazing that a single factor model can so nicely line up.  With an understanding of incentives to dealers/buyers, advertising spend and supply issues I’m certain the model could be improved but at the end of the day we’re validating that social media does matter.

Social media clearly has an impact on sales in this category and others.  Many times the relationship will not be this clear or the category may be so complex that a single variable model cannot tease out the results or there could be errors in the metric (source spam/problems, bad language model, etc…)  but if you believe in the 3 tenets, it’s not surprising and quite intuitive that there should be a link between a social media metric and sales.


Healthcare debate analysis

September 9, 2009

We (MotiveQuest LLC) have decided to start tracking & analyzing the healthcare debate using our online anthropology tools and techniques.  The first report in this series can be found here:  The Raging Debate

For this series we will be monitoring & analyzing the online conversation around healthcare and providing weekly updates as to what is driving the conversation, advocacy for different options and the emotional tenor of the chatter.Healthcare Topics

Here is one chart from the report showing the key topics and drivers of concern – you can see the landscape has changed significantly from June through August.

You can see that rationing shows the biggest increase from June through August.

We will be posting a follow-up report next week which will reflect today’s speech by President Obama.

If you are interested in more information you can contact me at tobrien at motivequest dot com.

Thanks – TO’B


Brand Advocacy Matters

May 19, 2008

Nice article in AdAge about MotiveQuest work with MINI linking brand advocacy to sales.

Linking Web Buzz to Mini Sales

We spent months working with a team at Northwestern University to prove out this measure of brand advocacy – the Online Promoter ScoreTM. OPS is a powerful measure of brand advocacy because when OPS goes up, sales go up 30 days later – and visa versa.

Nice thing about OPS – it is a natural (not surveyed) measure that we can simply observe in online conversations. While it is an online measure – it reflects all of a person’s brand interactions, feelings and experience – online and off. This is an online measure of real-world brand advocacy.

More coming soon.

TO’B


Conversational Marketing Measurement

April 4, 2008

Brian Morrissey over at Conversation Quotient has an excellent post up about conversational marketing measurement.

Social media metrics are still a work in progress

Here’s the money quote:

“But with the rapid growth of social media has come the challenge of measuring the new ways consumers are interacting with and sharing advertisers’ content and brands. In many cases, the biggest difficulty is not just figuring out what to measure but what to ignore — and how to square the need for metrics-driven accountability with the more qualitative feedback endemic to conversation-based channels.”

There is a measure. We have tested it and co-related it to real world results. It’s called Advocacy.

Advocacy is the willingness of people in the online conversation (blogs, forums, newsgroups) to say “you should buy this thing”.

We (MotiveQuest) worked with MINI and their agency BSSP to:

  1. Understand what drives community engagement among MINI owners.
  2. Provide input for a campaign designed to connect to what the MINI community is most excited about.
  3. Measure the results on Advocacy.
  4. Compare changes in Advocacy to changes in MINI sales.

The result?  When Advocacy goes up, sales go up.  When it goes down, sales go down.  I’ll post that case study soon.

TO’B



But can we measure it?

December 12, 2007

An excellent post on web metrics from Avinash K over at Occam’s Razor. True to the title of his blog, one of his key arguments is to strive for a simple, understandable and powerful metric. (Take the simplest explanation that fits the facts!) The post is rather long – but well worth the read.

Of course measurement is a huge deal, and there is a land grab on to define measurement of marketing effectiveness on the web. I think it is unlikely that there will be a single measure that will work for everyone, but we are focused on a measure (Online Promoter Score) grounded in online conversations and tied to real-world results.

Taking a look at Avinash’s criteria, I think we can tick the box on all four.

  • Simple (well, he said uncomplex)
  • Relevant
  • Timely
  • Instantly Useful

I will post more about this soon.

TO’B


Hybrid Sentiment Scoring

December 10, 2007

Matthew Hurst over at Data Mining has an excellent post on sentiment scoring

Sentiment Mining: The Truth

His conclusion? This is not a case where people are better than machines or visa versa. It is a complex problem requiring problem specific methods and approaches.

(His post was triggered by Nathan Gilliat’s Friday roundup of social media news including KD Paine’s announced partnership with BuzzLogic.)

I have to agree with Matthew – in that sentiment scoring is challenging for humans or machines, but done right, machines can successfully score sentiment. MotiveQuest uses high level tools that enable our strategists to build linguistic models for the particular category or question at hand. The model is used to score sentiment in the entire context of the message – not just particular words. This is hard to do well, and our approach is to build sophisticated, parameter driven tools that our strategists can customize to do a great job of automated sentiment scoring in a given context.

We have done side-by side tests with human scoring (Northwestern grad students) and found wide divergence with disagreement among human scorers on ~30% of posts, but when we consider the sentiment of a post to be based on the majority of human scores, we have seen a 96% agreement between automated and human scoring. This kind of side by side testing is required if you are going to rely on results.

Simple automated methods won’t work, and a purely human approach cannot handle the throughput so I propose a new approach – Hybrid Sentiment Scoring.

Discuss . . .

TO’B