digitalwellbeing.orgHow to thrive in our hyper-connected world

Social Media Metrics – SIM Score vs Net Reputation Score (NRS)

So it’s still January which means we are still allowed to talk about trends for the coming year.  Shiv Singh, VP Social Media at Publicis-Razorfish, author of Social Media Marketing, and along with Jeremiah Owyang one of the smartest brains in this space, has just published his 5 top trends for Social Media Influence Marketing in 2010. Presentation below.

For us, the 5 predicted trends are less interesting than the underlying Social Media Influence Marketing model that Shiv is promoting. (For the record, the trends are 1) Razorfish’s own brand of social media marketing will become big, 2) people using social media will drive sales, 3) we’ll see more of the human faces behind brands, 4) we’ll rely more than ever on word of mouth to inform our decisions, 5) marketers will obsess over measurement).

Razorfish’s answer to 5) is the ‘SIM Score’, a Razorfish jam on the Net Promoter Score (NPS) to create a social media reputational index (as opposed to the NPS customer feedback metric). The SIM Score for a brand is the ‘Net Sentiment’ for that brand in social media divided by ‘Net Sentiment’ for the industry.  Net Sentiment is defined by Razorfish as “Net Sentiment for the Brand = (Positive + Neutral Conversations – Negative Conversations) / Total Conversations for the Brand” (Fluent p25).

First thing to note about the SIM Score is that the dividing going on means that the SIM Score loses some of the elegant simplicity of the NPS (NPS is simply % total mentions that are positive – % count of all other mentions). This, and the fact the the SIM Score is a derivative of another score you have to calculate first (Net Sentiment) means it’s difficult to imagine, for example,  a CMO explaining or using SIM Scores at a board meeting or with analysts without eyes glazing over.  It’s good, but maybe it ‘s not quite intuitive enough.

But complexity in itself isn’t a bad thing – what’s worrying about the SIM Score is that the dividing makes weird things happen.  Basically your SIM Score can jump to infinity, drop hundreds of points with little change in conversational profile, and can plummet from positive to negative even when your conversational profile improves. Illustration follows – but feel free to jump to the conclusion and comparison with the Net Reputation Score if you don’t want to wade through the calculations (Shiv please comment – perhaps we have this wrong?):

To illustrate, let’s follow the actual formula offered by Razorfish for the SIM Score in their Fluent Document introducing and explaining the SIM Score; SIM Score = Net Sentiment (Brand) / Net Sentiment (Industry), where Net Sentiment = (Positive + Neutral Conversations – Negative Conversations) / Total Conversations.

Imagine then, that you have 30 positive conversations, 30 neutral and 40 negative so your Net Sentiment (thus calculated) = 0.2 (30+30-40/100). And imagine the industry has a profile of 60 positive conversations, 60 neutral and 80 negative; Net Sentiment =  0.2 as well (60+60-80/200). So your SIM score = .2/.2 = 1 or 100 if expressed as a percentage:  When your sentiment profile is the same as the industry, your score is 100 – a useful baseline.

Now look what happens when neutral conversations for your brand increase by 10 but overall industry conversations don’t increase – your SIM Score goes up to 136 ((30+40-40/110) /(60+60-80/200) * 100 = .27/.2 * 100 = 136).  That makes sense – so as your profile improves (neutral is positive in Razorfish-land) vs. the industry the higher the score right? Wrong.  Look what happens when your conversations don’t change at all, but all other industry are negative – your SIM score is minus 140! (30+30-40/100) /(30+30-80/140) * 100 = .2/-.14 * 100 = -140).  Now add just 10 neutral conversations for the industry (just as you did for your brand in the first instance), and your SIM Score plummets from -140 to -300.

Something very strange is going on. But if we turn back the the Fluent document (excerpt posted below), we see where we have gone wrong – there seems to be an ambiguity in how to calculate the SIM score. If we follow the ‘walk through’ example rather than the formula offered, we discover that Net Sentiment is not calculated as Positive + Neutral Conversations – Negative Conversations) / Total Conversations for the Brand – but simply  Net Sentiment = Positive + Neutral Conversations – Negative Conversations.  (No dividing go on, just a conversation count).  So in their illustration of GM,  positive + neutral – negative conversations = 83,992, and for the industry = 1,623,779. The SIM Score is 83,992 / 1,623,779 * 100 = .05 * 100 = 5.  So problem solved right? Wrong – this actually makes things worse.

For example, let’s see what happens if we calculate SIM Scores this way; Imagine again, you have 30 positive conversations, 30 neutral conversations and 40 negative conversations – so your Net Sentiment = 20. And that industry has 60 positive conversations, 60 neutral conversations and 120 negative conversations: Net Sentiment = 0. So your SIM Score is 20/0 = infinity. Hmm.

Now imagine that your Net Sentiment remains the same at 20, and that the industry sentiment profile is 60 positive, 60 neutral and 80 negative: Net Sentiment = 40.  So your SIM score is 20/40 * 100 = 50.  So far so good – if your Net Sentiment goes up by 10, so does your score – to 30/40 * 100 = 75.  But now imagine that your original profile doesn’t change, but all other industry conversations are negative – so your profile is still 30, 30, 40 but the industry is 30, 30, 120, so relative to the industry your profile has improved.  But your SIM Score is now 30/-60 * 100 = -50.  You’re doing far better than the industry – in fact anything good anybody has to say about it is about you – but your SIM Score falls to -50.

Enough already. It may be that we are reading the instructions wrong, but whatever is going on here it’s complicated – and rather counter-intuitive.  Surely it’d be easier to stick with the elementary NPS logic and adapt it into a Net Reputation Score (NRS) = % total mentions that are positive – % count of all negative mentions (just as you do for the NPS).  So if all mentions were positive your NRS would be 100, and if all negative it’d be -100.  It’s easy to get, easy to calculate.  You could then compare your NRS to that of competitors or the industry – and add a simple share of voice (proportion of total mentions that are yours) to gauge competitive volume. Along with the ROI, the NRS and Share of Voice provide a simple social media metric trinity for measuring the value of social media marketing.

Net Reputation Score = % total mentions that are positive – % count of all other mentions

(e.g. brand X has 350 positive mentions, 100 neutral mentions, 50 negative mentions so NRS = 350/(350+100+50) – (100+50)/(350+100+50) = 70 – 30 = 40).

In sum, whilst the SIM reputational score may make headway in the social media metric debate (more so than other contested and convoluted engagement/participation measures), our initial reaction is that the SIM Score is just too complicated to communicate and more importantly to rally a business around.  For those who like their metrics simple, try the NRS – Net Reputation Score.

More generally, by creating SIM, Razorfish are potentially complicating social media marketing – the promotion of goods and services using social media – itself.  SIM is defined as employing social media and social influencers to achieve the marketing AND business needs of an organisation. (A social influencer in Razorfish-land is someone who uses social media to influence other people (quite distinct from ‘key’ or ‘peer’ influencers – except presumably when they are using social media to influence)). SIM is therefore a strange concept, since the M stands for marketing but it appears to encompass non-marketing activities – in fact any business use of social media, marketing or not.  So it’s not quite clear whether using social media for market research, human resources, internal communication/collaboration, customer service or innovation falls under the remit of SIMarketing.

The term Social Influence Marketing also invites confusion conceptually: Social influence is an established area of social science, looking at the way people influence each other, through coercion (use of force), persuasion (use of argument) and suggestion (use of example)).  In marketing, social influence is sometimes used as an umbrella term for word of mouth and the copycat effect (i.e. persuasion and suggestion – I wonder why we talk less about coercion ;-) …) .  But for Razorfish, social influence is about influencing using social media.  For all the talk about being people-led not technology-led, social influence marketing is techno-centric marketing in extremis, since it defines itself in terms of the use of a particular technology.

Vanity jargon aside, the real challenge for SIM is how Razorfish is positioning it – as a third pillar of marketing, alongside and distinct from brand marketing and direct response marketing.  According to Shiv, SIM is not brand marketing and not direct response marketing, but something different altogether – and potentially just as important. But why? Social media can be used to create choice-shaping associations in the mind of an audience (brand marketing) or it can be used to stimulate a purchase-associated action (direct-response).  Maybe we’re suffering from a lack of imagination, but what other forms of marketing are there?  We think SIM would be all the more successful if it was folded into existing marketing, rather than standing outside it.

So to sum up, we think ‘Social Media Marketing’ and the ‘Net Reputation Score’ work just fine – we don’t quite get why things need to be complicated with Social Influence Marketing and SIM Scores.

But then, I always liked Occam as a first name.

Written by
Dr Paul Marsden
Leave a Reply to Tweets that mention Social Media Metrics – SIM Score vs Net Reputation Score (NRS) | Social Commerce Today -- Topsy.com Cancel reply

9 comments
digitalwellbeing.org

digitalwellbeing.org

Digital wellbeing covers the latest scientific research on the impact of digital technology on human wellbeing. Curated by psychologist Dr. Paul Marsden (@marsattacks). Sponsored by WPP agency SYZYGY.