The worst best practices in measurement.

Great post from Gary Angel over at the WAO Factor: 

Here’s a few simple, oft-repeated best-practices in digital measurement dashboarding that you may already know:

• Focus on a small set of site-wide actionable KPIs
• Measure your overall Site Satisfaction and compare it to your industry
• Trend your NetPromoter scores to track your efficiency in creating brand advocates

What you probably don’t know is that every one of these “best-practices” is wrong. Deeply, fundamentally, and completely wrong.

More here

Smart thinking on the issues around marketing ROI

Liked this article from McKinsey on measuring the worth of marketing very much. From the article: 

We’ve found three things that are always true in managing complexity within the marketing organization. First, you’ll require a number of specialists. You just will. You can’t get the skills and knowledge you need in just one person, and you’re not likely to get everything you need internally. Second, you’ll need somebody who both integrates marketing efforts across channels and communications vehicles and focuses on the bottom line. In packaged-goods companies, this was—and may still be—the role of brand managers, but the basic requirement is that it must be done by someone. Finally, you’ll need absolute clarity in processes, roles, and responsibilities not only within the marketing organization but also throughout your company (across functions and business units) and externally (with agencies and external vendors). The trust-based relationship between companies and agencies isn’t at risk, but everyone will have to accept that roles are changing. 

SOCIAL DATA QUALITY: SILENTLY HARMFUL

I am trying to figure out how to raise awareness for the biggest issue in social data analysis: data quality. 

My team and I do a lot of social data analysis, with a range of platforms. And the sad truth is that the data varies widely between those platforms. We should not be paying to visualize or manage CRM programs for underlying data that is spotty, unreliable and old. I'm not at liberty to discuss our results in detail, but the differences appear in the following areas, in order of how critical they are -   

1. Volume - The number of articles, Tweets, Facebook shares, forum posts, etc. We routinely see some platforms return several times more valid results than others. Every provider's idea of what blogs are is different, but the poor level of overlap between providers data in what should be consistent channels - like Facebook or Twitter- for the same search is disturbing, to say the least. 
2. Recency - How fast does the platform capture data. In a world where social analytics platforms are routinely being asked to support same-day or even real-time decisionmaking, lag times of more than 15 minutes or so can be a real problem - but we routinely see refresh rates for major blogs at multiple times daily
3. Spam - How much spam content is in the dataset. It's unfortunate that we are so good at filtering spam email, but providers have had little success at programatically filtering spam social media content.
4. Channel breadth and depth - A constant and growing problem. Everyone's ideas of what blogs are is different; the problem is even greater for hard-to-crawl forums.  

My suggested solution is twofold: standards for publicly available API's for most social data, and the marketplaces (I'm looking at you, Gnip and Datasift) to facilitate consolidated data search and retrieval should be publicly tested against the data from common platforms. For those of us who are serious, a Gnip or Datasift data certification is much more valuable than guessing about what the platform may or may not have. 

As an industry, until we solve this problem, our credibility is at risk. 

Salesforce acquires Radian6 - social CRM gets serious

This morning I woke to the startling news that Salesforce has acquired social media monitoring tool Radian6. This acquisition makes a lot of sense; Salesforce has demonstrated with Chatter that they believe in the importance of doing social customer relationship management, but data aggregation is a critical step in making social CRM work and few solutions can currently claim truly robust data across social channels as an input to a social CRM system.

Some background on the alphabet soup; Social CRM is the intersection between lead management and social media; think sourcing and tracking sales leads and engaged customers in the same system that can capture and uncover revenue associated with each of them. This allows you to forecast critical metrics like lifetime customer value associated with individual leads within social media; once customer lifetime value predictions are available, further optimization may become possible.

In plain language: you could filter and prioritize conversations with your brand on twitter or other social channels by their predicted resulting direct revenue. This is likely most valuable for b2b clients but has potential implications across a wide variety of companies.

How deep is the integration going to be? The release indicates that radian will remain separate, but deep integration between the engagement console and Salesforce would be a compelling offering; I expect to quickly see API hooks that will allow operations within salesforce from the social media monitoring console - the first step in an efficient cross platform workflow.

Twitter Tools With no Clothes – Unmasking Endemic Twitter Data Weakness

At Mashable's panel "Measuring social media – lets get serious," I had the opportunity to question Kevin Weil, Twitter's product lead for revenue, on an issue concerning Twitter's data that has been bothering me for a long time. Christina Warren, the Mashable writer who moderated the panel, went on to write a post about the questions I put to Kevin, here.

First, a bit of background. There are two ways to get data from Twitter: via the application program interface or API, or via the "Firehose" of all tweets (or some percentage thereof). The API is free and easy to access for anyone with programming knowledge, but is capped, so you can only get a portion of what is actually there. The Firehose is supposed to be all tweets for a given time period, and is quite expensive to access, whether via Twitter directly or through Gnip, an aggregator that helps other providers get access. Twitter has not disclosed which tools have full Firehose access.

The major problem that has arisen from widespread use of the API is the crop of slick-interface social monitoring and analytics tools that use the API instead of the Firehose and yet represent themselves as though they are appropriate analysis tools for significant amounts of conversation. While okay for small businesses that don't have much volume, for brands with medium to large amounts of conversation, the data provided by the API is incomplete because the API will only give away so much data for free until the data cap is hit and it stops serving information.

This is a major problem. Bad data = bad research = bad decisions = bad results and damaged relationships with stakeholders. In turn, this results in damage to your ability to use social media to grow and prosper, whether you are at a brand or an agency. The amount of money being spent on the basis of bad research done with API data is certainly in the high tens of millions of dollars and possibly many times that amount. There is simply no way of knowing how many bad decisions this reliance on incomplete data might have caused over the past few years. 

These tools routinely present themselves as real competitors to monitoring or analysis tools that have access to all the Twitter data that would really be necessary to offer the pretty graphs that they do. Unfortunately, it's impossible to know which of them is telling the truth. Twitter has not taken action to either shut these tools down, change the nature of the data they provide via the API, or disclose those providers who have full Firehose access, and publish guidelines as to when the others might be appropriate.

I spoke with Kevin briefly after the session and I believe that Twitter is going to take appropriate action. While this may (and probably should) be painful for a large number of tools that currently depend on API data, ultimately the industry as a whole will benefit greatly from increased consistency and measurement accuracy.