Stephen Baker

The Numerati
Home - Viewing one post

Paradox: Are Big Data successes largely anecdotal?

March 8, 2013Hop Skip Go

Those in the world of the Numerati, or Big Data, tend to pooh-pooh analysis based on anecdotes. And why not? They can easily be statistical outliers, and often they are. The trouble is that human beings relate to stories. They're so accessible, and perfectly suited to a sales pitch. So it shouldn't be any surprise that much of the hype around Big Data, whether in marketing or medicine, is based on stories.

Paul Barsch, a marketer at Teradata, makes this point in a blog post. He writes:

The truth is that some companies are having wild success reporting, analyzing, and predicting on terabytes and in some cases petabytes of Big Data. But for every eBay, Google, or Amazon or Razorfishthere are thousands of companies stumbling, bumbling and fumbling through the process of Big Data analytics with little to show for it.

Sadly, success in Big Data doesn't lend itself, at least at this juncture, to statistical analysis. Customers and vendors keep their failures to themselves. And we usually only hear about them (as anecdotes) after someone gets fired. This leaves it to the successes (and the liars) to trumpet their greatness. The Big Data narrative is built on anecdotes.


©2021 Stephen Baker Media, All rights reserved.     Site by Infinet Design

Kirkus Reviews -

LibraryJournal - Library Journal

Booklist Reviews - David Pitt

Locus - Paul di Filippo

read more reviews

Prequel to The Boost: Dark Site
- December 3, 2014

The Boost: an excerpt
- April 15, 2014

My horrible Superbowl weekend, in perspective
- February 3, 2014

My coming novel: Boosting human cognition
- May 30, 2013

Why Nate Silver is never wrong
- November 8, 2012

The psychology behind bankers' hatred for Obama
- September 10, 2012

"Corporations are People": an op-ed
- August 16, 2011

Wall Street Journal excerpt: Final Jeopardy
- February 4, 2011

Why IBM's Watson is Smarter than Google
- January 9, 2011

Rethinking books
- October 3, 2010

The coming privacy boom
- August 17, 2010

The appeal of virtual
- May 18, 2010