Extraordinary Popular Delusion and madness of Crowd (data)

I wrote this in 2014.

“Such is our confidence in the promise of big data.

My bet is that our optimism will crash in the not too distant future. Probably in time for the 2016 presidential elections. In USA, these elections tend to provide the needles that pop bubbles.

The bubble bursting event will come when there is a spectacular failure that is traced to a reliance on well accepted prediction algorithms on well accepted data. Bubble will burst when we realize we can not rely on algorithms to make decisions.”

Hypothesis Discovery

In earlier posts I suggested a better term for Big Data would be Crowd Data.   The characteristics that distinguish big data from other data are similar to what distinguishes crowds from other aggregations of people.    The challenges and risks are similar as well.   Crowds can get unruly.

In a recent post, I suggested that the employment market for data scientists to deliver the promises of big data may be overly exaggerated setting up a generation of grads burdened with college debt burdened with insufficient data science work to employ them.    That argument hinged on the typical under appreciation of macro economic capacity to find efficient use of computer science labor.  In particular, the demand for big data solutions will be met by large solution vendors employing a relatively smaller number of jobs with skills in the data science specialty of computer science.    Every big data…

View original post 1,807 more words

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s