Open Secrets: data with strings attached

These cases are often described as open-secrets.   Many people in the community are aware of the information about individual cases and about the pattern of behavior, but there has been some kind of understanding that the past events are resolved in some acceptable terms, and that ongoing behavior is restrained by certain conditions.   The oxymoron of open-secrets can be resolved by defining the open-part as being observed data, while the secret-part is restraints on how this data may be used in future decision making.

Advertisements

Indifference to dark data

We should learn from recent experience of large data technologies the lesson that decision making can benefit from streaming data in addition to (and often instead of) the publication science of one-time experiments.    It is clear now that policy making needs access to a continuous stream fresh data about old ideas, especially when that data accumulates over time.   With access to the technologies to do this work, it is unacceptable to base policies on the failed approaches of the past that rely on published studies.

The Data Male

The title is a play on words from the analogy of the supposed hierarchy structures of wolf packs with alpha males, beta males, and omega males where roughly speaking the alpha males lead the pack and get first choice of resources, the beta males are cooperative and share resources more equitably, and the omega (or…

Human eras defined by modes of communication

The above video presents four eras of human communication from an evolutionary perspective where there was a long time when humans only gestured and grunted, then there was a long time when humans spoke but did not write, then a long time when people wrote. The useful information is the progression of information content possible with each era. The unnecessary information is evolutionary explanation. For this discussion to work, there doesn’t have to be a specific period of time when human culture flourished with illiterate people fluent in verbal languages. There are clearly expansions of content starting with gestures, then adding verbal languages, then adding written languages. The Internet era allows us to publish and retrieve information separately from the story-telling.

Spontaneous Data

With big data, we end up with deep historical data from distant events. There will be something needed to fill in the gaps that were mysteries at the time. That gap filler will be spontaneous data whether we acknowledge it or not. Even if we as humans leave the gap unfilled, we can’t be sure that our data analytics or machine learning algorithms won’t fill it. When it does, how can we be sure it won’t come up with a supernatural explanation that it keeps to itself?