Consider the case of a big-data store the was able to store all of the individual answers keyed with sequence numbers, time stamps, and specific individual identification. I don’t think anyone would voluntarily discard that data in exchange with anonymized data consisting of just a few categories. The value of data reduction into categories is for people who don’t have access to big data. Those people are the consumers who wish to have an external assessment of what kind of person they are, allowing them a shortcut to introducing themselves, similar to the 1960’s approaching of introducing oneself as a zodiacal sign.
In the analogy of the recent sexual controversies coming out of Hollywood, the repeated behavior by one individual results in the group ostracizing the offending individual even though each of his transgressions were satisfactorily settled individually. When it emerges that the group is not adequately policing itself, that offends an external group such as one part of the industry versus another. This becomes a new injustice requiring a settlement between groups. This process of cascading continues with every larger groups demanding some type of settlement from the other group.
Government of data and urgency permits as bright-data the observations of events leading up to the then-interpreted injustice and the observation of the terms of the subsequent settlement. The only data that is excluded is the dark-data of the now-settled prior-claim of injustice.
The delusion of injustice is that there cannot be privately negotiated justice that may be acceptable only to the private parties based on the particular context that they agree to keep private. As a result of this, we overrule such private agreements and require a fresh trial in a public or formal setting based on current sensibilities and on lost information about the full context of the earlier event. In the re-airing of past events, society regresses to a more immature level where every human interaction needs adult supervision and every offense be reported to authorities for a formal justice.
A conflict may be brewing, but it is not going to be like the Civil War of the 19th century. The groups are not concentrated in states and certainly not in contiguous states. Instead a better analogy is a divorce from the existing arrangement for coexistence of men and women in society.
These cases are often described as open-secrets. Many people in the community are aware of the information about individual cases and about the pattern of behavior, but there has been some kind of understanding that the past events are resolved in some acceptable terms, and that ongoing behavior is restrained by certain conditions. The oxymoron of open-secrets can be resolved by defining the open-part as being observed data, while the secret-part is restraints on how this data may be used in future decision making.
We should learn from recent experience of large data technologies the lesson that decision making can benefit from streaming data in addition to (and often instead of) the publication science of one-time experiments. It is clear now that policy making needs access to a continuous stream fresh data about old ideas, especially when that data accumulates over time. With access to the technologies to do this work, it is unacceptable to base policies on the failed approaches of the past that rely on published studies.