In many earlier posts, I described ways of distinguishing types of data sources.   In contrast to the data stewardship tasks of evaluating the quality of data (cleaning, deconflicting, deduplicating, conforming, etc), this topic concerns the assessment of data sources themselves in terms of the degree of independence between observations and preconceived models of expected observations.  At one extreme, I described as bright data that data that are well-controlled and documented observation free of any assumptions.   At the other extreme, is dark data that is purely calculated from accepted models.   Most data are between these extremes.

Bright data is necessary in order to discover new hypotheses about the nature or human world.

In a recent post, I added an energy dimension to data with the example of IQ being bright data that is very energetic.  I described IQ as ultraviolet data because of the controversies that arise from any hypotheses informed by IQ data.   Bright ultraviolet data must be handled with more care than lower energy data.

When I started writing about a meta-analysis of data, I was thinking in terms of the difficulties I faced trying to get useful insight from data from many data sources where that data often conflicted with expectations or presumed infallible truths.   I realized there was a big difference between a measured observation and a simulated measurement even though both generate quantitative results that appear to be interchangeable.   If I drop an object from a known height at a known time in conditions when things like air-resistance is negligible, a direct measurement of position at specific time intervals can be replaced with a calculation using the gravitational equation.   Even in that case, I would prefer the observation over the calculation.

Later in this blog, I began to think about the ultimate goal of using data for analytics to support decision making.  I described dedomenocracy as an extreme example of such data-driven decision making: where humans are governed by algorithms using data.   For discussion, I proposed a particular model that replaces the current model of near-perpetual laws with algorithm decided rules that are at any time few in number and limited in lifespan.   When the ideal data source is a pure observation undisturbed by models, the ideal default government is one where there is no rules that can distort human behaviors.   Rules are necessary to restore the peace, but they also need to be absent to allow collection of data to observe shifting boundaries of what is acceptable.  I described this form of government a Libertarian government punctuated by Authoritarianism.   The Authoritarianism is absolute, but restricted in scope and in time duration so that most people most of the time will live their lives without any interference from government.

As I thought about this type of government in contrast to our current government, I realized that we need to rethink punishment.  In the current model, we perceive our laws as being indefinite.  If the laws are not perpetual, they are expected to remain in effect for most people’s lifespans.   In such a model, we can justify punishment by long periods of incarceration.   However, if we adopt a Authoritarian punctuated Libertarian model, then the rules will change before a prison term is completed, making continued imprisonment irrelevant to current rules.   In addition, incarceration from obsolete transgressions will bias the collection of observations to base future decisions.   For dedomenocracy to work, we need to have everyone to be free to participate under current conditions.   As a result, we need to find new forms of punishments to replace long periods of imprisonment with briefer and probably more intense punishments (with the requirement to not result in lasting disability or disfigurement).   We will need to redefine what is acceptable punishment, what is unacceptably cruel or unusual.   Imprisonment is contrary to the needs of a well functioning dedomenocracy.

More recently, I wrote some thoughts about how libertarian type of government would evolve in response to self-association of various groups to exchange services with people who share their world view.   Consequently, there will arise within these groups some locally enforced customs or rules.   There will arise some method of distinguishing in-group from out-group members of the community when different groups intermingle.  Within a particular territory, one group would be the host and the other group would be guest.  Different rules will apply to hosts and guests.

These tribes or similar groups that identify by some shared identity are a likely consequence of the punctuated libertarian model.   The authoritarian aspects of the model will also evolve to oversee the interactions between tribes that compete over territory, resources, or commerce.   I imagine the authoritarian element will be analogous to past colonial systems where there is an external governorship to assure the productive and peaceful interactions between conflicting or even exclusive communities.

I imagine an algorithmic dedomenocracy would become the colonial ruler of the self-associating communities.   I further imagine a mostly benevolent or hands-off government that only intercedes briefly during urgent crises where the rules would be consistent with the most recent observational data.

It is in this context that I consider the nature of data illustrated by recent news about various high profile individuals having past objectionable behaviors exposed.   In the recent news, this information is harming or ending their careers that until now have been highly respected.    In most of these cases, the revelations are not really new observations, but instead old facts that have emerged at a particular time when the information is damaging due to surrounding discussions in popular or political discussions.

Specifically, the complaints concern abusive behaviors often of a sexual nature.  However, most of these complaints are often from distant past.   The follow a pattern of some transgression perpetuated by one party onto another where the immediate result is some kind of understanding that this would be excused if it does not recur at least against the same victim.

As I read about these examples, I think about the principle of protection from double-jeopardy where the alleged offence is adjudicated among individuals with some kind of mutual agreement that in some cases involved some form of punishment in the form of a monetary settlement.   In many cases, there is a decision by the victim to merely decide to avoid future interactions with the perpetrator, without any direct attempt at resolving the problem.   This decision, too, is a form of resolution to the problem.

Although this was not formally prosecuted in governmental judicial system, I am inclined to consider this type of interpersonal agreement to be closure to a case.   As a result, the re-airing of prior complaints strikes me as being a form of double-jeopardy.

It appears we are trying to argue that in reviewing the evidence about the cases the original decision did not go far enough.   While we are free to change our opinions about old cases, I do not feel comfortable about imposing new penalties on a transgression that had previously found some form of agreement between the perpetrator and the victim.  I saw this even with the understanding that the circumstances of that agreement involved an unfair power differential between the two.   There was some kind of agreement that the victim would have some form of apology for the past action, with a promise that it would not be repeated in the future.   That should be the end of the story.

In these recent cases, there emerged a secondary story about a pattern of behavior.   The misbehaving actor repeated similar offenses with multiple other victims where many of these also ended up independently negotiating some form of apology and promise against future transgressions.   In this case, there is new information in the form of a pattern of behavior.   Even if we should not re-prosecute some individual transgression, we can prosecute the new evidence of a pattern of behavior.

However, further revelations showed that even this meta-observation of a pattern of behavior was also already known.  In this case, there was some form of agreement of apology or establishing of rules for restraining what is allowable for future continuation of this behavior.   In other words, even the pattern of behavior has already been adjudicated by the community and a sentence has been passed.   It seems unjustifiable double-jeopardy to prosecute this a second time to establish some new penalty.

I mention all of this in context of my model of dedomenocracy that needs to use data.  These examples are of bright data (generally reliable and documented observations) that should be available for dedomenocracy algorithms.   However, these data are different from other bright data in that there should be some constraints on how this data can be used.   In particular, we should have access to this information about particular people in order to optimize future roles for these individuals, but we should not use this same data to penalize the person further beyond what had already been agreed to.

In the recent examples, an example of an acceptable use of this data are the decisions about continuation of association with certain projects.   An example of unacceptable use of this data is to seek further retribution from the perpetrator for actions already having some kind of agreement between the perpetrator and the victims.

A well functioning dedomenocracy needs this type of data, but this data has strings attached.  The data may be used for only certain types of decision making.   If some algorithmic decision concerns assigning guilt to some individual, then the data available to that algorithm should not be contaminated by some past decided case.   In contrast, some decision involving some future assignment for the individual can use that data.

These cases are often described as open-secrets.   Many people in the community are aware of the information about individual cases and about the pattern of behavior, but there has been some kind of understanding that the past events are resolved in some acceptable terms, and that ongoing behavior is restrained by certain conditions.   The oxymoron of open-secrets can be resolved by defining the open-part as being observed data, while the secret-part is restraints on how this data may be used in future decision making.

The main part of the open-secret is that it is a secret outside of the community.   When the information gets disseminated to outside communities, it is new information to those communities and they may expect some new resolution to compensate for their being misled interaction with the community keeping the secret.   I think this is an important issue that needs a resolution, but I think it is incorrect to resolve this by imposing some new penalty on the individual involved.

This new revelation is a complaint that one group that kept a secret from another group.  A colonialist approach offers a different way to resolve this new complaint.  In a colonial system, the effort is to seek an inter-tribal agreement to resolve the transgression.   This does not require additional penalties on the individual perpetrator.   Instead, the agreement may involve some form of agreement about changing the group’s governance of its members, or some kind of concession to the other groups, such as disqualifying the perpetrator from certain types of external interaction with the outside groups.   In any case, the inter-group conflict is resolved at the group level instead of the individual level.  The colonial-style dedomenocracy will facilitate the resolution of the terms between the groups while maintaining the sovereignty of the separate groups to regulate the behaviors of the in-group members of their group.

In this post, I described a different categorization of data to represent what we call “open-secrets”.    It is a special case of bright data that has limitations on what types of algorithms are permitted to access this data.   In government context, open-secret data is analogous to classified data with need-to-know limitations on who may use that data and for what purpose.   As I have previously presented dark-data (model generated data) as the opposite of bright data (observation data), there must be some opposite of open-secrets.   That opposite of an open-secret may be fake data invented specifically to facilitate some outcome: a topic for a future post.