Dark data betrays bright data

This blog obsesses over two ideas of science expressed in two ways: in terms of data and in terms of government.   Although my discussions of both data and government are different, they are actually consistent with a more fundamental observation.   That observation is what I describe the difference between past-tense science and present-tense science.

Past tense science involves the remembrance of previously tested theories and applying them to current circumstances, and as a result insisting that the present resembles the past.   In contrast, present tense science respects the original concept of science as a process or method to process observations and find rational explanations for what is happening now even if it involves rejecting notions accepted in the past.

An alternative distinction is between scientists and science.   In particular, scientists who advise, instruct, and lecture about current understanding applied to what we are seeing now.   These scientists insist to explain the present as a special example of an already understood phenomena.    Science is a process for processing the new information from the present and recognizing the explanation that best applies to what we see, even if that explanation contradicts past explanations that we were told to expect to apply.    While only certified trained people can claim to be scientists, everyone can do science by observing and testing what explanation works best currently.   The basic method of doing science is well taught in elementary schools, so everyone is capable of doing it.

We are commanded to respect or even love science especially when making decisions in all areas of our lives, from personal life choices to governmental policy making.   The science in this commandment is specifically what scientists tell us.   As a result, it is commandment to not do science ourselves.

The two concepts are fundamentally in conflict.  Scientists are invested in the validity of past tested theories and as a result will stretch the theories to explain new observations, or use the theories to justify the rejection of conflicting observations a outliers.   Doing science puts priority on finding the best explanations of the new observations, consequently making past theories disposable if they don’t explain what is happening now.

From a data perspective, I describe two types: dark data and bright data.   Dark data is the data from science models (and thus from scientists), and bright data is data from observations (and thus available to everyone).

From a government perspective, I describe two types: current government systems inspired by enlightenment ideals of the superiority of established science, and an alternative model of government by data and urgency inspired by the concept of allowing machine algorithms (such as machine learning) freedom to find new explanations for current observations.    The second type is a very recent possibility due to the emergence of affordable mass data collection, storage, retrieval, and analysis.   While this is a new possibility, it closely resembles our human governments at the beginning of the enlightenment where people still had a healthy suspicion of past explanations and a popular culture of doing science among the general population including many without formal education.

I believe that the earlier forms of enlightenment-inspired governments were much more resilient to new circumstances.   Their repertoire of established science was still young and thus viewed with suspicion.   In addition, they had a popular culture of doing science as a form of pass-time in addition for entrepreneurial efforts with payoffs of economic success.   This period of more dynamic science-doing had failures such as producing damaging pollution or harmful products.   The same system innovated to solve these new problems by finding new science that incorporated.

The early forms of enlightenment-inspired governments used the process of making new science with the intention of making a better future, even if sometimes they failed.   Part of the doing of science is publishing the results.   We teach these published results with carefully crafted experiments that replicated the old science.   Eventually we ended up with a generation trained to respect the published science’s ability to explain easy to replicate experiments.

The replication of old experiments proves the superiority of published science (or teachings of scientists) over any new theories that attempts to explain better observations that we are seeing in the present.

Our current governments or being run by generations indoctrinated on respecting the science that explains reproducible experiments.   As a result, the nature of government is fundamentally different from our earlier governments presumed based on the same principles.   It is true that the expressed ideals in governmental constitutions have preserved the same basic principles.   But there is a difference in terms of attitudes toward science.

We started with a vision of science to explain current observations with the intent of making a better future.   We ended up with a vision of science that already explains anything so anything new is either an error in observation or something that must be eliminated.

In data terms, we started our enlightenment experiment with our full attention on bright data of the most recent observations and we ended up with our full respect to past explanations to the extent that we permit those explanations to override or disqualify any recent observation that hints at something new.

I fear the current system of government is starting to go out of control because it has replaced a working closed loop control system (based on active science discovery of new information) with a dead-reckoning system that science tells us we should be given the initial conditions.   In the sea navigation analogy, dead reckoning convinces us that we should not believe our eyes when we see we’re approaching some land.    The sight of land is necessary some kind of mirage.

In the COVID19 response, we set out on a plan months ago based on our confidence of computer models.    We convince ourselves that the plan must have worked when we fail to see the case and death counts predicted.    In addition, we convince ourselves that the mounting evidence of ruination of people’s livelihoods will be a temporary recoverable condition consistent with what our models told us a few months ago.

The current democratic government is run by politicians, bureaucrats, and electorate who all are gaslighted into distrusting their own observations that disprove the original explanations and projections.   We need to ignore what we are seeing and continue on the original plan because it was based on infallible science, science we know is proven because we can replicate the experiments that prove it.

There are a lot of observations that should tell us that we were wrong in our original assessments and thus the resulting policies were wrong.

The betrayal of our past-science mentality is that it is blind to observations that prove the original policies were wrongly decided.    Our faith in the infallibility of science described by respected scientists drives us to accept that the original policies indefinitely.   Instead of a simple observations that things are not working out as originally promised, we demand extraordinary evidence to disprove the original conclusion that this was a virus that presented an existential threat to all of civilization.

I think a government by data and urgency could better serve us in this kind of scenario.   Such a government acts tentatively based on current information and current assessment of fears of the population.   Such a government makes no pretense that it knows the absolute and final truth, and this is proven by issuing rules that expire quickly.

The biggest difference between the two governments is what happens after the first trial period.

The government by data and urgency backs off and needs a new demand for action by the population and then the government makes a completely fresh assessment of what best to do, having no obligation to remain consistent with the past policies.

Our current government based on our faith in science as expressed by respected scientists will set no significance to any fixed date.  If we do demand an assessment at some point, that assessment will be to report on what has changed to describe progress of the original policies.    Our confidence in our science maintains our commitment to the original policy.    If data does suggest a change is needed, the only considered changes is an adjustment on the current policy.

We can never consider completely rejecting the original policy because doing so would reject the infallibility of science.   Similarly, we will never penalize the scientists who advised a failed policy for the same reason.    The consequences are that we are committed to following a bad policy and we’re doomed to make similar mistakes from the same scientists.

I would prefer to return to an earlier era where culture was more suspicious of past science and more eager to discover new science.   It would make better use of current observations and it seek a science that promises a better future that accounts for the current observations.

We are stuck with a culture that respects past science to the degree that we have to ignore observations to preserve the correctness of our science.    In addition, we have to accept the objective of making the future resemble the past where the science came from.    For the COVID19 situation, the only permissible objective is to return to the past when we never had to deal with the COVID19.   We need to annihilate the virus or achieve some kind of universal immunity.     There is no consideration that maybe we can have a brighter future if we just learn to manage it to be comparable to the seasonal flu.

A possible remedy to the current science-is-authoritative culture is to require that any new policy include a criteria that automatically repeals the policy.    The policy already is based on some kind of scientific justification of predictions of what would happen if the science is correct: bad things will happen if we don’t implement the policy, and less bad things will happen if we do.    The same science should be able to identify the disqualifying measures.    The same models that predict a future impact with the policy should include bounds on that impact so that if actual measurements go outside those bounds then we should conclude the original science supporting the policy was invalid and thus the policy itself would be invalid.

In the current situation, we originally were told to expect a certain number of hospitalizations and deaths even with the extreme lock-down orders implemented in many areas.    Those predictions did not come true, so we should be able to completely void the original policies as being based on flawed science.

This will never happen.   If we admit that science can be flawed than our entire government would be flawed.    Our current government is based on the trustworthiness of the scientists.  We cannot allow them to be exposed as frauds.

Leave a comment