Navigators for data

Linked above is a video by Captain Disillusion concerning the various special effects used in the 1986 movie called The Flight of the Navigator. I enjoyed this video on multiple levels. I enjoyed his admiration for the movie as well as the work that went into the movie. I enjoyed watching his own skills in creating the video. Finally, I enjoyed being reminded about a movie I watched only once, a long time ago, and yet left a lasting impression on me.

Working only on the distant memory of the film, I would hazard to summarize the movie as an autonomous space craft of extraordinarily advanced technology seeking out a navigator. The machine was very advanced with a wide range of capabilities. It was sentient and yet it was uncomfortable exploring worlds alone. I don’t recall the details of why it needed a navigator, but I recall it being a deep rooted need at an emotional level.

Now being reminded of this story, I think about how it relates to topics I discuss in this blog. In particular it relates to my discussion about science and data, and about the concept of being governed by data.

Since the production of this movie, we have experienced a different kind of advanced technology. Instead of spaceships with shape shifting materials, exotic propulsions, and interdimensional travel, we have data technologies that certainly I didn’t anticipate back in the middle of the 1980s. Back then, we had all of the basic concepts for data used today. I did not anticipate the scale we would be able to achieve in terms of collecting, storing, retrieving, analyzing, and presenting the data with costs affordable by private individuals. From the 1980s perspective, the current capabilities would have been a fiction of scale instead of a fiction of science.

A repeated complaint in this blog is the tension between observations and science theory. We have an impressive library of scientific theories that are very successful in explaining and predicting nature and create new things. Each theory encapsulates some explanation about past observations, with a preference for observations in carefully controlled experiments. My complain or concern is that if a theory can calculate an observation, then we don’t need a fresh observation. In particular, if a fresh observation does not agree with the calculation, we either reject the observation or correct it to conform.

We use scientific theories to guide most of our public policies and governance. The observation task is to provide inputs to the theories. We base our policy decisions on the theory’s conclusions about the observations.

My observation is that the theory summarizes historic data, sometimes very old data. We do not allow fresher observations to adjust the theory. This is especially true for periods of crisis. During a crisis, we believe there is no time to update the science. We have to act on what the established science tells us.

The recent example of the COVID19 response is a good example. Even now, a year after the crisis started, we are still operating on the presumption that this situation is a pathogen that the human immune system is unprepared for. The science tells us that if a pathogen is unique enough, then every human would be vulnerable to infection, the infection would become serious to the point of causing death. The science needed some simple observations about the contagion factor or the R value, and about the case fatality ratio. From these, the theory predicts a calamity unless steps were taken to restrict people’s activities and to intervene with some medical injection.

I am relating this to the movie in the following sense. The ship of the movie is analogous to our data capabilities. These capabilities are outstanding. We can measure so much about just about everything. The problem is that we don’t know how to use this data in its raw form. Our data technologies need a navigator.

In the movie, the navigator was a young inexperienced boy. In our world, the navigator is our scientific theories that are the opposite of youth or inexperience. The theories are based on old data and old assumptions. The advocates of the theories are generally well aged or must agree with someone who is older.

The movie stated something profound. The spaceship needed a young navigator. A young navigator explores the world so that everything is a fresh discovery. The navigator lacks training, maps, or even a firm destination. The ship needs and desires this kind of navigator, one that is open to discovery.

To be useful for governance, our data requires a navigator. The movie at least suggests that we have a choice of navigators. One choice is the old experienced well trained navigators who would confidently charge ahead based on his experience, similar to the captain and pilot of the Ever Given ship that blocked the Suez Canal recently. The other choice, is something that pays close attention to the current conditions with an open mind, ideally with a mind that is open to wonder and awe.

We do have these choices, but we are obliged to hand the navigation to the seasoned navigator we call science. Doing anything else would be irresponsible from the perspective of our governments and judicial systems.

The alternative choice is the young navigator with a mind open to new discovery. Our modern technology of rapid machine learning is the analogy of the young boy in the movie. There are a lot of different approaches to machine intelligence where many do have constraints to follow science. I am referring primarily to deep learning technologies that have to discover their own world models based only on recent observations that exists in extraordinary abundance.

On this site, I describe a concept of governing by data and urgency only. I call it a dedomenocracy with a very specific implementation. This government defaults to imposing as little interference on the population as possible. The goal is to collect unbiased data about the natural competencies and behavior of everyone and of the groups they form. When there is a super-majority demand for action during urgent times. the government will take action. At that point, the government will use all available observations with a preference for the most recent to come up with an optimal policy that optimizes the public’s prior statements of their priorities of their most desired goals for the future, and of their most feared hazards.

When this type of government is triggered, I can see it being like the boy in the movie. The government is navigating through the crisis, dodging the hazards and seeking out a happy ending. This kind of navigation has no predetermined playbook. We cannot predict the destination and thus we can not predict that path that gets us there.

Here is another way to distinguish the two navigator options. Our current government prefers to manage a crisis with the goal of returning things to the way they were before, or perhaps to use this opportunity to achieve some political goal not previously possible. The alternative approach is to explore the current circumstances and be open to discovering what makes this situation unique.

Our current reality is full of new discoveries concerning both hazards and opportunities. The disease is not producing the outcomes that our models predicted. Our emergency use interventions are not as safe or effective as science assured us they would be. Despite this, we charge ahead trusting everything to our seasoned navigator in the form of following the science. Increasingly we are seeing new looming catastrophes that appear to be the consequences of our navigator’s choices. Of those who do see what is likely to happen this coming Fall and Winter, many are increasing their support for their old navigator to avoid the fates that happened to the Ever Given, or the 737Max crashes.

We are effectively on a spaceship and we have to hope that our pilot will safely deliver us to our desired destination. To do this, we picked the best pilot, the one that has been trained for this situation. The pilot will apply his training and experience to select what data to pay attention to, and what to ignore. He will then steer according to what should happen even as evidence mounts that things are not behaving as they should.

There comes times when the situation is outside of the navigator’s experience and training. In those times, the old navigator may be incapable of opening his mind to fully pay attention to what is actually happening. Some times call for a younger mind that is learning in real time, absorbing the recent observations with youthful wonderment. Those are usually times of the most severe crises.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s