In an earlier post, I complained about the repeated failure of administrators to make appropriate decisions about closings of schools and businesses based on over-confidence of weather forecasts. As I begin writing this post, it is 9 AM on 2/17 following a snow event in the Washington DC area. As recently as 24 hours ago, the prediction was for up to 12 inches or even more snow will fall overnight. As the day progressed, the snow amounts dropped but still the upper limit forecast was over 8 inches. This morning the sun is shining and there is about 4 inches of very light (dry) snow. Of course, snow fall amounts vary widely over the metropolitan area and I’m sure some have received more.
My first observation is that the quality of the recent snow forecasts do not seem to be any more accurate than they were 20-30 years ago. I recall in the past there being adequate warning of snow events with surprises as to the actual totals that resulted. We knew about a day in advance a snow was coming and to prepare for it. We just had to wait for the event to end to find out how much snow would fall and how much that fallen snow interferes with our plans. In terms of impact on the day, there a lots of different types of snow that require different amounts to become problems.
In the past, we acknowledged the uncertainty in the weather forecast and usually waited until the early morning before making decisions such as when to shut down businesses or schools. I recall 40 years ago waking up early listening to radio announcements of closings. The announcements usually started around 5 in the morning, and based on seeing what is actually on the ground in addition to the forecast.
Today, the forecasts are not much different in accuracy. The difference is that we have more expansive computer simulations with high quality visualizations of hour-by-hour forecasts. The visualization creates a virtual reality that is very convincing. Yesterday, I saw some weather stations presenting a simulation of the weather radar image that looked like the real weather radar but was based on the simulation. They then showed a comparison of the current weather radar with the simulated image to show that they were very close so we we can trust the future simulation even more. I suspect the comparison of the current weather radar was with the most recent simulation run so it was probably unfair comparison, but it did the intended job.
Instead of waiting for the morning to see if schools and businesses will be closed, nearly everything announced by 9 PM closings for the following day. The governor of the state announced a preemptive state of emergency and warned of a snow event that would rival the large storms that hit 5 years ago. The federal government closed as well.
Snow had ended in the early morning with just 4 inches on the ground and it was a dry snow that that is easy to plow. Certainly, it was good to have less traffic in the morning to allow the snow plows to complete their plowing, however, it shouldn’t take long to get most of the roads ready for traffic.
I don’t think the snow event justified shutting down everything (federal government in particular) for the entire day. Rolling back to a couple decades ago, the same event would have the government wait until the early morning to assess the road conditions and probably would have issued a 2-hour delay or liberal leave policy instead of shutting down everything for the entire day.
What is remarkable is how we reject this wait-and-see approach to making such decisions. Everything was announced closed by 9 PM of the previous day when the snow had just started to fall. As soon as the ground started whitening, there were insistent demands for announcements of a snow-days for the following day. The same forecast that accurately predicted when snow would start falling also predicted the accumulations would be overwhelming. Despite the fact that this is follows previous examples of inaccurate snow-accumulation forecasts, we still had confidence in the capability of weather forecasters.
I’m not sure confidence fully explains the demand to act on simulation results. Perhaps a more accurate reason is fear. The simulations presented the possibility of a high snow fall event and the mere possibility demanded action out of sake of caution. The caution not only applies to school closings where heightened concerns for erring on the of caution has some justification. It also applies to businesses and even the government itself. My impression is that the fear is not that people would be endangered by attempting commutes to work. Instead it is the fear of being criticized by people who were inconvenienced by longer commutes or by have to get a tow to get out of ditch somewhere. These are annoyances but most people would have been able to get to work today, perhaps delayed a couple hours, and few if any would have been in much danger in the attempt.
My point is that had the decision-makers waited until morning to make their closing calls, they almost certainly would have announced a 2-hour delay with liberal leave policies (allowing people to make their own choice about attempting the commute). Instead, everything was announced the night before and before the main precipitation was predicted to arrive.
From my perspective, that decision to close schools and businesses in advance was an illustration of dedomenocracy in action. The data technology (in this case weather simulation models) generated a result that compelled decision makers to make an early decision. The human decision makers did make their decisions, but they were not allowed to act on their own judgement, fears, or doubts. Instead, they had no choice but to follow the advice of the computer models. The result is a decision that they can not personally defend.
So we have the present situation of government and businesses being inappropriately shut down based on an automated decision that no one is responsible for. The administrators had to make a decision based on weather forecasters. The weather forecasters had to announce results from simulation runs. The entire decision to close everything down was completely automated by data. This is dedomenocracy in action.
My recent posts have promoted the concept of dedomenocracy as a legitimate form of government. In much of those discussions, I assumed that it was futuristic form of government because data technologies are not yet sufficient to automated normal government decisions. I’ve already noted that we are already seeing the authoritarianism of data in health care. A similar case can be made concerning metropolitan area decisions based on weather forecasts. In these areas, we are experiencing automated decision making where humans given no choice but to follow the recommendations from data (and in particular simulation models).
Earlier on this blog, I raised objections to automating the human decision maker. In government, we need human accountability for decisions to persuade us that bad decisions were based on good judgement at the time. The examples of recent failed weather forecasts illustrate the need for human decision makers to learn lessons from poor quality of simulation predictions so that they can better defend their personal judgement of doubts and caution.
In recent weather forecasts, the weathermen (or simulation model gamers) are using their technology to tell stories. The value of human decision makers is to recognize that stories are being told. The decision maker needs the autonomy to challenge the story-tellers for their assessments of uncertainty and then hold them accountable when their confidence is betrayed by the actual outcomes. The accountability may be as simple as discrediting their products for future decision making.
Despite the big data simulations and the very attractive animated visualizations, modern weather forecasting is not really that much more accurate than forecasts from decades ago. This is especially true for winter events that are dependent on complex combination of factors of locations of cold air, moist air, and low pressures. As a result, decisions makers should be justified in following the older wait-and-see approach and hold of the cancellation notices until the early morning hours instead of the night before.