In earlier posts (such as here) I presented my concerns about the misplaced confidence that big-data supported predictive-analytics should obligate decision-makers to follow the computed recommendations.  My focus was specifically on completely automated decision-making, but I also suggest this is a subset of a broader mandate to obligate decision-making to be evidence-based.   Increasingly, we are lead to expect that all decisions are forced by convincing evidence.   Conversely, decisions are invalid if they are not supported by convincing evidence.  An example of the latter comes from the recent court decisions striking down same-sex marriage bans such as this one because

The ruling stated that the defendants’ argument that bans on marriage for same-sex couples promotes “the welfare of children, by encouraging good parenting in stable opposite sex families,” had no evidence of to support it.

“Because defendants have failed to demonstrate that these laws further any legitimate purpose, they unjustifiably discriminate on the basis of sexual orientation, and are in violation of the Equal Protection Clause,” the ruling states.

Decisions are not legitimate unless they are backed with compelling evidence, and decisions must be compatible with compelling evidence.   Evidence obligates decision-makers to make the one best evidence-based decision.  With big data and fast automated analytics, we have the opportunity to automate decision-making because the evidence is in the data and the decisions can be made quickly.   Advocates of evidence-based decision-making encourage us to embrace this approach (automated or not) as being optimal decision making.

I disagree.  Legitimate decision-making result in decisions that are contrary to available evidence.  Such decisions may be better than blindly following the recommendations based on evidence.

I think the present Ebola crisis provides an example of this.   We are told that all of the available evidence is that Ebola will not be a crisis in highly developed and wealthy countries like the USA.  The disease is hard to transmit and requires direct contact to body fluids from a symptomatic patient where that contact has access to an opening to get inside the body.

Ebola has a low infection rate.   Based on hard evidence from West Africa, the R₀, the average number of people contracting the disease from an infected person, is measured to be between 1 and 2.   Also, we have evidence of demonstrably infectious practices in those countries where these practices involve traditions or poverty conditions that do not exist in this country.   Based on evidence alone, we should make decisions based on the expectation that the R₀ for this disease in USA will not be greater than 2 and probably will be much less than 1.   The evidence says that the disease is not a threat to USA, and thus any suggestions of doubts or fears that it may get worse are not legitimate.   We dismiss anyone who suspects it might get worse as fear-mongers or as having the nefarious goals of politicizing the controversy especially during the run up of national elections.

My last post presented some observations I had about the recently learned evidence of what happened following the first case Ebola in USA.   These evidence-based observations suggests some improvements in our handling of the disease.

Prior to this event, all of the evidence was of observations of the disease in West Africa.   Presently, the news is focused on all of the mistakes we made.  However, at the time the mistakes were made, I think we were following legitimate evidence-based decision-making based on the premises I outlined earlier. Ebola is hard to transmit and its spread and severity in West Africa are a consequence of conditions unique to those countries.   The evidence suggested that common practices an any US hospital should be more than sufficient to experience an R₀ of less than 1, and closer to 0 than to 1.

The recent evidence gave us new evidence that the disease is robust even in this country.   Again, in hindsight, we explain the new infections as resulting from already-known-risks involving direct contact between patient and caregiver.   I still think that at the time, there was confidence that even our routine contagious disease treatment practices were sufficiently superior to practices available in West Africa to prevent the spread of this disease.

Evidence-based decision-making suggests that the only legitimate decisions are those that are backed with compelling evidence.   Objections to such decisions must themselves be backed by already accepted evidence.  When nurses complained that they felt insufficiently protected, they lacked the evidence that the practices they were asked to perform were in fact dangerous.    Later, when one of the nurses self-reported a feeling of illness and requested advice about whether to proceed with travel, the evidence at the time clearly supported the recommendation that she was safe to do so.   At that time, she had not yet exhibited symptoms that indicated Ebola, and without those symptoms she was not at risk of infecting others.  That is what the evidence tells us.   It is too early to tell if she may have infected others, but I doubt we will find any cases.   But, our reaction now is that that earlier recommendation was wrong and should not be repeated.

Now, the recommendation is that people who had possible contact with an Ebola patient sufficient to be at risk for contracting the disease should not travel until 3 weeks have passed.   Previously, the policy is that there was no risk for such travel for the potential patient not yet exhibiting symptoms, nor was there a risk of infecting others.   This previous policy was valid evidence-based decision-making.   We have no evidence that supports with any statistical confidence a conclusion that such travel would have been risky.  Just like we had no evidence that supports that the practices employed by the health-providers of first Ebola patient were excessively risky (despite the fact that we later declared those practices to be errors).

Evidence-based decision-making is that evidence should determine a decision, and that any alternative decisions are invalid when they do not have supporting evidence.   In my opinion, the treatment of the first Ebola patient in US followed valid evidence-based decision-making for each and every step.   It is only after experiencing bad consequences that we condemned the earlier practices as wrong.  We are able to do so only because we now have new evidence that US hospital practices were insufficient to save this first patient’s life and were insufficient to prevent the spread to healthcare workers.  Prior to this experience, we did not have sufficient evidence to object to any of the practices that occurred at the time.   The evidence was that USA is not West Africa in many ways that must make the disease easier to manage here than there.   We learned this was incorrect only because we had new evidence that the treatment and management of Ebola is more challenging here than we expected.

I argue that decision-makers need the flexibility to consider more than just hard evidence in making their decisions.   Decision makers need to make judgments that balance the evidence with their fears and doubts.   At the start of this post, I referenced the court decisions to strike-down bans for same-sex marriage (SSM) because there is a lack of evidence to justify fears and doubts that underlie these bans.   The absurdity of comparing the risks of issues surrounding SSM with a deadly contagious disease illustrates the role of doubt and fear in decision making.  For SSM, the fears and doubts are illegitimate because there are no supporting evidence for those fears.   The same appeal to evidence applies to our arguments for the illegitimacy of fears and doubts about USA’s capabilities to treat and control the spread of Ebola.   The fears are doubts are illegitimate for public debate because there is no evidence to support those fears and doubts.

A counter-argument for the need for consideration of unsupported fears and doubts is illustrated when things do go bad.  The first Ebola patient in USA did in fact die despite access to superior US healthcare systems.  We have two confirmed cases where this patient transmitted the disease to two others.   Our response to these failures is properly to hold the leadership in the hospital and in the CDC accountable for these outcomes.   They had to answer for what went wrong and what they will do differently in the future.   These are two different questions in terms of evidence-based decision making.  The second question about future practices is an obvious outcome of evidence-based decision making because it must incorporate the new evidence of what Ebola is like in a USA healthcare system.

From an evidence-based decision making perspective, none of the actual decisions (later to be declared mistakes) were at the time illegitimate.  They were all well supported by the evidence.   In fact, although humans were involved, the decisions appear to have been almost automated.   Everyone was following available documented procedures.   Following the logic of the above judge’s opinion in SSM, these were the only legitimate procedures to follow.   For whatever reason, the preparations and guidance were evidence-based best practices at that time.  Not following those procedures was not a legitimate option.   Clearly we can argue these policies need improvements base on recent evidence.  However, we had at that earlier time predetermined that the evidence supported the policies that were followed.   I would expect that an automated evidence-based decision-making process would have come to the same decisions.  At the time, there was no evidence that the practices would not succeed because USA is not West Africa, and the disease is inherently hard to transmit.

I firmly dismiss the argument that decision-making must be constrained by evidence.  When all decisions must be consistent with evidence and must be based on evidence, decisions can be automated especially as we advance our big-data analytic technologies.   Automated decision making dismisses any role for unsupported fears or other doubts.   I argue that we need human decision makers to employ human judgment and ethics to weigh the evidence against fears and doubts.   Because we still demand human leaders to be accountable for their decisions, we need to allow them the ability to come up with the decision in the first place.  We can not argue that someone made a mistake when in fact he had no choice at the time but to follow the evidence-based recommendations available at that time.

When a decision was determined on the best available evidence at the time, we can only declare that decision to be a mistake if we expected him to have fears and doubts that were contrary to the evidence.   We want human-decision makers to make decisions consistent with their fears and doubts that are contrary to the evidence.   We select and retain decision-makers based on our being persuaded that their fears and doubts are reasonable.

There will always be a role for fears and doubts for all decisions, not matter how much evidence is available.  While these fears and doubts are legitimate topics of debates, they are also legitimate objectives of the debate.  We should allow decision makers to persuade us of his fears and doubts.   Using the term fear-mongering improperly dismisses as illegitimate.   If there can be no fears and doubts, then there is no need for human decision-making.   We can automate decisions based on evidence, as it appears to me that we effectively did in the treatment of the first Ebola patient in USA.   As I discussed in my last post, the leaderships dismissed the nurses expressed fears and doubts based on the evidence at the time.  A big part of that evidence was that USA was not West Africa.  It is only afterwards that we learned that this distinction doesn’t matter for this disease.

Even now, we are informed that the evidence is that we should not fear an Ebola epidemic in the USA.  Even on a local scale, we will experience a widespread breakout of the disease.   The evidence is that we’ll quickly isolate new patients with superior medical attention.   We are not going to allow Ebola patients to remain where they can spread the disease.  The evidence supports the government’s conclusion that USA is not at risk of an Ebola epidemic, largely because the West African experience is fundamentally different to the conditions in USA.

I find the following quote to be relevant to my objections to evidence-based decision-making:

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.

Donald Rumsfeld

Evidence-based decision-making relies mostly on the known-knowns.  Also, by using analytics we can also gain confidence on the unknowns that we know about.   In contrast, evidence-based decision-making has nothing to contribute to the unknown unknowns.   Fears and doubts exist in the unknown unknowns.

In the case of Ebola, we know from evidence what practices help save a patient and prevent the spread of the disease in West Africa: these are the known knowns.   We also know that we have a lot to learn about the hardiness of the virus and about how to provide better treatment to assure a lower fatality rate: these are the known unknowns.    We do not know what the virus will do when it begins spreading in a more affluent and more expensively equipped nation like the USA: these are the unknown unknowns.

I suggested earlier that some of our modern practices may actually be facilitating the spread and severity of this specific Ebola outbreak.  This outbreak is far worse and lasting longer than earlier outbreaks.  Coincidentally, it is spreading rapidly in more affluent and better educated communities.   While I agree that conditions could be better, I am impressed that many of the communities affected do appear to have very well-informed populations who are doing the right things available to them to prevent the spread of the disease.   This is circumstantial evidence but the fact that this new outbreak is worse than earlier outbreaks at least suggests that more primitive practices of earlier outbreaks may have a self-limited effect on the virus.

There is a possibility that our more modern practices are more favorable for this virus.  If that is the case, then perhaps USA may present an even more favorable environment for this virus to spread.   R₀ for USA may be higher than what is observed in West Africa, perhaps even much higher.

In the the most underdeveloped areas in West Africa, the infected body fluids (vomit, diarrhea, blood) are disposed directly on the ground that exposes the virus to a naturally hostile environment that may assure its quick destruction.  In contrast in our country, at least the earlier stages of the disease, these are disposed through common sewer systems that happen to have continuous moisture, constant temperature, and protection from solar radiation.   The virus could survive modern sewage treatment plants and enter the water supply that may be available for livestock that may later become asymptomatic carriers of the virus.

We already suspect the virus vector in Africa to be so-called bush-meat of generally small animals used as a food source.   There is the possibility that the virus could find a suitable host in one of our vast populations of livestock consisting of dense factory-farms with stocks of very limited genetic diversity.   Either a large part of our food supply (of animal products) can become shut off due to contamination, or that supply will suddenly enable a nationwide epidemic of everyone eating the meats raised in USA before we trace down the problem.

In the US we have a much larger population who have the wealth to own pets, and in particular cats and dogs.  These animals could become hosts of the virus (perhaps without symptoms) and spread the disease by their contacts with neighboring pets.   This may be more limiting in more impoverished areas that can not afford to keep such pets.   Already there is concern that dogs can be asymptomatic carriers, are naturally inclined to eat feces from their own species, and very frequently transfer their saliva to humans.   The disease may find a way to spread through our pets in a way we have not yet observed in West Africa.

In the US, we are more likely to use synthetic materials for clothing, bedding, and especially for medical protection, while poorer areas may be more inclined to use more natural materials.  The synthetic surfaces may be more hospitable to the virus allowing it to survive longer thus giving it more time to expose others and later situations where exposure would not be expected.

The above are a few examples of possible unknown unknowns about the virus’s activity in USA.   These are on top of the more popularly discussed concerns about our more mobile and more crowded culture.   During any person’s typical day, he may find himself in a lengthy period of time surrounded by individuals who come from a wide geographic area.   During the earliest stages of the disease before the disease is recognized, the lesser degree of contagiousness of the disease may spread because it has more abundant opportunities.    For every known unknown such as the opportunity to spread within airline cabins, there are many unknown unknowns such the opportunity to spread in restaurants and grocery stores, or the risk of spreading through open food service bars like salad bars.

For each of the above cases we do not have any evidence that they will present a reason for concern.  For evidence based decision-making, making decisions based on these fears and doubts is illegitimate precisely because there is no evidence.   These types of fears and doubts do not yet have records of observations in some database or scientific study anywhere.   A large part of the official statements that we should not fear Ebola is based on the fact many fears and doubts lack any evidence to back them up.   That’s the same environment we found ourselves when the first Ebola patient entered an emergency room in a USA hospital because of an illness that turned out to be Ebola.   Everything we did then was supported by the evidence at the time.  That confidence led to a whole sequence of bad choices that we now insist we will not repeat.   Evidence-based decision-making failed us already.  Fears and doubts by decisions-makers are not only reasonable, but they should be expected.   It is reasonable to anticipate there remains a lot that will still surprise us as Ebola establishes itself in USA.