In my work experience, I was involved in assisting decision makers or at least participating in a broader decision making process.   These were large decisions with long term consequences.   Individual decisions were resolved over long periods of time with multiple sessions to evaluate different considerations about the decision.

There are many types of long term decisions.  Sometimes the decision was to choose between two alternatives.   Other times, there was only one option but the question was when that option should begin.   In general, the decisions were large in scope and duration and involved a considerable investment in evaluating and debating the underlying issues.

Every decision confronted good arguments from alternative proposals.   In many cases, it is easy to imagine that the alternative would have worked.   A choice had to be made, and ultimately the decision was based on what side had the stronger support.    A change in make up on the coalition of support could have decided on the alternative.

In earlier posts, I described an approach viewing history as a consequence of decisions made in the present moment.   In that discussion I place the present instant in a different reality than historical data.   I asserted that the present instant what I mean by the entirety of reality.    This view of history makes history dependent on the present moment and in particular the decisions made in that moment.

I contrast this view of history from history being about the available data of what happened.   Data may in the form of electronic information, or it may be in the form of material evidence or human accounts of events.    In several posts, I discussed ways that the data may be unreliable witnesses to the actual events.   However, the facts of what happens has to be in some way consistent with the data (or evidence).    This is how we normally see history.   History is about facts that best fit the available evidence.   History is dependent on these facts.

Both views seem to say the same thing.   Evidence of facts had to be created at some point in time that was at some moment the immediate present instant.    I’m trying to draw a subtle distinction of the dependency of the understanding of history: history is dependent on decisions made in the current instant, or history is dependent on the remaining evidence from a past event.

Deeper into this distinction is an assumption I have that free will is available to us in the immediate present moment.   We and all living things have multiple options available in the present moment.   We choose one or the other based on whatever mechanism is involved.   Even plants show signs of making decisions between choices available to them (see for example this 40 minute presentation).    The distinction I’m drawing is based on this assumption that we have choices.

A definition of history that is dependent only on the facts consistent with evidence does not need a concept of choice.   The facts just exist.   This view of history does not even need the concept of choice or free will: every event is completely determined whether it was predictable or random.   A definition of history that depends only on evidence may be quite workable with no need to consider alternatives to the facts of what actually occurred.

I am in the group that accepts the idea that we (and all living things) make choices.   In an earlier post, I attempted to distinguish the present moment from the past by suggesting the the present moment is timeless.   The recognition of time is what makes the event historical.   I’m speaking from my experience working with data instead of making a broader philosophical or scientific claim.   I’m just observing that as soon as I can get a time stamp on something, that something is no longer available to change.   For example, although I may be able to measure my height over and over again, I lose the ability to measure it at a specific time when that time has past.   The combination of observation plus a time stamp makes it historical.    This means that history starts in the smallest unit of time, or that reality ends just before that unit of time is measurable.     The opportunity to act by choice is during that instant when history has not yet started.

A definition of history as dependent on choices creates a counter part of history that I call missed opportunity.    Missed opportunity is a different concept than history.   Missed opportunity leaves no evidence.   Also missed opportunity includes all of the possibilities from all of the rejected options.   Even a single decision can result in rejecting a large number of alternatives.

Because rejected options or missed opportunities leave no evidence, there is no real way to distinguish them using facts.   This lack of evidence or data distinguishes missed opportunity from history quantitatively.    There is only one history that we can attempt to reconstruct from the evidence.   There are innumerable alternative histories that we can only speculate upon.    In an earlier post, I attempted to redefine good and evil in terms of history and missed opportunity by anthropomorphizing both.    If the consequences of a decision turn out well, I imagine that history smiles and missed opportunity frowns.   If the consequences turn out wrong, history frowns and missed opportunity smiles with satisfaction that one of the other options might have worked out better.    The alternative to history is similarly abstract.

During a process of making a decision, there is a lot of effort to consider the advantages and disadvantages of the different options.   Ultimately we choose one option and most often (in my experience at least) that decision results in good outcomes or at least not painfully bad ones.

The question I raise is how can we evaluate the effectiveness of the decision.    The only objective way to evaluate a decision is to refer to the evidence and that means to consider only what has become history that resulted from the decision.    There is a speculative option of simulating alternative histories, but even when these simulations are convincing they lack the hard evidence of what really would have happened if that choice was made.

How do we evaluate decisions?   It seems most of the time we evaluate the evidence of following the accepted practices and that the results were beneficial.     This is the evidence-dependent view of history.   All we have are the evidence of what actually occurred and either we liked or didn’t like the consequences of what happened.

We make a decision.   We diligently followed accepted procedures.   The outcome was at at least acceptable.  The decision must have been at least an acceptable one.

From an evidence-dependent view of history that is all there is to work with.   Even when we evaluate historical people, we evaluate them on their decisions.  Their capabilities are constrained by what they actually accomplished.    Historical people are dead so they are no longer around to inquire about other capabilities that they have not had an opportunity to demonstrate.    We can only measure the evidence available to us.

There is an active genre of fiction based on counterfactual history: considering what might have happened if some event had turned out differently.    Sometimes this speculation is supported by considerable research of other facts that would have constrained the possibilities.   For example, a decisive battle may have turned out differently but that might only have changed the timing of the eventual results because of other sociological factors.

This is similar to using simulations for challenging the results of a prior decision.   This later analysis of the possibility of an alternative historical choice has the advantage of information of what actually happened after that decision.    However, there are credible objections that we don’t really know what other influences may have become important if the other option were followed.

An example may be a routine physical that detects a possibility of a cancer and that results in further tests and treatment that does in fact prevent that cancer from being the cause of death.    The question is whether the same result that the cancer would not have been the cause of death even if the treatment were not provided.   We can simulate the known information about the diagnostic results to have some confidence that indeed there was a high risk of killing the patient.   But we can’t simulate how the patient would have lived his life if it had not been interrupted by the concern of cancer and scheduling of treatments.   He could have died from a stroke following a high stress meeting that the treatment coincidentally prevented him from attending.    We don’t know what other events would have been encountered in the alternative scenario.

I don’t find the counterfactual simulation approach to be very convincing.   While it may show that an alternative may have avoided some problems we actually encountered, we don’t know all of the potential problems the selected version managed to avoid that would have caused problems for the alternatives.   My impression is that I’m not alone in this assessment.   It seems we rarely invest in re-analyzing past alternatives in terms how much better it would have been for the past.   Such an approach has limited value in any case because the past is lost to history.   We can’t change the events resulting from our selected decision.

Still I think this is valuable to have some way to evaluate the decision in comparison with the options available at the time of the decision.    The alternative approach is to demand more from the evaluation of alternatives to include very detailed predictions of what will happen when the decision is made.   These predictions may include a range of assumptions so that we may interpret the eventual reality as being between some set of predictions made at the start of the decision.    These predictions would also be available for the alternatives, but the most valuable set of predictions would be for the option that is ultimately selected.     This approach is to require additional information beyond the information needed to make the decision.    This additional information will be specifically available for later evaluation of the actual consequences of the decision with the predictions based on that decision.    This will give us more insight into the quality of the decision by determining how much of the good outcomes from the decision was based on the decision and how much was based on chance or luck.

This approach at least provides some method to evaluate the the quality of decision making independent of whether we were happy with the consequences.    I admit that this may be unconvincing.   There is always the third option of not evaluating the decision at all.   The past decision is a fact of history, completely determined by the fact it occurred.    This third option is to dismiss the missed opportunity as a something that needs to be considered.   This is the earlier option of considering history to be completely determined by the evidence of what actually occurred and not about availability of possible options.  In essence it suggests it really doesn’t matter what decision was made because we’re stuck with what we have in any case.

I prefer to have some method to evaluate the past decision.   Even if we follow good practices and made a well justified decision based on information available at the time, we still should consider whether we made a good choice.    I want to consider the missed opportunity as still being available for inquiry even though there is no way to recover that opportunity.

In particular, I want to consider the evaluation of management decisions, specifically in terms of hiring new staff.   In current employment recruiting practices, we invest in a lot of procedures to identify, qualify, and evaluate candidates to decide who to hire.   But once we hire the staff, that evaluation becomes irrelevant because from that point forward the person is member of the staff and the goal is to manage that staff.     Do we have any method to evaluate how well we did in selecting the staff and whether we may have missed an opportunity with a different candidate?

The evaluation of this decision seems to follow the pattern described above.   We look at the results and if the results turn out to be beneficial we assume we must have made a good decision.  If the results worked out well, then we have confidence to repeat the same process and practice for filling the next opening.

Although there are occasions when a new hire does not work out, in most cases new hires do find a productive role in an organization and will generally advance in their careers to some degree.    I have a hard time believing that the frequency of good choices is fully accounted by the excellence of the selection and evaluation process.   This is especially hard to believe when we note that the success of the staff is in areas that were not part of that initial evaluation.   This happens frequently.  For long term employment, this happens most of the time: a person still working after 30 years in the company is likely not doing what he was originally hired to do.

How did a very specific selection process for a specific job opening end up finding a person who finds success in areas unrelated to that original job opening?   There is the possibility of getting lucky.   But this is interesting because in most cases we are not unsatisfied with our hiring choices.    To me this implies that most people are capable of filling the roles needed from them.    Yes, we did select a good candidate, but we probably could have selected any number of other candidates who would have done just as well even if they didn’t appear to be as well qualified.

Over time, the hiring process has become much more demanding in terms of demonstrated experience, education, prior training, or certifications.   The evaluation process often includes some type of detailed questioning or testing to verify the staff has the qualifications for the opening.    We pick the best qualified for the current opening.    My question is whether this is really necessary.  It is likely that many of the ones we rejected could have worked out just as well or even better.   How do we know?

This is the point of evaluating the missed opportunity.    The only facts we have are the result of the decision we made.   We selected one staff for a position and we have evaluations of that specific staff.   All we know for sure is whether the staff is performing at an acceptable level.

As in the above discussion of simulating the counterfactual, it doesn’t really make sense to look at later successes of candidates we passed up.   There is no way to know if they would have enjoyed the same level of success if we had hired them.

There is however an approach to evaluate the candidate based on comparing actual performance with what we predicted would happen at the time of the hiring.    Job openings often have very specific requirements and the evaluation usually involves assessing these qualifications.   From this information, we could predict that the candidate will excel in exactly the role advertised and evaluated.   This is different than a performance review that finds that the staff has performed well within the organization.   He may have performed well in ways unrelated to the original job opening.    The evaluation of the hiring process should evaluate is whether the candidate performed as expected for the exact work described by the job opening’s requirements.

If we attempt to evaluate this formally, I think we will find that this evaluation is very difficult because the actual job evolved so that the candidate had little opportunity to demonstrate these specific skills he was hired for.   Instead we benefited from his employment because he excelled in other ways.    If this is what we find, then this could raise some doubts that the successful hire was not fully a result of the diligent hiring process.    More people may have been equally qualified for the position given how the position ultimately evolved.   This discovery may allow us to relax our hiring criteria, allowing us to fill the position sooner, or hire someone with some unrelated but otherwise promising talents.

To me historical data is data about decisions.   Decisions imply the concept of missed opportunities.  Missed opportunities can not be evaluated objectively because they don’t leave any evidence for use to evaluate.    We can attempt to simulate the counterfactual, but this is very unconvincing.  An opportunity to evaluate the decision is possible by comparing outcomes with the predictions presented at the time of the decision.    If things worked out consistent with predictions, then we have some confidence in our decision making process.   If the predictions don’t match the facts or can not even be evaluated, then we may question that decision making process.

We may be misled into confidence of our decision making capabilities by a string of good luck.   In the particular case of hiring staff, human nature is such that many more people will adapt to new opportunities than we allow ourselves to consider through too strict hiring requirements.

With sufficient motivation, the luck is on the side of things working out at least at an acceptable level.   After all, this assumption is the basis of most theories for managing staff that are already hired.   The staff are whoever is on board at the time and the challenge is to motivate them to excel.   The decision of their initial hiring has little if anything to do with their future success.