In a couple recent posts, my argument stumbled over confusing maturity with competence.   I attempted to clarify that the two ideas are distinct and serve different purposes.

I want to return to the original mistake of implying that maturity can be a proxy for competence.   I suggested other “bright” (well defined and controlled) observations that can indicate competence.  These include obtaining professional certifications, prior experience, and recommendations or appointments.    They seem to get closer to the goal of identifying competence to do a task, to make the right decisions at the right time and follow through when things get tough.   In practice, these observations are about as much a proxy of competence as maturity would be.

The one thing we really want to know is whether a person will be competent at a new task.   Even with excellent credentials, a person may turn out to be a the wrong person for the job.   Or the right person for the job just happens by chance to be in the right place at the right time despite not having prior history to suggest he can succeed.

Competence is one of many elusive measures that we really want to learn but may never have have direct evidence for.  Competence is similar to knowing where the stock market is heading tomorrow, or whether some crime will occur in the next 24 hours.   These are and maybe never observable except in the past tense.

Competent people often write autobiographies or have biographies written for them.   They describe certain unique traits as well as common experiences and luck.    It is tempting to read these for insight into identifying future competence.   This is similar to reading the bible narratives of individuals to figure out traits to emulate for similar divine approval, or studying past events leading up to a crime to predict a future crime (such as criminal profiling).   In each case, what is missing is all of the narratives of the multitudes who didn’t get spotlighted with back-stories.   In most cases, there really is very little unique about the back story of the subjects compared with the back stories of everyone else.  There may be no observation that reliably predicts a future outcome.   At best all we can do is to understand what led to a past outcome.

Much of the motivation for Big Data is to combine enough data across a large enough space to construct answers to questions that has no observable data.  We want answers to questions that lack any direct measurable observations.

Frequent business-intelligence examples include identifying demographic patterns that are likely to be more interested in a service, or identifying sensibilities of targeted demographics.    There is no direct observation for these ideas, but big data can expose patterns that at least provide a direction to pursue.   That direction may or may not succeed, but at least we have the motivation to pursue it.

When we seek talent for new positions, we seek competence for the position, but that is one thing that cannot by queried by either searching texts of resumes or asking questions in an interview.  Some interviewers are better than others in terms of picking out competence, but even they have a hard time transferring that competence-detection skill to others.   Competence is not encoded in the data.

For candidates (job applicants, potential contractors, business partners, etc), we look for a miniature version of big data about that candidate.   In particular, we seek out multiple dimensions of evidence of competence.   This includes prior experience, recommendations, education and certifications earned, participation in professional societies and any presentations to or awards from such societies, publications,  examples of prior work products, etc.   The more different angles to evaluate the candidate, the better.

In addition, we may be turn to broader big data (where available such as hiring records of large corporations).  We can perform big data analysis to compare evidence at the time of the interview with the results after hiring.   The patterns may indicate certain combinations that should be sought or avoided.

But in the end, the information is indirect at best for determining whether a particular choice will be competent at the particular job.   I don’t know the statistics, but my experience is that every environment is unique and a personality will either thrive with an environment that suits him, or fail with an environment that doesn’t suit him.   I never look forward to the opportunity to hire based on contents of a resume, a brief interview, and checking references.   No amount of preparation seems enough to answer what I really want to know and that is whether the person will be competent at this particular job.

As I mentioned earlier, competence is an example of a type of data that we really want to know but will never have a direct observation.   Just like predicting stock markets or future crimes, we will only have the observation after it happens, not before.   We use various data to take the place of competence and we combine multiple dimensions of data to strengthen the case, but even that derived result is a substitution for what we want to know.

When I started this post, I was thinking about competence in a political context rather than a business one.   In particular, I was thinking about several of the disappointing results of popular uprisings especially in the middle east.   In examples like Libya, Egypt, and Syria the size and enthusiasm of the crowds gave me the impression of political and administration competence that can build a stable government.    The size and persistence of the crowds gave the impression that the crowds had what it takes to be competent at politics and running government.    I learned the lesson that the crowds were a false proxy for competence.   With the exception of the initial protests in Egypt, most of the other examples dissolved into chaos that is hard to see as preferable over the predecessor regime.   Even with Egypt the group that was competent to take over actually came outside of the groups involved in the protests.

Crowd size is not an indication that there is a viable competent alternative for regime change.  Crowd size means there are a lot of people who can get enthusiastic at participating in large crowds.   What we really want to know is whether there is a viable and more beneficial alternative to the current regime.   Crowd sizes don’t answer that question.

Despite the ongoing tensions in events in Ukraine and Venezuela, I am somewhat encouraged that things may work out for both.  

My impression is that Venezuela is asking for a change of direction of a regime instead of its complete downfall.  From my distant vantage point, I don’t see the crowds as being led by some group wanting to take over.   Instead I see an insistence that their concerns be taken seriously and addressed instead of ignored or demonized.   An incremental reform may satisfy them.   At least my hope is that they find some incremental reform solution.

My impression of Ukraine is that the group who took over appear to be very competent at state craft within their borders and with Russia.    But the original protests were different than usual in that they seemed to be well organized with layers of support.  Instead of having everyone in the protests, there were people who were helping behind the lines to support the protesters, a hint that they had some access to political and administrative competence before the protests prevailed.   I’m hopeful that they have competence to stabilize the situation.

Of course in both cases, my hopes may be dashed as they had in so many other places.   I think these examples illustrate how difficult it is to judge competence and how eager I am to interpret unrelated observations as a proxy for competence.

I’ll end with another reference to my recurring theme of taking history so seriously.   One of the things we most desire to learn from history is how to find and foster competence, especially in leadership.    Despite the fact that we haven’t found the magic formula that will deliver us the competence we most desire, studying competent leaders of the past may be our best hope for finding insight into competence.