A recurring topic in this blog is the metaphysical assessment of different types of data that I describe using metaphors of light. Most of the discussion is on the distinction between recently observed data versus model generated data, the latter typically from computer simulations of scientific equations or statistical models. I propose that an optimal government would give higher credibility to recent observations over the historically derived models. Part of my reasoning is the recent availability of abundant and extensive data that should permit us to rediscover or at least to re-verify the science. Conversely, the recent observations may suggest some new discovery including one that disproves some previous knowledge. Certainly this is speculative and fanciful thinking but it is a topic that I find fascinating, thus the repeated returning to the topic.

The basic taxonomy of data involved metaphors of brightness. The brightest data is direct observations from a trusted sensor of some specific phenomena. The darkest data is the simulation outputs given speculative inputs, even if the simulation is thoroughly tested and is based on well-established science. Bright data is good. Dark data is not so good. Most data is in between where some direct observation is massaged or cleansed to derive some phenomena based on observations of other phenomena. I call this dim data and there is a scale of dimness from very bright to very dark. I also allowed for other variations including spark data that is brief unexpected data that often is deliberately introduced to distract attention, ultraviolet or open secrets that are observed data that we we don’t permit being used in considering policies.

As illustrated above, I uncritically used the concept of a scale to describe the various metaphysical or metadata forms of data. While I can define specific categories such as fully bright or fully dark, I presumed a continuum in between these categories, and this continuum can be ordered as more or less bright. It occurred to me that this scale itself is a form of dark data. I expect the world to be well ordered so that I can always compare relative dimness of two forms of dim data, and also that between any two distinct levels of dimness I can expect to find some third level of dimness that lies between the two. This scale is dark data in the form of a theory that the metaphysical properties of data are sortable into a unique and unchanging order.

I can argue that is not true, but that argument would be equally dark. I don’t see a way to observe a sorting order of different qualities of different observations. Given a certain dim data that massages an observation with some scientific derivations, sometimes I would prefer it over other dim data, and other times I would prefer to discard it. There is no absolute ordering of dim data sources. Almost all of the data we have in databases fall into the dim category. Even something as simple as brightness is actually a calibration that converts some sensor’s voltage or current into a brightness measure. That calibration introduces some darkness to the data.

I need to revisit my model to eliminate the implicit scale between dark and bright data. There are only three categories: bright, dark, and dim. The last category makes up the bulk if not entirety of data in our data stores.

My explanations for preferring bright data over dark data has little practical benefit. Most data is dim and there is really no absolute way to sort dim data on a scale from dark to bright. These explanations are simply an allegory to avoid stating flat out that when confronting a conflict between observation and science, I favor the observation over the science. An extreme example may be a measurement of the speed of light that exceeds by a large margin the accepted value. Instead of discarding it as a bad measurement or an outlier, I want to find a way to include it in my analysis. The observation may be telling us something new.

In a more recent discussion about the value of adult males as judged by other adult males. I started with a single data point of a popular topic of the high value man and how one achieves that value. From there, I immediately introduced a scale of that would include lower value men. With that scale, I insisted there must exist a zero value man and then set out to explore what characterizes a zero value man. I settled on equating a zero value man to be a villain to society. The high value man is society’s hero. In that discussion, I placed the comic book Superman as a villain because society does not know how to contact him when someone has something that needs to his assistance. The zero value man hides his skills and resources from the public so extensively that there is not even a network that can eventually reach him.

That discussion of a scale of value for men is a consequence of inserting a scale to the singular topic that there can be a high value man. The discussion of a high value man is mostly descriptive and perhaps aspirational. It stands apart from making any assessment of other men other than to exclude them from the high value club. For those discussing high value status, there is just that status. Either a man has it, or he doesn’t. There is no need for a scale, but I inserted it.

I am now thinking I should treat with suspicion the concepts that values can be sorted along some absolute scale. The scale brings a value judgement of the muddy middle, some parts of that mud is better than other parts. There is no way to observe this distinction. It is all just the middle.

Think of a simple scale such as the Celsius temperature that sets 0 as the freezing point of water, and 100 as the boiling point. Within that range we can map out what temperatures of most daily experiences. For example, we recognize that 0 degrees requires a coat, 20 degrees is comfortable, and above 40 is uncomfortable. We extrapolate the scale to below zero and above 100 to describe conditions that we would not want to experience directly on our bodies. The scale was defined in the range the permits liquid water and thus the prospects for life. We extended this range to describe conditions that do not permit water in liquid form, broadly speaking.

I do not see any harm in using the temperature scale, but I prefer the Kelvin scale that defines an absolute zero. This makes all temperatures positive and it is largely detached from the concept of liquid water. When discussing very high temperatures such as distinguishing melting points of different metals, the numeric value is what allows some mathematical model to work. Such temperature values are mathematically useful and they are measurable, but the values themselves are a consequence of belonging to a orderable scale.

Temperature does work in the way I said does not work for ordering dim data. We can unambiguously agree that one thing is hotter than another. Also, we have confidence that if there will be a temperature between any two temperatures and that third temperature will fall within the space separating those two temperatures.

We use scales for a lot of things. One example is intelligence. We recognize that some people are very smart, and other people are very slow witted. We also recognize that the bulk of the population’s intelligence lies in a closely packed space in the middle. The intelligent quotient measures this intelligence and this assigns everyone a value that can be sorted. That sorting suggests that any two people can be sorted into one being more intelligent than the other. This conclusion is a consequence of placing each individual on the scale. Prior to the scale, we would have a more nuanced and circumstantial assessment of intelligence, and probably not even use the word intelligence at all. For society purposes, we assess people as whether they are a good fit for whatever relationship we expect to have them, especially when it comes to fitness for certain tasks. If we lacked any assessment of a measured intelligence, we would find out what a person can do. If someone needs to do some task but is unable to do it, we would find a way to adjust the assignment so that he can accomplish it in a way that the entire team will succeed. Introducing even a mental construct of an intelligence score disrupts this kind of relationship building.

The IQ scale is different from the temperature scale because it places 100 at the center of a humped curve describing the population abundance at that level. The central hump falls off reasonably symmetrically in both directions with lower intelligence tending to zero and higher intelligence tending to infinity. Actual limits of human intelligence may lie in a range such as between 50 and 150, but the scale suggest the possibilities of no less than 0 and yet of much greater than 200. Personally I don’t see a value to distinguish different IQ values above or below a certain value. There is not much value in distinguishing the middle either.

While some form of categorizing intelligence may be useful, we do not have to adopt a scale. We could instead adopt a series of five buckets: average, meaningfully above or below average, and exceptionally above or below average. The choice to base analyses on a sortable scale results in certain findings that we generally find uncomfortable. IQ is an example of the ultraviolet type data: a measurement that we don’t want to use.

I was thinking more specifically about the case of autism. I don’t recall hearing the word until I was an adult and I didn’t have a good explanation until I was middle aged. By that time, the autism was described as a spectrum. At one end is the profoundly autistic unable to interact with others. At the other end is the well function

I was thinking more specifically about the case of autism. I don’t recall hearing the word until I was an adult and I didn’t have a good explanation until I was middle aged. By that time, the autism was described as a spectrum. At one end is the profoundly autistic unable to interact with others. At the other end is the well functioning autist, or Asperger syndrome, that can still be distinguishable from the general population. This defines a scale starting with 100% autistic to 0% autistic where the latter is indistinguishable from the general population.

When I started hearing about autism being a spectrum, I disliked the concept but I think I understand the intention.

In comparison with IQ where we want to avoid separating individuals by a score, in autism we do not want to distinguish different conditions by a category. There is no clear boundary between different categories and making the distinction can harm in both directions: one to deny support for someone deemed reasonably adapted and the other is to deny opportunity to someone deemed unable to adapt.

Two different measures of mental strength. One we prefer broad categories over a scale, and the other we prefer a scale over categories. There may be a good reason to approach the two differently in terms of using a scale or using broad categories. It is possible that we chose the opposite one than we should have.

IQ is measurable in a scale where the individual’s score is closely repeatable across various tests and across different times in the person’s life. Autism does not appear to be so easily quantifiable but instead we see clusters of different characteristics that could as easily be described as distinct syndromes.

Defining autism as a scale with the implied 0 being a match with the general population presents us with the possibility that the spectrum could extend on the other side of the population. There might be a negative value of autism where the individual is unable to function in society for the opposite reason than being able to connect with others. A negative autism may be someone who has extraordinary abilities to pick up cues from others. The overload of equally valid cues could overwhelm an individual to the point of voluntarily withdrawing from society.

Alternatively, the zero level of normal populations involves a limitation that allows most people to follow the cues of the majority and thus conform to the majority. Such conformity does lead to a well adjusted and highly functioning life, but it may be form of autism itself. The general population is autistic in being able to properly observe the cues of outlier groups, and especially in the ability to simultaneously process the clues from all of the different and even mutually antagonistic groups.

Defining autism as a scale suggests that some may have this capability to process all of these minority group cues in addition to the majority cues. It is impossible to reconcile this information with normal behavior in a way that permits a reasonably adapted or functional person. Such a person may rebel to the point of ostracization or punishment by the majority. Alternatively, such a person may withdraw from society, so that he does not have to act on his knowledge. That withdrawn person can appear to be autistic on the autism spectrum again, but this time it is by choice with a superior power to perceive a broader range of cues than what most of the population can see.

The above description is a prediction based on the choice to fit autism to a scale. Once we have a scale, we can conceive of what happens when extending beyond the original limits. It is like extending the Celsius scale to what happens when temperature is above water boiling temperature. The scale implies not just a sortable continuum, but also an extension beyond what has been measured before. If there is scale for autism than that scale can extend to the other side of normal behavior.

One of the qualities of autism is a hypersensitivity. Sometimes we (non autists) can recognize the stimulus and conclude that the autistic person needs to learn to not overreact so much. Often we do not recognize what is stimulating the response. In that case, we assume the autistic person is reacting to something going on inside his brain. It may be possible that the autistic person is reacting to a perception of the world that we can not perceive. I don’t see how we can distinguish these possibilities.

Perhaps autism may be the absolute value of the distance from normal population. Autism is a characteristic of being unable to have a well adjusted life with the general population. The reason for that inability is irrelevant to description of that person’s predicament. We desire to help him better participate in the general population, or at least to inform the population of the need to sympathize with his predicament. This appears to be the approach we are taking.

From a data collection point of view, it may be a detriment to miss the possibility that among the population of autistic people there can be many who perceive a more multidimensional reality that escapes the perception of most people. Assigning such persons in a single diagnosis spectrum such as autism obscures those individuals who might able able to inform us of something important and something that we are missing. This is compounded with application of absolute value function to measure autism as a departure from the normal population, where the goal is to move the individual closer to the normal.

Some among the autistic spectrum may be on the spectrum on the other side of normal.

This discussion about autism is not really meant as a description of autism. Also, much of what I write here I have heard before so it is not new, and certainly one should seek out a more authoritative discussion of the topic.

My basic point here is to challenge the notion of imposing a scale on observations. In context of distinguishing bright data of observations from the dark data of theory, I should categorize the existence of a scale itself to be a form of dark data. Once observations are assigned to a scale we are invited to sort the observations and we are invited to extrapolate beyond the original extent of the scale. Such interpolations and extrapolations may be the source of new speculations but we may confuse these speculations with actual observations. For a purely data driven decision making, it may be beneficial to prefer finding and defining clusters or categories instead of scales.