This is a post about my thoughts about what should happen when someone uses story-telling in an attempt to persuade a decision-maker to approve an otherwise evidence-based recommendation.   To make my case, I’m going to tell a story where I pretend to understand something about the continuing controversy concerning the Rolling Stone article A Rape on Campus.

In my last post, I stealthily made some comments about the recent controversy that started with the Rolling Stone article on rape at University of Virginia.  Part of the popularized controversy involves questions of journalistic and editorial practices in terms of how much research should be performed before publishing a story in a particular magazine with a recognized (and respected) brand name.   As I noted at the end of that article, Rolling Stone had issued a statement to apologize to its readers that some aspects of the story had not been as thoroughly investigated as it should.

That notice of apology occurred after an earlier notice that attempted to clarify that the story was focused on telling one person’s point of view.  My post focused on this earlier notice.  I attempted to argue that that first notice would have been unnecessary if we accept a newer concept of journalism that is employed for big-data audiences instead of popular audiences.

Big data audiences are the large data stores that support ad hoc queries and more importantly large-scale analytics and visualization to obtain new insight about the real world.   To make this work, we need a lot more data, particularly from individuals and more particularly from those individuals who are reluctant authors in social media.   Consistent with earlier posts such as this post, I argue that the journalists, using their investigative and interviewing skills, are ideally suited as data sources to collect more narratives of individual experiences and perceptions.  The difference in my concept and the older concept of journalism is the market for their work.  Instead of compensating journalists by selling entertaining narratives to the general population, we need a new market that sells content directly to data stores.

Eventually, the published material targeted to a general audience will end up being found by bots that will collect newly published content and parse it for data stores designed to support analytics.  The problem with this modern compromise is that it is inefficient and never compensates the journalist for any value obtained from the larger purpose.  The controversial consequence of the Rolling Stone article demonstrates this inefficiency.   I think the article as published (without subsequent clarification or declaimer) has inherent value in presenting particular person’s perspective.  If the goal is to populate a data store with first-person perspectives (consistent with the initial clarification by Rolling Stone magazine), then there would be no need for that initial clarification or the later apology.  This is no different from anthropology or sociological case studies documenting some culture’s folk-lore or superstitious customs.  We don’t require that the folk-lore be accurate.  Instead we require the chronicler accurately capture the folk-lore told by a credible folk-lore teller.   The Rolling Stone article served that purpose for a campus culture setting.   To better understand the complete the picture of the folk-lore, culture, and customs on this particular campus (as well as what true event, if any, became the source of this particular rape story), we need many more articles like the one from Rolling Stone, but to interview more individuals and get their perceptions about the campus culture or about this particular event.  I am fine with this because this is collecting data for a big data project to eventually learn something new about the reality.

Following the initial publication of the article, there have been many more journalism articles coming out concerning this same campus and focused on the same event.  Some articles present stories from other person’s perspectives of culture or of the event.  Other articles have criticized the original article for various reasons.   From my perspective as a dedodemologist, this is all good.  These articles represent new data points.  In my taxonomy of data, these articles represent generally bright information: well-documented accounts of individual perceptions or experiences, whether those are from people involved in a particular alleged event or from people involved in journalism.   This is good data and it is being accumulated at least in terms of being found on the Internet (and I do not doubt that countless entities are archiving these into their private data stores).  With the accumulation of all of these articles, we will have access to a more complete picture of the topics raised in the Rolling Stone.  Gradually we will learn more about many issues ranging from a specific alleged event to a general observation of modern campus culture.

Again, from my perspective, I don’t think it was necessary for Rolling Stone to have even issued their initial clarification let alone their later apology.   Obviously, the story had generated a controversy to the point where some have called for Rolling Stone to retract their article.  This is another throw-back to the older journalism model of published journalism where the general reader would have access to only one account: the one in this particular print article.   The older journalist model would involve some type of retraction of their earlier article when shown not to meet some standard.   I disagree.  I think the article should have been left to stand as is.

In journalism, there is no unpublish button.   The decision making within a magazine is to decide to publish for general distribution some internal document.  They had their opportunity to withhold this information (and thus not realize any revenue from its publication) before they decided to publish it.  Once published, the article becomes part of the data.  In this case in particular, it will be harmful to retract the article because all of the other articles will lose their original meaning of responding to a published article.  If Rolling Stone retracted the article, then they will render moot all of the subsequent articles.   From a data perspective, I think this will be very unfortunate.

I would have preferred to see a completely different response by Rolling Stone to the criticism they received.  Instead of offering clarifications and apologies, they should have remained silent.   These clarifications and apologies themselves are published articles and require their own life-cycle of fact-checking and evaluation of suitability for publication.   Instead, I would have suggested incremental appending to the original report a list of links to subsequent published articles that responded to their articles.   They should present this as a list of hyperlinks (trackbacks) with no further commentary.  The generated discussion speaks for themselves.   The mere publishing of links to these consenting views is sufficient recognition of that criticism.  That would be sufficient for me.

I believe we are in a transitional period where we are becoming a dedodemocracy (a form of government dependent on data technologies).  Given that this is changing government itself, the role of journalism is also in transition.  Future journalists will need to focus on become data sources for data stores, but where their specialty is to obtain perspectives and experiences of individuals, especially those who are reluctant to publish their own accounts through social media.  The nature of the new form of government is that conclusions based on old data will motivate the collection of more data.

Compelling narratives will emerge gradually and incrementally as analytics combine the various stories to answer different questions.   The Rolling Stone article presents a lot information that can be useful for a wide range of queries: from what happened to a specific person on a specific night, to what is the modern campus culture concerning attitudes and responses to crimes.

In data-stores, the compelling stories are continually under construction.   These stories will appear later as info-graphics, some predictive or prescriptive analytic result, or some multidimensional visualization based on the data available at that time.   In a dedodemocracy, we are not going to satisfied with single isolated (supposedly representative) stories.   We will demand an overwhelming quantity of diverse stories to support the data-derived narratives.

Instead of criticizing Rolling Stone for publishing this article or to demand that the article be retracted, we should encourage (with financial or reputation-building incentives) the production of a multiple of more similar articles to feed our data stores.   Certainly, there is no popular-media market appetite for millions of similar or contrary accounts, but this is exactly what we need to complete our data collection.

We need to find new methods to encourage more data data collections to populate our big data stories with extensive data about individual experiences and perceptions.   We need a new kind of economy for journalism where that economy compensates journalists for their collecting of new data to add to data stores that will eventually provide high value narratives.

On the other hand, I still think there is a need for journalism ethics and judgement when it comes to offering a new article.  As I noted in my data taxonomy, I prefer bright data that is well documented and controlled.   I think this definition of bright data conforms the higher standards of journalism.

Rolling Stone’s initial clarification of their article reflects this standard.  Because they checking of sources to determine the credibility of the subject, the account the subject provides about her perception of events meets some level of brightness for data.   Note that determining the credibility of an authentic participant in a culture is distinct from determining the credibility of the claims within the stories that that participant tells.   The bright data is the credible account given by an authentic participant in a culture.  As I mentioned earlier, this standard is modeled on anthropology or sociology field notes that capture first-hand accounts from authentic participants in a particular study.  Such field case studies are typically associated with modern scientists documenting foreign or less modern societies.  The Rolling Stone article effectively is as a one specific case from a field study of the distinct culture of what occurs on a particular campus.

This verification that determine the credibility and authenticity of the interviewee requires less effort than the more exhaustive effort to verify the claims the interviewee makes.   Even with this lower bar, there remains a need for decision making.  The field-scientist (or chronicler) needs to decide that the work is ready for submission for publication (in the new economy that may be to add it to the common data store).  The editor of the publication (or the owner of the data store) has to decide whether the contribution meets their standards.    Even though I am advocating for an economy that encourages a vastly larger number of such first-person accounts, there remains a need for quality control and accountability.   The submission needs be traceable back to the publication that released this information to the public, and to the chronicler who made the submission.   Both will have their reputations (and future earning power) at stake with this submission.

There remains a role of a human decision maker to decide that some narrative is ready for submission.   This is what Rolling Stone confronted prior to deciding to publish this article.   Before publication, they had an opportunity to review the article and judge it according to their standards.  They had the opportunity to reject the article.  They chose the publish the article.  Once that choice was made, there is no taking it back.  It instantly provoked many other investigators to seek out more information.   I suspect Rolling Stone would rather have demanded a higher standard for the article prior to publishing it, but that is a lesson to be learned for future decisions instead of an opportunity to somehow undo the earlier decision.

The bulk of the decision to publish this article relates to one part of the article that details a specific accusation of gang rape.  The article could have been published without this information and still detail the account of one person’s experiences in dealing with authorities in attempts to report some vaguely defined incident.   This part of the article provided better verified information.  That information certainly would not have received the level attention provoked by the less verified details of the incident.

In a previous post, I described how modern non-fiction interjects fake stories as a means to illustrate the substance of the non-fiction facts of the work.   The fake story should incorporate as many facts as possible, but because it is not the focus of the work, the required brevity will require some creative writing to make the story interesting and illustrative.   Generally, readers are expected to distinguish the fake illustration from the underlying non-fiction exposition.  (In this context, the word fake can include storied with untested claims).   However, there is always a risk that the reader will interpret the fake illustration as one of the facts with comparable grounding as the other facts in the work of non-fiction.

I noted that earlier publication practice for non-fiction had no tolerance for any unsubstantiated material.  As a result this early practice produced non-fiction that rarely became popular reading material.   The earlier publication practice assigned the more readable combination of fact and fiction as a category clearly identified as fiction.   Although the readers understood the work to be fiction, they expected the author to research the facts of things that can be verified.   Alternatively, the fiction writer was often motivated by the desire to popularize non-fiction facts using fictional illustrations to keep the reader’s interest.   The older practice simply defined these works as fiction even though they could contain a wealth of factual information.   In contrast, modern authors will reorder the same mix of facts and fake illustrations in order to classify the work as non-fiction.

The Rolling Stone article matched the modern concept of a work of non-fiction.  The non-fiction was the detail of one student’s experiences dealing with a painful experience, both in terms of personal depression and of frustrations in attempts to make a complaint to authorities.   The fake (in the sense of being unsubstantiated) illustration was the detailed description of that painful experience.   That detailed illustration serves the purpose of getting the reader’s attention to consider the non-fiction parts of the article.

I think modern audiences of non-fiction are accustomed to expect and anticipate a mixture of fiction with the facts.  They recognize the fiction as a tool to draw their attention to the non-fiction part of the article.  Without the fake illustration, the article would not get as much attention.   However, the point of the fake illustration is to draw attention to the non-fiction, in this case the student’s experience dealing with a past experience.   The illustration itself should be treated as fake.

At least that is my impression of the modern expectation for works of non-fiction.   Personally, I prefer the older model of introducing these works as fiction instead of non-fiction.   The old fiction categories still carried the expectation that they will include well-researched facts to back up any details of the story that can be verified.  Generally details not directly involving character development and action were expected to be based on facts.   In earlier times, audiences of works of fiction did not demand proof that the characters exist or that the events really occurred.  However, they did demand that the everything else in the story to be facts that they can confirm for themselves.

Today, we moved this kind of mixed literature into non-fiction categories and allow our fiction to be completely unconstrained about having any factually verifiable content at all.   I nostalgically prefer the older definitions where works of non-fiction includes nothing that is not verified and works of fiction can and often will include a bounty of factually verifiable information.

Modern journalism for publications (as opposed to the future journalism for data stores) are built upon the modern notion of non-fiction.   The primary goal of journalism is to provide facts.  However, journalism must produce content that can appeal to a large audience.   The article needs some creative writing to provide illustrations of the facts.  These illustrations themselves are not the facts of the article.  For aesthetic reasons of brevity and providing entertainment, the illustrations will have some degree of being fake.   The expectation is that the audience will recognize this as a welcome opportunity to maintain interest in continuing the reading the factual part of the story.

My earlier posts (such as this one) discuss the role of story-telling in decision-making.   Especially with evidence-based recommendations (such as big data analytics and visualizations), there is a need for some additional information to persuade a decision maker to accept the recommendation.  Sometimes this persuasion includes analogies or story-telling.

As I have been writing about this Rolling Stone article, I have been thinking about the publisher’s decision to publish this article.  I offer my usual disclaimer that I am not a publisher of anything and I don’t know anything about that kind of business.  However, I will want to discuss something about story telling in decision making using this particular episode as an example.   For this discussion, I imagine myself in the role of the decision maker who approves publication of this kind of article.

As I mentioned above, this article is a work of non-fiction that includes an illustrative example (detailed discussion of a rape episode).   The non-fiction part concerned the experiences of a person who is coping with a bad experience on campus and is has certain experiences interacting with authorities to address the concerns.   The factual account of this experience in dealing with a past event suggests that there are certain limitations of how much the University will help someone in similar situations.   The discussion suggests the University could do more to help in situations like this.

This non-fiction part of the story is augmented with an illustrative scene the describes the victim’s account of the the details of the bad experience that initiated this issue. This illustration provides a context to understand the main non-fiction part of story: to understand something about the current state of mind of the victim and something about how this complaint would appear to authorities when the victim approached them.   Again, I’m generously treating the rape scene as an illustration that supports the more factual reporting of events that occurred after that experience.   In this article, the details of the rape scene were not thoroughly researched to independently confirm some details or to seek comments from the accused persons or organizations.  As an illustration of the main non-fiction topic of the piece, the illustration did not need to be as fully researched.  In fact, it could have been completely fake and still fit within the modern style of non-fiction writing.  The illustration is simply the victim’s account of the upsetting episode to provide context for the later experiences that were confirmed.

The question for the publisher is whether to publish this piece.   I assume the magazine values its reputation as a trusted source for non-fiction reporting.   Stripped of the illustration that provides details of the rape, the piece still offers valuable information about the difficulties of a student to address a complaint within the University system.  The article makes certain points about the adequacy of the support system within the University.  This article presents certain facts.  The author may recommend that this article be published.

However, without the illustration, the story does not have as much impact.  Without the illustration of detailed description of the victim’s account of the incident, the article may not appear to be worth publishing.  Few people would read it.  The illustration is needed to explain why the rest of the article is worth publishing.

In my earlier discussions of decision-making, I described cases where there is a recommended course of action based on trusted analytics of trusted data.   In general terms, the decision maker may hesitate to make a decision even after observing results of the analytics and the data.  The decision maker has doubts or fears that come from his responsibility to be accountable for the decision.   In order to get the desired decision based no data analytics, there is a need for the data scientist to provide some persuasive argument.  One of the tools of persuasion is to provide a metaphor (a story) that is outside of the actual evidence.  The story can provide a way to understand the implications or consequences of the known facts and analytics.

When used this way, the intention is not to get the decision maker to accept the truth of story.  The story is fake.  The purpose of the story to persuade the decision maker that the non-fiction-based recommendation should be accepted.  The illustration of the rape details in the above article is a story that provides a case for why the rest of the article is important.

From my perspective, even though I accept the notion that an illustration may be fake, when the illustration becomes key to persuading the decision to publish, then the illustration needs to be verified.  Based on this principle, I do not think the article should have been published.  Stripped of the illustration, the remaining information, though verified, lacks an urgent reason to be published.

I think this example is generally true of many decision making scenarios.  The data and analytics provides a certain recommendation that lacks a persuasive argument to accept the recommendation.  Frequently, when we hear of data scientists being story tellers, they are telling stories that end of cycle.  Story telling is a the final step to persuade the decision-maker to accept some recommendation.

A skilled decision maker will recognize that the story is external to the data and analytics.  When the story provides the deciding factor to accept the recommendation, the skilled decision maker will identify the story as a discovered hypothesis.  Discovered hypotheses need testing to verify that they are are not easily falsified.   In these cases, the decision-maker can demand to know what attempts were may to to test or to falsify the story.  If that attempt did not exist, then there is a need either to provide a different method of persuasion or to postpone the decision until the story (now a discovered hypothesis) is acceptably tested.

Human accountable decision-making permits the decision-maker to question recommendations and to demand answers to his questions.   To reach a decision, the decision maker must approve the recommendations even when the recommendations are backed by good data and analysis.   Sometimes even very solid recommendations lack the urgency to make a decision immediately.

I think this is analogous to what the Rolling Stone article would be if it were stripped of the details of the rape episode.  There was good reporting of a person’s attempts to address some concern within the University system.   However, that story lacked any urgency to be published in place of competing stories.  The additional story of the non-verified account of the rape episode provided the urgency that made the decision possible.

When a story becomes the key factor to making a decision, a skilled decision maker would recognize the story as a discovered hypothesis.   A fictional story would be acceptable if the non-fiction part of the recommendation was sufficient to convince the decision maker.   However, when the fictional story becomes the persuasive factor for the decision, the story becomes a hypothesis.   Skilled decision-makers will demand that the hypothesis be tested.

My impression is that Rolling Stone would not have published this article without this detailed account of the rape.  If that is the case it is an ethical obligation of any decision maker (not just journalism editors) to demand that this hypothesis be subject to tests.   A tested hypothesis is one that failed to be disproved by a reasonable number of attempt to disprove it.   The decision should have been based on a tested hypothesis.

If the non-fiction part of the article (without the details of the rape incident) were sufficiently urgent to merit publication, the inclusion of the description of the victim’s account of the rape might have been acceptable.  For example, the rape story could have been presented as a way to describe the victim’s state of mind when seeking support from the University.  However, if the article would not have urgency to merit publication without this detail, then this detail becomes part of the non-fiction of the story.  The non-fiction must be backed with evidence that there were reasonable attempts to disprove it.

In my earlier writing of the decision life-cycle, I have placed story telling at both the beginning and the end of the process.  I am suspicious of story-telling at the end of the process, even when the story is heavily decorated with facts as seen in info-graphics, multidimensional visualizations, computer simulations, etc.   The facts may be unimpeachable, but the story that arranges those facts can and should be challenged for what it is: a newly discovered hypothesis.

When the invented part of the story provides the persuasive power to make a decision, the decision maker should recognize that the story is a discovered hypothesis.  The once entertaining story becomes one of the facts.  The decision maker must insist that the told story be verified before he will allow the story to persuade him to make his decision.

Observing patterns suggest a story. Such a story is a discovered hypothesis.  A discovered hypothesis lacks any authority until it is tested.   Testing involves challenging the hypothesis with new data.  Once the hypothesis is tested, we may use it to make a prediction to form a recommendation to a decision maker.

Often, the decision maker will need persuasion to accept the recommendation.   Part of that persuasion may include inventing a new story with the intent of helping him understand the data and analysis.    There is a line between stories that illustrate the non-fictional recommendation and stories that add information to the recommendation.

By calling the persuasion process a process of story-telling we can allow ourselves to disregard this boundary.   Any story that gets a decision maker to accept an evidence-based recommendation is acceptable.   It requires diligence by the decision-maker to recognize when a story provides some key information for the decision.   When that happens, the decision maker should demand evidence that the persuasive story is true.   For stories invented for the purposes of persuading a decision-maker to accept a recommendation, the skilled decision maker will require verifying analysis of the story or he will reject the relevance of the story for his consideration.

In modern data science projects with automated data collection and analytics, the hypothesis-discovery occurs at the beginning of the process.  The modern decision maker ideally would participate at this early stage of the process to select discovered hypothesis that are self-evidently persuasive.  The following data collection and analysis that supports this hypothesis will lead to a simple decision that does not require any last-minute invention of a story to earn the decision-makers approval.   After the decision, additional invented stories will serve only the purpose of illuminating the underlying non-fiction of the data and analysis.

The previous paragraph describes my ideal of how decision making would work in the data-driven age.  My impression is that the decision makers within Rolling Stone did participate early on to define a self-persuasive hypothesis (Universities have trouble addressing rape complaints) and they did publish when this hypothesis was tested.   That leaves the untested hypothesis of the actual account of the rape incident.  That account was included in the article in such a way that is appears to be one of the facts instead of an illustration of the remaining facts of the individual’s struggles to cope with a problem and attempts to find a remedy within the University system.

I mentioned before my preference for the older model of non-fiction that had no tolerance for any fake information even if it would make the work more appealing.   This example demonstrates the ambiguity of interpretation that can occur with the fake (as in unverified) illustration.   For the same reason, I am very suspicious of creative story-telling at the end of a data project (through compelling info-graphics, multidimensional visualizations, or animated simulations).  These stories are ambiguous as to whether they are facts or fakes.

A skilled decision maker would not accept fake stories as facts.