Agile development practices originally started in software development, especially in terms of developing user interfaces or user experiences.   The concept recognized an advantage of short development cycles to begin learning from users what is workable and what needs more work.    Among the various agile approaches, it is common to create a smallish team of peers who address an immediate list of problems assigned for completion during the current short development cycle called a sprint.   Older software practices started with functional requirements and scaled development duration to suit the requirements.  In contrast, the primary requirement in agile development is the defined sprint cycle (commonly just two weeks) and then identify functions that can be completed in that cycle.   The agile development may have some larger vision or road-map from which they can select new tasks, but the tasks themselves are redefined to meet both the need to complete in the sprint and the need to offer stake-holders something of immediate value: a minimally viable product (MVP).

Where practiced, agile development transforms software development into flatter more peer-like teams that define workloads according to what is needed to complete the sprint on time and with a product that will appeal to the stakeholders.   Over time, agile techniques have become so popular as to define the modern practice of software development.   The basic concepts of agile approaches have spread to other fields and even business models.  The modern workplace is becoming an agile workplace.   Teams of workers are solving short term goals with the primary requirement being a very near term deadline to deliver a valuable increment of capability that they initially committed to completing.

The idea of the MVP in agile approaches is that any cycle has potential to conclude with a successful product.   The hope is that a workable product can emerge from a sprint cycle when the original road-map imagined it would take longer.   That hope is frequently realized.

The sprint cycle itself has a consistent sequence of backlog pruning, planning, committing, executing, demonstrating, review and retrospection.   Within the cycle there is a daily sequence of group meeting (such as daily scrum) to review previous day’s work, preview current day’s expectations, and identify roadblocks that may be solved by reallocation of priorities.

From a higher level perspective, I’m intrigued by the similarity of the agile model of work with the machine-learning neural networks.    They share a lot of similar features and both are enjoying a concurrent surge in popularity on a comparable scale.   Both have boasted success stories at around the same time.   Both appear to have arisen independently, but I suspect there is some exchange of ideas between the two concepts through informal channels of popular culture.

Neither agile nor neural networks are new.

I definitely recall learning the principles of neural networks (with its partial derivatives of linear algebra) back in the 1980s and even then it seemed old although mostly a curiosity because of the lack of computing power to do anything meaningful.   Except for a few new innovations in optimization algorithms or proofs for new shortcuts, the basic math and concepts of neural networks are the same now as they were 40 years ago.

Similarly, in the 1980s, there was a push for more formal software techniques (structured programming, software modeling with pictures, functional requirements, etc) that was responding to a history of programming in the 1970s that resembled modern agile practices at least in terms of churning out MVPs.  Examples of the agility of the early 1980s practices is the very successful generation of a multitude of command-line utilities in Unix or DOS, developed independently that proved to be very valuable despite their diversity of (command-line) interfaces.   The push in the 1980s was to replace agile practices with a new embrace of the waterfall model approach of careful prior identification of requirements with mathematical of simulated modeling to verify requirements before committing the project to code.   The modern approach to agile practices are more formalized in its processes to focus on teams compared with largely independent efforts of the 1970s, but I see more similarities than differences of the two approaches.

The agile concept of time-boxing every activity turns the software process into a iterative process.  Work is defined by what can happen in a fixed-length time slot, whether that is the daily scrum or the biweekly sprint.  In either case, the job is complete when the time box ends.  The incentive to produce value (such as a MVP) at the end of each time box is to assure continued investment in future future time boxes (future sprints).   Even though a sprint produces something of value, the overall expectation is that multiple iterations will be required to find an optimal solution.   Inherent in this expectation is the optimism that each iteration will discover some new information about what will satisfy the owner.

At the end of each sprint iteration, the product stake-holders review the product, determining whether the product is satisfactory, and what new items to add to the product backlog.

Basic neural networks using linear or logistic regression learn using a process that is very similar to agile sprints.  For each iteration, there is a forward computation of activation (a mathematical calculation).  After the activation computation, there is a comparison of the computed activation with the objective.  This review can determine that the activation meets the objective, but most of the time it will identify an delta between the activation and the objective.   Finally, this delta propagates backwards to build a plan for the next iteration.

Sprints involve human activities while neural network learning involves linear algebra calculations but both operate similarly with a clearly defined iteration or time-box with a review step, and a back-propagation or product-backlog step.   Simple (one hidden layer) neural networks often accomplish surprisingly good results that we concede begins to appear intelligent if we allow a sufficient number of iterations.   Agile sprints operate similarly to one-hidden-layer neural networks.  Agile methodology also also exhibits intelligent progress if we allow a sufficient number of sprints.

At least for software development, the tools are maturing to support agile methodologies by leveling the skill requirements within the team.   Although many teams have sub-specialties such as database design, user interfaces, communications, security, testing, etc, the tools are rapidly improving to formalize and automate the specializations so that anyone on the team can perform any task.   The agile team’s value comes primarily from the number of team members instead of the specialties of the individuals in the team.   Using modern software development tools, everyone on the team is equally capable of performing most of the tasks within a sprint.   This equalization of workers is important for the success of the sprint because it assures full utilization of the team throughout the variety of tasks that occur within a sprint from initial design, to implementation and debugging, to testing, and to deployment.   In an ideal agile team, no one should be idle on any particular day due to lack of appropriate work to do.  The purpose of the daily scrum is to be sure everyone has a productive task for that specific day.

The operation of members in an agile team resembles the operation of neurons in the same layer of a neural network.   The neurons individually are initially indistinguishable and have equal access to the input (knowledge of the task) but with random allocations of weights.  In sprint teams, the randomization of weights for specific inputs come from differences in personal experiences.   This helps to allow each team member to offer distinct valuable contributions despite equal access to the same information as everyone else has.

Within a sprint, the delivered product will reflect the combination of distinctive contributions of the team.  Some portion of each contribution is learned during the sprint based on different responses of applying past experiences to current challenges.   At the end of the sprint, the entire team receives the same message about the suitability of the delivered product but each member will react independently in terms of identifying adjustments of the approaches for subsequent sprints.   The review and retrospective stages also help to expose the deficiencies between delivery and expectations.   In the planning phase the identification and acceptance of sprint backlog items is partly informed by these experiences so that the combined demonstration, review, retrospective, and planning phase acts as a back-propagation of differentials to the team.  The next sprint is another attempt at a computation of activations based on the differentials (lessons learned) from the previous sprint.

Although agile methodologies grew out of software development and in particular the development of software with significant front-end user experience requirements, the methodology has become popular for many organized activities.   The appeal of the methodology is the recognition that it is impractical to discover true requirements without trying something out.   Also appealing is the steady progress governed by time boxed activities.  In contrast to milestones of classic planning, time-box schedules by definition can not slip.  What slips instead is the requirements, but this is appropriate because we admit we don’t know the true requirements in the first place.

The operation and coincidence of both neural networks and agile methodology are very similar in terms of their coincidence in recent popularity and success.   This similarity may not be accidental but instead reflect a common cultural shift toward a recognition of how intelligence arise out of crowds instead of individuals.  Facilitating both are recent advancements in electronic computing and communications that permit efficient coordination of crowds of independent actors.

To me this similarity suggests an opportunity by merging the two concepts to come up with a new way to organize people’s activities in a corporate or government environment.   Everything could become a neural networks.  Computerized neurons use mathematical algorithms when they are available.  Humans can organize as hidden layers where the algorithms require deeper intelligence available from human experience, ethics, and cultural awareness.

In data-driven (evidence-based) decision making, I envision an agile team of data scientists each with equal access to analytic tools on a common data lake containing all of the relevant information.   Given a set of objectives for a particular sprint, each team member will tackle the analytics to contribute toward a specific recommended decision.   Although each member has the same tools, objectives, and data, the differences in prior experiences of each member will tune them to approach the analysis differently so that the final contribution is an aggregate over many viewpoints.   The intelligence of the layer comes from this aggregation of individually learned perspectives of the same data.

A neural-network inspired organization will be very different from the standard hierarchical corporate organizations of single managers of numbers of people.   The neural-network inspired organization replaces middle managers with successive hidden layers of teams who have simultaneous access to all of the contributors of the layer before it.   The first hidden layer produces a separate data trail from each contributor.  All of these trails will be available to the entire next middle layer.   The next middle layer operates similarly to the prior layer, using agile methodologies with a different data input being the products of the prior layer.  Similar to multi-layer computational neural networks, the corporate neural network will back-propagate different information through the review and planning stages of the sprint cycle.

In this model, there is a hierarchy of layers for progressively refined intelligence about some problem, but each layer is comparably staffed.  The is no unique talents that distinguish members of one layer from the one below it.  Each member of any layer is roughly equivalent in capability in terms of access to data science technologies and ability to work in agile teams.   The only differences between layers are the content of data available as inputs and the objectives that the succeeding layers provided at a the start of the sprint.  Coincidentally, this will level the organization in terms of compensation equality and redirect human incentives toward team advancements instead of individual advancement.

Although the model takes inspiration from computerized neural networks, most human-level problems (especially involving government) can benefit from organizing into neural networks where the neurons consist of human intelligence.   The actual flow of intelligence alternates algorithm-driven neuron layers with human-driven neuron layers.   The human neuron layers operate in agile sprints using algorithm-driven neuron layers to provide data inputs and also to deliver intelligence to the next human layer.

Following the lessons from computer neural networks, we should recognize that intelligence in an organizational neural network arises within the network itself.  It does not dependent on hierarchical decision makers.  Neural-network organizations have no need for individually accountable human decision makers such as managers or officers.  Such an outcome is consistent with the goals of evidence-based decision making that ideally obligate decisions based on the evidence alone and not on whim of a designated leader.

The new organization model is emerging in the so-called disruptive business plans (such as Uber or AirBnB) that exploit mobile and cloud technologies to organize a new economy described as a sharing economy.   The initial reward to the innovator is increasingly resulting in rewards to the ones doing the sharing as they learn to exploit the tools for their benefit.  In the new sharing economy there is a degeneration of the distinction between consumer and provider.  As people become more competent data scientists, they are organizing into hidden layers of a vast neural network that obviates legacy structures.   The intelligence comes within the network, not from leaders.  Eventually it can replace government itself.