I recently watched this presentation on the Kanban concepts for self-organizing teams. The presentation was very effective with two complementary speakers who collaboratively built a board to describe the production of the presentation itself.  I thought that was very effective approach and the presentation seemed natural and realistic.

I want to focus on one point made about the team interaction when discussing the board.  This point is that the team discussion should focus on the physical board with sticky-notes and not on each other.   In particular, they discussed applying this to the daily stand-up exercise characteristic of the daily scrum.   They referenced a blog post that makes a good case of what goes wrong in typical daily scrums by focusing individuals instead of the objectives.  In particular, it was reassuring that it is common experience that daily scrums last 1 hour despite efforts to stick to 15 minutes.   I agree with the point that the problem is at the core of the round-robin discourse of the three questions (what did you do, what are you planning to do, and what is blocking you) that focuses on individuals instead of the objectives.   Daily scrums are often a waste of time.

The kanban board visualizes the current tasks and individuals using different icons and avatars arranged in terms of various activity stages.   The bulk of the above presentation discussed applying the overall approach, but I want to focus on this specific image (slide 7 of second session) of the team gathered around the board with everyone’s attention on the board instead of each other.  Even when someone is speaking, people are looking at the board and not the speaker.   Within this image, the ghost of intelligence is captured: intelligence is in the space between the team and the board.

Agile software practices vary a lot according to how they arrange concepts like kanban, but they share a general assumption that the best way to learn the requirements is through the process of attempting to make something new.  In contrast, tradition project management approaches places more burden on prior knowledge and theoretical confidence prior to committing to a project.   The agile manifesto succinctly describes the preference for the continuous knowledge discovery over emphasis on prior knowledge.   The assumption is that the dynamic team activity of building a project will lead to more effective solutions than the approach that relies more heavily on prior knowledge.   In practice, the agile approach appears to be very successful.  I believe its success is transforming the entire economy, and certainly the workplace culture, to emphasize the primacy of continuously engaged teams focused on particular short term goals.

Where agile approaches are successful, a good part of that success emerges from the very frequent team discussions that examine their processes in depth.   The kanban boards provide the visualizations of those processes so that the team can identify smarter decisions about how to improve the process goal of delivering value to the customers.   For one example, very often the improvements involve addressing work-queues or bottlenecks instead of optimizing productivity of sub-teams.  The team discussions around the kanban board discover the intelligence needed to identify the queues or to identify ways to reduce the impact of those queues.

This is not a discussion of kanban or agile practices.  Instead, I’m reflecting on the demonstrated emergence of intelligence from the team.  The dynamic teams are most effective when they target their collective attention on a visualization of the problem.   New items appear in that visualization as new ideas come up.   Over time the visualization becomes more accurate description of the process.   At the same time, the process becomes more streamlined and efficient.   The two parts (accuracy and improvement) are intertwined: improving the process makes it easier to understand, but also understanding the process makes it easier to spot future improvements.   This form of intelligence is a dynamic process that is not possible with more traditional approaches that isolates the accuracy of knowledge from the process of improvement.

I am thinking about the source of intelligence in these teams.  The teams are doing something intelligent: discovering more accurate depictions of their processes and discovering ways to improve the process.  Given the success of agile practices, it appears this form of intelligence is more effective at these sorts of problems than traditional approaches the place a premium on individual expertise.   Even though the intelligence of the agile team emerges from the interaction of admittedly intelligent agents, the new intelligence appears to reside outside of the individuals.   As I mentioned earlier, the daily scrum meetings improves when the topic shifts from what each individual is doing to how each activity is contributing to the objective.   The key part is the shift of attention from other intelligent agents in the team to the external visualization of knowledge such as depicted on kanban boards.

While contemplating team intelligence, I’m also thinking about machine intelligence.   The recent growth in popularity of machine intelligence comes from its recent successes.  Using basic statistical concepts and mathematical computation, machine intelligence has demonstrated ability to uncover new knowledge in addition to becoming better at mimicking processes we imagine is unique to human intelligence.   In my last post, I described the resemblance of the success and popularity of both machine learning and learning from agile methodologies.  This correlation may be just spurious, but I suspect they are inter-dependent.  I imagine a causal connection between the two.

The popularity and success of both involved an emergent intelligence from outside of the intelligent agents (members of agile teams, or algorithm-designers of machine learning).  In both cases, the intelligent human agents sets up the conditions for intelligence to emerge, but the intelligence appears to be outside of the human actors.  That intelligence speaks back to the human agents through visualizations.  The intelligence of machine learning arises from adjusting weights for various features.   The features are comparable to the human agents in agile teams.  The weights are comparable to allocation of rewards and penalties with discretion according to the desired external outcome.

For machine learning, it is not controversial to recognize whatever is emerging from computations is not coming from a human mind.   I propose that the source of intelligence from agile teams also is not coming from the human minds of the team.   In both cases the agents (human minds or algorithm features) interact to create something that humans recognize as intelligent.   The intelligence emerges from the interaction.

In general terms applicable to both machine features and agile team members, that interaction involves acquiring new experiences.  The experiences that count are precisely involving the immediate objective.   There is no need for prior experience on unrelated projects.  This is obvious in the case of machine learning where initialization involves either starting at zero, or starting with some randomized set.   It is more implicit in the team-building trajectory of the agile teams to condition the team to focus on the immediate objective as opposed to objectives outside of the scope of the project.

In both cases, the intelligence comes from the diversification of experiences.   For machine learning, the diversification is explicitly sought though iterations with discretionary allocation of weight adjustments for various features.   For human teams, a similar process occurs but usually it is subtle and not explicit.   What counts for emergent intelligence in agile teams is the creation of new diversity of experiences while focusing on the team’s objectives.  The agile process itself diversifies the experiences of the team members.  Arguing by analogy, I suspect that this diversification occurs by discriminating between different members according to their relative contributions at various parts of the problem.  The discrimination comes in form of rewards or penalties in response to individual contributions.

The inherent difference between features (in machine-learning) and humans (in agile teams) is that the feature definitions are static while humans are malleable.   Humans on teams can change in response to their received rewards or penalties.  In contrast to machine learning, the features that are humans are capable of redefining their definitions at the same time that the process is trying to allocate weights to the different features.

Within a particular learning phase of machine learning, the definition of features remain stable.  Data scientists may use multiple iterations to adjust features to find better solutions, but within each learning iteration the feature definitions remain static.  Despite this constraint, machine learning has demonstrated remarkable capabilities for conforming to the training data sets.

Meanwhile, agile teams of humans also demonstrate a capacity of learning to conform to its training set of some business objective.   However, different attempts at agile practices have had various degrees of success.   Some teams fail to learn while others demonstrate remarkable success.   Sometimes the success is recognizable even if the teams do not enjoy business success.   We can still see the impact in making measurable improvements in their processes.   Other teams will fail to make even these internal improvements.

Continuing my argument by analogy, the treatment of individuals in the team may be a possible factor that determines the which agile teams will be effective at learning.   The more successful agile teams may treat their membership similar to how machine-learning algorithms treat their feature topology.  A success pattern for agile teams may be to restrain the participants from changing their roles.   The ultimate learning of process improvements involves finding the optimal arrangement of more-or-less statically defined members.

I think this is illustrated in this training course that provides some mathematical justification of the benefits of lean or agile software practices.  One of the messages in this course is that optimizing a particular part of the process, such as making one group more efficient, can result in lower performance overall.   The penny-game in lesson 4 illustrates that improving individual performance (using both hands and working on larger batches) is much less optimal than working with smaller batches without that improvement.   Later lessons go on to describe other simulations that demonstrate the problem of queues that accumulate variability.   While the lessons make their point about focusing on the overall process, there appears to be a secondary message that the components parts (team members) should adapt more slowly than the changes to the process.

In the agile team demonstrations, there is an discussion of “T” shaped individuals: people who have deep expertise in one area but who have some working knowledge of the areas immediately before or after their roles.   The explicit message in the lessons is that T-shaped individuals will learn to adapt.   However, they still remain T-shaped: their primary expertise remains the same.   The T-shaped individuals behave similar to the machine learning features in that they remain rooted in their definitions.   The analogy is even closer for convolution algorithms that permit communication between adjacent features.   Convolution-partnered features are similar to T-shaped agile members.

I hypothesize that successful agile teams effectively constrain their team members to their assigned areas of expertise.  The agile optimization is a process that learns the optimal arrangement of these units of expertise.   That learning is more likely to succeed if those units of expertise remain fixed.   To clarify this point, I recognize that teams encourage continuing education to keep current on their expertise.   What needs to stay fixed is the area of expertise (such as user-experience story-boarding, user-interfaces, business-logic, data stores, or data-mining).  Obviously individuals may grow over time, but as they do they need to occupy different roles (in same team or another team), leaving a vacancy for the previous role.  The successful agile teams manage to learn by keeping the roles relatively stable during the learning-phase analogous to the training phase of machine-learning.

Unsuccessful agile teams, or teams that eventually reject agile practices, may be those that refuse to restrain their members to stick to stable roles.   These teams may choose to emphasize older practices of staff development through frequent role rotation.  One such practice may be to assign a staff to a role or a task for a set amount of time based on the goals of developing that staff member instead of optimizing business processes.  The goal is to build future managers or leaders who can manage a company using similar old-style companies.   In these schemes, there isn’t enough stability within the teams to effectively learn optimal arrangements.  Successful agile teams benefit most with teams that remain stable over multiple iterations where the team membership and roles remain more-or-less fixed.

Agile techniques optimize the flow of value through a system by eliminating queues between handicapped roles.  The optimization is not helped by improving performance (removing handicaps) of individuals roles.  In fact, as the prior-mentioned courses emphasize, such individual level efficiency improvements confound the goals of optimizing the flow of value.

The best agile teams may be the ones that operate more like machine-learning algorithms.  When the membership approximates the fixed features during a training phase of the machine-algorithms, the agile teams will more likely succeed in finding a path to its goal of optimizing flow of value.   Agile practices are shown to work.  Such practices are integral to many modern successful businesses.   Similarly, machine learning algorithms are showing their success.  At the time of this writing, it appears that both will play a large role in the future economy.

However, their similarity suggests something that bothers me.  Agile practices imagine humans as T-shaped features: fixed in a particular role with some awareness of its peers (similar to convolution-connected features in machine-learning).  Older management practices, that we now consider inferior to agile practices, emphasize more malleable and rounded staff.  Adopting agile practices requires abandoning the older practices that strive to develop staff over time for the sake of preparing future managers and leaders.   Agile practices gain their success without need for such leaders (or at least not in the same numbers).   Perhaps this is great for business.  I worry that it blocks the individual’s opportunity to grow.

Sure it is great that the business process is optimized for the right work load between different specialties of software development.   But we expect the 25-year-old data-store expert will continue to find comfort in this “T” for the duration of his career.   Unlike machine-learning features, eventually humans need to grow.