This article describes the consequences of uncertain work hours resulting from what is described as just-in-time scheduling.   The discussion is specific to retail industries that are using data analysis tools to predict the needs for staffing based on historical trends.   These tools leads to more flexible employment contracts where the employee agrees to work when the work is needed.

As I read the article,  I see a trend we should take seriously in that it will increase in the future.   I also see a need to pay close attention to the underlying assumption of basing these decisions on historical data.

There are two human costs to this just-in-time scheduling approach.

One is that irregularity of pay due to hourly pay where billable hours vary weekly.   The data analysis tools allows for prediction at a weekly or even daily level.   This is similar to older techniques of seasonal employment such as a garden shop hiring more staff during the planting season.    The difference is that the older definitions are easy for workers to anticipate.   The new definitions are due to data patterns from queries of multiple years of same-store sales.   The worker has no access to this underlying data and thus has no ability to anticipate what kind of hours will be available in the near future.

The second problem is the loss of any control of scheduling even within a work week.   The algorithms for data analysis are imperfect.    Actually, the problem is that the past data does not guarantee future needs.   In a physical store, the population who uses the store will change over time as the nearby community changes, grows older, or experiences different economic or fashion trends.    Contrary to established trends, the retailer may experience an unexpected surge in customers thus requiring them to call in previously unscheduled staff, or the predicted traffic does not appear and they need to let the staff go earlier than expected.

People working in retail are trying to manage their lives, paying monthly bills, arranging for child care, etc.    With this type of scheduling retail jobs become increasingly impractical for people who need steady income.

This phenomena is a consequence of better data collection and analysis.    It is easy to collect data of in store sales and on-floor customers.    This data offers the opportunity to have the right number of staff available to support the customers.    At any time there is an optimal number of sales associates for a given number of customers, not just for sales but for customer satisfaction.    Customers do not appreciate being asked by a dozen different idle associates whether they need help.    Ideally, a retailer would like to have the optimal number of associates available for the number of customers present.

This scheduling approach asks sales associates to manage their lives as being in the store’s warehouse to be brought in when needed.    The problem is worsened by the imperfection of the data models to predict the future.   There is no certainty of how many hours are going to be needed on any particular day.

It is not surprising that retail stores will be early adopters to this approach.    Considering the availability of historical data analysis tools available, this should be predictable.   Despite that fact, I was surprised to learn of the phenomena in this article.    Now, I think it is obvious.    Inevitably this is going to define employment for areas far beyond retail or customer service.   Eventually the concept of steady full time employment may be rare and anachronistic.

In the above article, the associate is making above average hourly rate and this rate is probably justified because that time is being more productive with available customers to serve.   The older predictable scheduling approach meant working longer hours at a lower hourly rate because a lot of that time will be idle time with respect to making sales.   I make the generous assumption that the two rates will eventually balance each other out over the long term so that annual accumulated pay is about the same fraction of total annual sales.    It is possible this approach could lead to more income by providing additional incentive to attract more customers to justify more hours on the job.   However, I suspect at least at this early stage it is a net loss of annual income for the typical sales associate.

I can think of many examples where people are employed full time but their services are not needed for that entire duration.    For example, in office environments, people are expected to cover their assigned office hours to conform to a standard work week.   During the day, the work load varies from being overwhelmed to being completely idle.    The employee’s pay is averaged over these periods to support the traditional notion of a standard work week.   In contrast from the employer’s perspective, its business suffers lost productivity when the workload overwhelms its full-time staff, and it has to pay for no productivity for idle periods.  When data technologies similar to the above retail solutions become available to predict work load requirements at an office, the businesses will begin to use it.   I think it is inevitable.    The standard predictable 40 hour workweek will be an obsolete concept.

As illustrated in the above article, the transition is going to be painful for the workers.   We need to figure out new ways of organizing our lives around an uncertain work schedule.    Some of this may be solved by changing they way we budget our finances.   For example, living paycheck to paycheck is increasing difficult if that paycheck loses its predictability.

Much the changes will be cultural and social changes that will take longer to accomplish.    For example, day care centers will need to adopt a similar just-in-time model to accommodate their just-in-time employed clients.    This is another reason why this phenomena will spread throughout the economy.    The effects will cascade throughout all of society to accommodate the ones affected earlier.   Just in time employment of retail associates makes a market for just in time services for those associates.   In turn those services will make a market for just in time suppliers.

Eventually most of consumers in our economy will be employed using just-in-time scheduling made possible by imperfect data analysis of past trends that require flexibility to burst or truncate hours as unexpected events occur.     Such a consumer economy will look a lot different than the current model where the majority of people have predictable work schedules and paychecks.

The future consequences are likely to surprise us.    I imagine a scenario where the retailer satisfied with its just-in-time employment suddenly finds all of his historical data rendered obsolete because all of its customers are just-in-time employed.    People begin shopping at irregular and unpredictable times.   Customers begin spending in unpredictable quantities.  The data models will require many years to settle out.   It is possible the entire project fails to the extent that forces the retailers return to their original models.

One of the problems of just-in-time scheduling is that the only opportunity most people have to make money is to sell their hours.   An idle period that releases an employee will benefit the employer’s costs but will deny that employee an opportunity to sell that hour.    A person’s hours are a perishable commodity.   An unemployed hour is forever lost opportunity for income.   Even if the hourly rate is adjusted to average out in an annual basis, the unemployed hour will still represent a missed earning opportunity for the employee.

This could be an opportunity for the employee to sell his unemployed hours to a different employer.   As the above article points out, this is opportunity is denied by the conditions of employments that require that the employee be available on short notice to fill unexpected needs or to be release on short notice.    Inevitably there will be conflicts where multiple employers will need the same person at the same time.   In effect, the contract prevents or severely limits the employee’s ability to sell the hours elsewhere.

This may work itself out over the long run with appropriate changes to lifestyles and compensation rates.  In the short term, there will be many difficulties.

One of the problems is that the employees are not easily replaceable.   Often employers prize certain employees either for the exceptional abilities (such as closing a sale) or for their ability to draw in customers (who build a relationship with that employee).   More commonly, the specialized training and dress codes of associates is part of what gives the marketing distinction for a company.    I think the examples in the above article fall in this category.   This business model demands both exclusive access to their specialized staff and flexibility for how many hours to employ them.

Perhaps eventually we will adapt to a reality that commits us to exclusive employment arranges for a fickle employer.    A possible adaptation may require some method of redistributing incomes among communities to smooth out the ups and downs of individual paychecks.

Alternatively, we may adapt by inventing new kinds of employment.   For example, retailers may get their staff from an agency that has a large pool of steadily employed staff who can take on the role of the particular employer.    This will require from the employees cross training (or certification) for participating as associates for a variety of employers and for the employees to work over a larger geographic area.

There are a lot of other ways for things to work out.  Whatever emerges will be very different than past or even present arrangements.   Assuming that just-in-time part-time employment is a trend that will grow in the future, then we should prepare for the changes it will bring.

The limited information in the above article suggests several problems from a data science perspective.

The first problem is the employee’s lack of access to the historical data that determines the expected work load for a week.   Part of the employment decision is the expected income.   In this case, that income is determined by the historical data.   To be able to make a fair assessment of the job opportunity, an employee should have access to that data for his own queries to see what the future income possibilities are.    This reinforces my earlier posts on the need to introduce data science education as part of early education.   Everyone who enters the workforce should be able to use data systems, construct their own queries, and come to their own conclusions about the results.    Employers should make these tools available to their employees.   Employees can use this information to better arrange their own expectations of free time and income or to decide that they may have better options elsewhere.

The second problem is the inadequacy of the predictive power of the available data.   One of the problems illustrated in the article is that employees may be called on short notice to fill a burst need, or they may be unexpected sent home due to unexpected low demand.    To some extent this is inevitable due to the future not repeating the past.    However, part of the problem is the inadequacy of the data being queried.   Perhaps there are problems with the historical data.  For example, the store may be using sales receipts as a measure for number of customers in a store.    A better option would be a traffic counter at the door to measure the number of customers on the floor at a time.   Alternatively, the in-store data may be incomplete such as not including the transactions involved in returns or exchanges that take more time to process.

Another potential problem is that the store’s data system lacks enough information.   A store catering to a young customer base may be in a stable neighbor of aging customers.   Such a store may be hampered by the lack of demographic information about its customers.

In store data collections may be insufficient for the task.  The store could coordinate with other data sources to improve predictive accuracy.   Those sources may include their competitors through some mutually agreeable arrangements to share certain types of information.   It may include data from government or marketing firms about the serviced neighborhood.    Merging and associating these various data sources will require much more investment in data scientists than exploiting preexisting sales receipt data.

My reading between the lines of the above article is that the stores relying too heavily on their isolated proprietary data.   They should be seeking out external data in order to improve the predictive power of their projections of staffing needs.

From the employee’s perspective there is room for negotiating higher hourly rates in exchange for their exclusive commitments and for their flexibility to accommodate the employer’s needs.   The article suggests that there is about a 50% premium in the subject’s hourly rate compared to standard steady-scheduled retail positions.   This suggests that perhaps this negotiation is already taking place.   There is probably room for more negotiation with better information available to the employee about the value of the flexibility.    There is great value to the employer to have enough staff when customers are present.  Their ability to draw in that staff when needed represents a higher value on the hourly rate than someone who is on the clock through a fixed duty period.

This hourly rate negotiation can follow old traditional models where the employer sets a rate sufficient to get employees.   This traditional model was optimal for finding employees whose reliability was assured by a predictable work schedule.   While it may be possible to get someone to sign an agreement to change schedules on short notice, it is likely that many may not cooperate when that time is needed.    The problem is that the commitment was inadequately quantified to the employee.    There is a more extensive sharing of information with the employee to quantify of how often and when unexpected schedule changes occur.    With that information, the employee can make a better decision on what kind of hourly rate will be sufficient incentive for him to make that kind of commitment.

I think it is inevitable that this trend of using historical data to adjust work hours to match expected demands on a weekly or daily basis.   Personally, I think it can ultimately be a good development for both the employee as well as the employer, but most of the above discussion followed the sense of the article that for now it is benefiting the employer more than the employee.   The long term success of this trend for both parties will depend on improving the quality and sharing of data.   In particular, if employees had better access to the underlying data they can optimize their employment arrangement by having more confidence in how their schedules will be affected and by having more leverage to negotiate a fair value for the flexibility offered.

Update 8/21/2014: Another recent article discussing the same phenomena can be found here.