This post is part of my collaborative research with Shinsei Bank on highly-evolvable enterprise software.  It is licensed under the Creative Commons Attribution-ShareAlike 3.0 license.  I am indebted to Shinsei Bank for supporting this research and to Jay Dvivedi for mentoring me in the art of enterprise systems.  All errors are my own.

The fourth edition of Herbert Simon’s Administrative Behavior contains a brief section titled “Applying information technology to organization design”.  In the industrial age, Simon says, organization theory was concerned mainly with how to organize for the efficient production of tangible goods.  Now, in our post-industrial society, problems of physical production have diminished in importance; the new challenge is how to organize for effective decision-making.  Simon characterizes the challenge as follows:

The major problems of organization today are not problems of departmentalization and coordination of operating units.  Instead, they are problems of organizing information storage and information processing–not division of labor, but factorization of decision-making.  These organizational problems are best attacked, at least to a first approximation, by examining the information system and the system of decisions it supports in abstraction from agency and department structure. (1997, 248-9)

In essence, Simon proposes that we view the organization as a system for storing and processing information–a sort of computer.  Extending the computer metaphor, organizations execute software in the form of routines (March and Simon call them performance programs).  Department structure, like the configuration of hardware components in a computer, has some relevance to the implementation of decision-making software, but software architects can generally develop algorithms without much concern for the underlying hardware.

If we take the organization-as-computer metaphor seriously, the enterprise software problem becomes broader, deeper, and more urgent.  It no longer makes any sense to consider as enterprise software only those routines executed by electronic computers: decision-making behavior is driven by both human and electronic routines, often tightly interwoven and interdependent, that together comprise the organization’s software.  This is the phenomenon that I attempted to describe in my dissertation Computer-Assisted Organizing.  Unfortunately, however, we do not seem to have any particularly effective methodologies for designing the electronic component of enterprise software, let alone the software for human-electronic information processing systems, which tend to be large, complex, turbulent, and conflict-ridden.

Applying industrial engineering techniques

Jay’s approach to the problem has been to apply industrial engineering techniques to information system design.  In contrast to Simon, who seems to assume that the problems of producing tangible goods and the problems of decision-making are fundamentally different, Jay seeks to manufacture “decisions” in much the same way as tangible goods.  Nissan manufactures automobiles; Shinsei manufactures credit approvals.  In an earlier blog post, I termed this system design methodology information assembly lines.

It is not my purpose here to investigate the fundamental similarity or dissimilarity of physical goods and informational outputs; rather, I would like to explore the idea that perhaps well-developed manufacturing techniques may be applicable to the design of organizational information processing systems.  Of course, the techniques of modern mass production developed over several decades–perhaps more than a century if we include Japanese innovations after World War II–so the most I can hope to achieve here is to pose, and begin to motivate, the question of whether applying these techniques to the design of information systems might be productive.

Decomposition is one important technique from physical production that appears to be similarly effective in information manufacturing.  In a recent New York Times article entitled “When the Assembly Line Moves Online“, Randall Stross wrote about how IT systems can be used to decompose work into tiny, simple, repetitive microtasks and deliver these tasks to workers over the Internet.  Jay also emphasized decomposition at Shinsei; for example, he transformed mortgage loan processing from a large, complex task performed by an expert loan approver (the informational equivalent of craft production) into a computer-controlled information assembly line.  The line uses a sequence of simple tasks performed by minimally-skilled workers to manufacture lending decisions.

Assembly lines

In physical manufacturing, decomposition and physical isolation of subtasks were combined to great effect in the assembly line.  Physically separate locations are specified for the performance of particular tasks, and conveyors move the work-in-progress from one location to the next.  Spatial distribution of work results also in temporal distribution, since work-in-progress cannot be in two locations simultaneously.  The conveyor system enforces consistent temporal ordering of tasks as well as consistent motion and orientation of the work-in-progress.  As a result, the assembly line has far fewer degrees of freedom than traditional craft production, where tasks can overlap in space and time, and the ordering of tasks can change if not strictly controlled.

Seen from the perspective of the constraints that it imposes on the production process, the assembly line can be understood as a particular kind of representation for the problem of manufacturing tangible goods in large quantities.  As Simon observes in The Sciences of the Artificial, “Every problem-solving effort must begin with creating a representation for the problem–a problem space in which the search for the solution can take place” (108).  Effective problem representations resemble good theoretical models: they capture the essentials while stripping away extraneous details.  “Focus of attention is the key to success–focusing on the particular features of the situation that are relevant to the problem, then building a problem space containing these features but omitting the irrelevant ones” (109).  The assembly line representation defines a problem space with the following characteristics:

1. Fixed physical distribution of tasks:  Every task is assigned to a single, specific location.  A given task can only be performed at the location to which it is assigned.
2. Fixed path for work-in-progress:  A conveyor system carries the work-in-progress, fixing its orientation and motion through the production space.
3. No branches:  The conveyor system has no branches; every piece of work-in-progress follows the same path through the production space.
4. Temporal isolation:  Since tasks are physically isolated, two tasks cannot be performed simultaneously on the same work-in-progress.
5. Fixed temporal sequence:  The fixed physical distribution of tasks and the fixed path for work-in-progress ensures that tasks are performed in the same temporal order.

These constraints rule out some–indeed, probably vast spaces of–conceivable product designs and assembly plans; for example, the assembly line cannot handle designs that require mutual adjustment during the performance of two separate tasks.  The result is a dramatically smaller, simpler, and more modular problem space.  Industrial engineers can divide the production problem into a sequence of physically and temporally isolated, well-structured subproblems.  For each subproblem, the input and output are well defined.  When the line becomes operational, workers at each location can take for granted the path and orientation of the work-in-progress as it passes through the workspace, the set of tasks that will have been performed on the work-in-progress before it arrives, and the task to be performed.

Imposing physical constraints on software design

Software engineers are generally not concerned with the physical organization of an information system.  Like Simon, who suggests that information processing problems can be handled “in abstraction from” the underlying organizational structure, they develop applications “in abstraction from” the underlying hardware architecture.  Provided that a database has certain properties, software engineers normally do not care whether it resides on a single server or exists in thousands of fragments distributed across thousands of servers scattered around the globe.

The ability to develop software in abstraction from characteristics of the underlying hardware has delivered enormous benefits by enabling software engineers to focus attention on high-level problems.  However, perhaps a case may be made for strategic reintroduction of physical constraints on information systems.  When we adopt the view of decision-making as manufacture of informational goods, the architecture of existing enterprise systems resembles traditional craft production.  Workers and tools (users and applications) cluster around giant piles of work-in-progress (databases).  Although some rules and discipline may help prevent workers from getting in each other’s way, everything tends to end up interdependent with everything else, and complexity gets out of control.

In light of the dramatic success of physical assembly lines in the context of mass production, it seems reasonable to ask whether imposing analogous physical constraints on the design of information systems might be similarly effective.  For example, what if an enterprise obliged its software architects to adhere to the following rules:

1. Decomposition:  The process of creating every output must be broken into several distinct, well-defined tasks.  Decomposition must continue until the tasks satisfy an objective test, such as the number of weeks required to develop software that implements the task.
2. Fixed physical distribution of tasks:  Every task must be assigned to a single, specific physical server.  A given task must be implemented in a standalone application, and the application must be installed and executed only on the server to which it is assigned.
3. Fixed path for work-in-progress:  A transportation system controls the movement of work-in-progress, dispatching the work-in-progress to one server after another.
4. No branches:  The transportation system does not support branching; every piece of work-in-progress follows the same path through the cluster of servers that make up the production space.

I expect that the immediate reaction of most software architects to these rules would probably be strongly negative; many might say that the rules are impractical to the point being infeasible.  (I imagine that early production engineers expressed similar misgivings about the assembly line.)  For example, the prohibition of branching may seem impossibly restrictive.  The flexibility of actual assembly lines may allay some of these fears.

During a recent visit to the Nissan Oppama plant, I saw several models of gasoline automobiles and the new electric Leaf being manufactured on the same assembly line.  Instead of using branches, contingencies were engineered into the line.  At one workstation, depending on whether the conveyor brought an electric car or gasoline car, the worker would install a battery pack or a gas tank.  Feeder lines were designed to bring the appropriate parts to the appropriate workstations at exactly the right times.  The constraints do create new challenges for engineers, but, if the analogy to physical manufacturing holds, the constraints should also increase modularity and reduce complexity.

Scientific management

To inform my exploration of the relevance of physical production techniques to information system design, I turned to one of the foundational texts in industrial engineering and organization theory, Taylor’s The Principles of Scientific Management.  To my surprise, I found many ideas in the text that resonate with my findings.  Below, I review Taylor’s work from the perspective of information assembly lines and my research at Shinsei.  Page numbers refer to the 1998 Dover Publications edition.  Although I am aware of the substantial literature on, and debate surrounding, Taylor and Taylorism, I focus here on the original text and leave consideration of subsequent studies and critiques to future inquiries.

Taylor attacks traditional management styles, which he characterizes as management of “initiative and incentive”.  Under this form of management, “workmen give their best initiative and in return receive some special incentive from their employers” (14).  The role of management resembles that of policy-makers in a market economy: design incentives so that workers, through their independent and self-interested actions, advance the goals of the collective.  The logic used to justify management by initiative and incentive mirrors Hayek’s argument for decentralized market economies outperforming centrally planned economies:

the essential idea of the ordinary types of management is that each workman has become more skilled in his own trade than it is possible for any one in the management to be, and that, therefore, the details of how the work shall best be done must be left to him (30)

The problem, Taylor asserts, is that workers left to their own devices are unlikely to figure out the best ways to perform their tasks: “it is in most cases impossible for the men working under these systems to do their work in accordance with the rules and laws of a science or art, even where one exists” (9).  Consequently, even though workers may be diligent and devoted, productivity will be limited by the failure to apply relevant knowledge.

As an alternative to management of initiative and incentive, Taylor proposes “scientific management”.  Instead of giving workers discretion over how they go about their jobs, management gives each worker precise and highly detailed instructions–“laws to replace rule of thumb” (53).  Rather than “leaving the solution of each problem in the hands of the individual workman”, scientific management is premised on “the substitution of a science for the individual judgment of the workman” (59).  Taylor’s “science” resembles modern-day industrial engineering:

The development of a science … involves the establishment of many rules, laws, and formulae which replace the judgement of the individual workman and which can be effectively used only after having been systematically recorded, indexed, etc. (16)

Decomposition and sequencing are also central to scientific management:

work can be done better and more economically by a subdivision of labor; each act of each mechanic, for example, should be preceded by various preparatory acts done by other men (16)

almost every act of the workman should be preceded by one or more preparatory acts of the management which enable him to do his work better and quicker than he otherwise could (10)

Since tasks are interdependent, the transition to scientific management cannot be left to the uncoordinated, independent actions of the workers:

It is only through enforced standardization of methods, enforced adoption of the best implements and working conditions, and enforced cooperation that this faster work can be assured. (41)

To stabilize production processes, Taylor proposes a variety of self-regulating feedback mechanisms.  He describes an inspection system for bicycle balls in which a group of inspectors monitor the quality of output, a group of “over-inspectors” check the inspectors, a chief inspector checks the over-inspectors, and periodic tests, consisting of specially prepared lots with known numbers of defects, are carried out by the foreman to check the functioning of the entire inspection system (46).  At the Bethlehem Steel Company, Taylor developed a system of colored slips that notified each worker of his performance on the preceding day.  “So that whenever the men received white slips they knew that everything was all right, and whenever they received yellow slips they realized that they must do better or they would be shifted to some other class of work” (33).  Another practitioner of scientific management developed a method of “measuring and recording the number of bricks laid by each man, and for telling each workman at frequent intervals how many bricks he had succeeded in laying” (41).

Scientific management had profound effects on the structure and performance of production organizations.  From a structural perspective, the organization became more complex and elaborate: groups were established for time study, teaching the workers, managing and maintaining tools, planning and monitoring work, and so forth (34).  Taylor’s calculations indicate that increases in productivity far outweighed the cost of these new overhead functions (35).  Effects on productivity were dramatic.  According to Taylor, applying scientific management to brick-laying increased the rate of work from 120 bricks per man hour to 350.  In bicycle ball inspection, productivity more than tripled, and accuracy improved significantly.

Scientific management and decision-making

The central tenant of scientific management–enforced adoption of production processes engineered on the basis of sound engineering knowledge–has been almost universally adopted in large-scale manufacturing.  It is surprising, though, the extent to which much organizational decision-making still resembles Taylor’s “management of initiative and incentive”.  Business schools teach managers to delegate authority to subordinates and refrain from “micromanaging” them.  Much effort is directed toward the design of incentive schemes that will call forth and channel the initiative of mid-level managers.  Certain decision-making processes, especially financial valuation and fully automated processes in the Internet and financial services sectors, may be subject to rigorous analysis and refined on the basis of careful experiments.  On the whole, however, decision-making remains primarily a form of craft production managed by “initiative and incentive”.

Decision-making activity is rife with rules-of-thumb and improvisation.  Estimating market size is a common task in business decision-making, but during my (admittedly brief) career as a management consultant, I never encountered any rules or formulae for performing this task.  Standardization, tools, and mechanisms for objective feedback were almost entirely absent.  In many firms, decision-making about pricing, marketing, hiring, and firing appears to be similarly ad hoc (automated price-optimization tools used in some retail environments are a notable exception).  Even if standard procedures are in place, they are almost certainly not based on sound engineering knowledge.  The disastrous performance of ratings agencies with respect to subprime mortgage securities seems to suggest that even seemingly well-engineered decision-making processes turn out, on closer inspection, to be haphazard and devoid of mechanisms for self-regulation

Why haven’t the principles of scientific management penetrated more thoroughly into the realm of organizational decision-making?  Perhaps decision-making activities are inherently less amenable to decomposition and standardization.  Or perhaps decision-makers have simply been more successful than laborers at defending their activities from rationalization.  Or there may be a third possibility: perhaps the tools for applying scientific management to decision-making have only recently become available.  In my dissertation, I argue that networked computer systems lower the costs associated with the “division of information processing”, thereby facilitating task decomposition.  Computers also facilitate measurement and feedback.

Taylor’s study suggests, however, that we should take seriously the possibility that organizational resistance may be an important factor impeding the application of scientific management to decision-making.  Many of the innovations developed by Taylor and other practitioners of scientific management were remarkably simple: for example, careful observation and analysis of bricklaying, together with adjustable scaffolds and packets for holding bricks, enabled a reduction in the number of movements required to lay bricks from eighteen to five, and resulted in nearly three-fold improvement in productivity (40).  These innovations seem well within the reach of any smart and tenacious individual.  So, Taylor asks:

Why is it, in a trade which has been continually practised since before the Christian era, and with implements practically the same as they now are, that this simplification of the bricklayer’s movements, this great gain, has not been made before? (41)

Lack of discipline on the part of managers may be part of the answer.  Taylor describes the failure of more effective shoveling techniques to spread from Bethlehem to Pittsburgh:

The Pittsburgh managers knew just how the results had been attained at Bethlehem, but they were unwilling to go to the small trouble and expense required to plan ahead and assign a separate car to each shoveler, and then keep an individual record of each man’s work, and pay him just what he had earned. (38)

Perhaps more problematically, scientific management threatens to devalue old knowledge.  Taylor describes the phenomenon in his discussion of machine tools:

The real problem is how to remove chips fast from a casting or a forging, and it matters but little whether the piece being worked upon is a part, say, of a marine engine, a printing-press, or an automobile.  For this reason, the man with the slide-rule, familiar with the science of cutting metals, who had never before seen this particular work, was able completely to distance the skilled mechanic who had made the parts of this machine his specialty for years. (53)

Like these mechanics, Shinsei’s mortgage loan approvers were replaced with low-skilled individuals equipped with the software equivalent of a slide-rule: a system that, instead of reading out the appropriate speed settings for machining a given material with a given tool, reads out the appropriate loan amount for an individual with a given age and income buying a property of a given value.  When decision-making processes are decomposed and engineered according to “scientific laws” (we might now prefer to call them theoretically motivated and empirically verified propositions), individuals who have spent years learning the intricacies of the process and developing rules-of-thumb for handling scores of special situations may suddenly find their accumulated knowledge worthless and their source of organizational influence swept away.

Concluding thoughts

Given the popularity of initiative-and-incentive management techniques and cultural aversion to micromanagement, and also the resistance of decision-makers wanting to protect the value of their hard-won knowledge, it is perhaps unsurprising that the application of scientific management–or, to use a less problematic term, industrial engineering–to decision-making has been so limited.  If, however, information technology increases the return to engineered decision-making processes, then we may witness a gradual shift toward greater decomposition, enforced standardization, and self-regulating feedback in decision-making processes.  If so, we may find helpful ideas in the work of Taylor, the assembly line paradigm, and the field of industrial engineering more generally.