This post is part of my collaborative research with Shinsei Bank on highly-evolvable enterprise software. It is licensed under the Creative Commons Attribution-ShareAlike 3.0 license. I am indebted to Jay Dvivedi and his team at Shinsei Bank for supporting this research. All errors are my own.
This blog approaches enterprise software architecture from a relatively theoretical perspective, so the implications for C-level strategic management may not be immediately obvious. I’d like to take a step back and explain why the concepts that I’m trying to develop are of real and pressing importance for large organizations.
Even in the most stable industries, the business environment changes constantly. Technology, customer needs, macro-economic trends, strategy and tactics of existing and emerging competitors, regulations, shareholder and union demands–all change constantly. Change may be gradual or violent, predictable or unexpected, but it never stops. To sustain performance in a changing environment, businesses must adapt.
In practice, adaptation almost always requires modification of business processes. For example, incorporating new media into advertising strategies necessitates changing the processes that plan, implement, and evaluate marketing campaigns. Similarly, to comply with new regulations, processes may need to incorporate different decision rules or generate new reports. It follows that the ability to modify business processes efficiently and rapidly is an important driver of organizational performance and a potential source of competitive advantage1.
Imagine, for a moment, that you, the CEO of a major enterprise, are approached by a mysterious man in a blue suit who offers to sell you a remarkable robot. This robot, he says, will perform much of the repetitive, time-consuming, routine work that absorbs the time and energy of your managers and staff, freeing them up to work on more interesting problems such as acquiring competitors, developing new products, or expanding into new markets. Although this robot is very, very expensive and takes several years to train, the man promises that the robot works so efficiently that it will save you a huge sum of money and pay for itself within a few years. Although somewhat skeptical, you agree to purchase the robot.
Once the robot is finally ready for use, about a year behind schedule, you flip the switch. The robot is a wonder to behold, continually dashing off in all directions to monitor inventories and account balances, manage employee salaries and vacation allowances, keep track of customer orders, and prepare financial reports. The robot has some problems, such as calculating taxes incorrectly for a few states and occasionally losing orders, but the man in the blue suit assures you that the problems can be fixed in a few months–at some cost, of course. On the whole, you are satisfied: training the robot took longer than you expected (and the man in the blue suit charged a princely fee), and even now the robot misbehaves from time to time, but it does work very efficiently.
After about a year, your firm decides to introduce a new service. Instead of simply selling widgets outright, you will also lease them, enabling you to optimize maintenance over the widget life cycle. Of course, you must adjust your accounting processes to handle the leased widgets, and create new processes for coordinating the maintenance of leased widgets. Aha!, you think, another task for our marvelous robot. You call the robot into your office and ask it to take on these new tasks. The robot stares at you blankly. It doesn’t seem to understand. So you consult with the kindly man in the blue suit. Of course the robot can do this, he tells you, but we will have to retrain it. And we will have to make sure that the new training doesn’t interfere with the robot’s current work. This will take a year or two, he says, and it will be very, very expensive.
Couldn’t you just buy another robot, you ask, that would handle these new tasks? That would be very difficult, the man says, because you will end up with two separate accounting statements and two separate maintenance schedules that will conflict with each other. That may be so, you say, but can’t the robots be trained to talk with each other and solve these problems by themselves? Yes, says the man, with a twinkle in his eye, but it will take several years and it will be very, very expensive.
In the end, you do as the man in the blue suit suggests. And as the years go by, you find with increasing frequency that you face unpleasant choices between passing up business opportunities, or paying the man in the blue suit astronomical sums of money to retrain your robot to help you to capture the new opportunities (after seemingly interminable delays). You find it increasingly difficult to compete with smaller, younger companies with newer robots. You begin to wonder if you should have bought the robot in the first place.
The robot, of course, is the traditional enterprise software system. Installed at great expense, it works (perhaps) for a while. Over time, it becomes increasingly inflexible and increasingly costly to maintain. It promises a Faustian bargain: improved efficiency in the short term, at the expense of adaptability in the long term. The cost and time required to modify business processes rise higher and higher, to the point where they may nullify the benefits of otherwise profitable adaptations. Organizations may find that stagnation and gradual decline are, unattractive as they may be, nevertheless preferable to costly, risky, and time-consuming system modifications. Moreover, organizations become increasingly dependent on the firms that maintain the systems. Since many organizations struggle to achieve even short term efficiency improvements, traditional enterprise systems are dubious bargains indeed.
Jay uses the metaphor of a tree to describe this process. The enterprise system takes root in the organization’s processes and extends ever more deeply into them. Over time, the organization becomes so tightly bound up by these roots that it loses the freedom to move. Enterprise systems are like the baobabs described by the Little Prince:
A baobab is something you will never, never be able to get rid of if you attend to it too late. It spreads over the entire planet. It bores clear through it with its roots. And if the planet is to small, and the baobabs are too many, they split it in pieces… (Antoine de Saint Exupéry, 1971, 22)
Pulling up the baobabs
Though the preceding exposition may be a bit whimsical, the threat posed by inflexible enterprise software is not. Fortunately, just as the Little Prince found that he could solve his baobab problem by uprooting the baobabs as soon as they became distinguishable from rosebushes, the solution in the case of enterprise software may be similarly simple, if not nearly so obvious.
Judging from Jay’s experience “pulling up the baobabs” (i.e., getting rid of inflexible enterprise systems) and replacing them with more congenial species, the problem can be solved by architecting systems according to a relatively small set of design rules. These rules make systems far more modular, and hence smaller and more manageable–more like rosebushes than baobabs. Some of these rules, such as defining system components in terms of their inputs and outputs, are relatively familiar. Others, such as extreme decomposition, mutually-verifying dualism, component-level interoperation, context-awareness, and reversibility, are less familiar or, in some cases, entirely novel.
Applying these design rules appears to yield systems that are less costly, less risky, and less time-consuming to build, easier to manage and maintain, and far more malleable. Moreover, the systems can be characterized in terms of machines and assembly lines that build data objects–that is, in terms accessible to CEOs and line of business managers–while banishing arcane software engineering jargon to the domain of implementors where it belongs.
Although I am optimistic about the potential of these design rules (otherwise, I would not bother to research them), much work remains to be done in order to articulate them precisely and evaluate them accurately. Thus, readers will probably encounter logical gaps, questionable assertions, and as yet unexplored byways. Please do not hesitate to chime in with constructive ideas, suggestions, or criticisms.