Ever since ZapThink published our Business Agility as an Emergent Property of SOA ZapFlash, we’ve been explaining in our Licensed ZapThink Architect course how SOA implementations must be complex systems in order to deliver on emergent properties like business agility. Yet, even though we’ve expanded our treatment of Complex Systems Engineering (CSE) in the latest version of the course, the reaction of most of our students is typically one of perplexity.

Not that we’re really surprised, however. Breaking away from the Traditional Systems Engineering (TSE) way of thinking is a huge leap for most technologists, as it shakes to the foundation how they think about architecture, not just SOA in particular, but even more fundamentally, the role IT plays in the enterprise.

Complex Systems: Order from Chaos in Nature
Complex systems theory is especially fascinating because it describes how many natural phenomena occur. Whenever there is an emergent property in nature — that is, a property of a system as a whole that the elements of the system do not exhibit — then that system is a complex system. Everything from the human mind to the motion of galaxies are emergent properties of their respective systems. Fair enough, but those are all natural complex systems, and we’re charged with implenting an artificial, human-made complex system. How we take the lessons from nature and apply them in the IT shop is a question that engenders the perplexity we see on our students’ faces.

There is a fundamental flaw in this distinction, however. Making such a distinction between natural and artificial systems is basically a TSE way of thinking because it separates people from their tools. In a traditional IT system, people are the “users,” but not inherently part of the system. In many complex systems, however, people aren’t just part of the system, they are the system.

In fact, any large group of people behaves as a complex system. For example, take a stadium full of people doing the wave. Each individual in the crowd decides whether or not to participate based upon the behavior of other people, but the wave itself has “a mind of its own” — in other words, the wave behavior is an emergent property of the crowd. Another example would be a traffic jam. An accident in opposing traffic will slow down your side of the freeway every time, even though each individual knows that slowing down to look will cause a jam. You and hundreds of people like you can decide not to slow down to look in order to avoid creating a jam, but the jam forms nevertheless.

In the wave example, no technology of any kind takes a role, while in the traffic example, vehicles affect the behavior of the system to a certain extent. In fact, changing the technology can have a dramatic impact on the behavior of the system: if the traffic consisted of trains instead of automobiles, your train might not slow down at all for a problem on a neighboring track. But regardless of whether it’s made up of trains or automobiles, the system includes individual people making individual decisions based upon their personal point of view within the system, and emergent properties result, just as they do in a natural system with no people involved at all.

The Enterprise as a Complex System
Any human organization is, in fact, a complex system, including those unwieldy beasts we refer to as enterprises. Enterprises all have policies and managers and lines of control, but the overall behavior of the enterprise emerges from the individual behaviors of the participants in it. Furthermore, the emergent behaviors of corporations and governments may depend entirely on the people who belong to such enterprises, independent of technology. But when we do include technology in our enterprises, we can dramatically affect the emergent behavior of those systems, just as switching from cars to trains changes how traffic behaves.

So, what do you get when you take traffic and subtract the people? A parking lot! Without the people, what was a complex system is now little more than a collection of individual, traditional systems, namely the cars themselves. Each auto is a traditional system in the sense that the properties it exhibits are the properties its manufacturer designed into it. The best you can expect with TSE, after all, is to deliver a system that does what it’s supposed to do.

Too often in the enterprise, people confuse complex systems with collections of traditional systems, which is just as big a mistake as confusing a parking lot full of empty cars with a traffic jam. In fact, architects are often the first to make this mistake. Of course, it’s certainly true that some architects are too focused on the technology, leaving people out of the equation altogether, but even for those architects who include people in the architecture, they often do so from a TSE perspective rather than a CSE approach. But no matter how hard you try, designing better steering wheels and leather seats and the like won’t prevent traffic jams!

Complex Systems Thinking and SOA
In traditional systems thinking, then, we have systems and users of those systems, where the users have requirements for the systems. If the systems meet those requirements then everybody’s happy. In complex systems thinking, we have systems made up of technology and people, where the people make decisions and perform actions based upon their own individual circumstances. They interact with the technology in their environments as appropriate, and the technology responds to those interactions based upon the requirements for the complex system as a whole. In many cases, the technology provides a feedback loop that helps the people achieve their individual requirements, just as brake lights in a traffic jam help reduce the chance of collisions.

Such complex systems thinking has been a common theme in many of ZapThink’s articles for several years now. Here are some examples:

  • In Best Effort SOA and the SOA Quality Star, we discuss how the business agility requirement complicates the SOA quality challenge. Because agility is an emergent property, we have to establish continuous quality policies that ensure that the delivered system is sufficiently agile. As a result, there’s always a tradeoff between agility and quality we call “Best Effort SOA.”

  • In The Buckaroo Banzai Effect: Location Independence, Service-Oriented Architecture, and the Cloud, we explore the “Next Big Thing” as SOA, Cloud Computing, Web 2.0, and mobile presence converge. Our conclusion? “The Next Big Thing isn’t a cloud in the sense of abstracted data centers full of technology; it’s a cloud of people, communicating, creating, and conducting business, where the technology is hidden in the mist.”

  • In Resilience: The Missing Word in the SOA Conversation, we discuss how SOA implementations must be resilient, that is, they must have self-righting tendencies that help them recover from adverse forces in their environment. Resilience is a property of the component systems in a SOA implementation that allows the overall system to exhibit the emergent property of business agility.

  • Finally, in the more recent The Christmas Day Bomber, Moore’s Law, and Enterprise IT, we introduce the concept of a “metapolicy feedback loop” that explicitly describes the relationship between humans tackling governance in the enterprise and the governance technology they leverage for the task. Only by taking a complex systems approach to the problem of governance do organizations have any chance of dealing with the explosion in the quantity and complexity of information in the enterprise over time.

The common elements to all of these arguments are the feedback loops between people and technology at the component level that enables the overall system to continue to meet requirements as those requirements change — the essence of business agility.

The ZapThink Take
If you still find yourself perplexed by this whole complex systems story, it might help to point out that complex systems aren’t necessarily complicated. In fact, in a fundamental way they are really quite simple. Traffic jams may be difficult to understand, but individuals driving cars are not. Best practices like Metadata-driven governance, the Business Service abstraction, and infrastructure and implementation variability, to name a few, are well within reach of today’s SOA initiatives. And the great thing about complex systems is that if you take care of the nuts and bolts, the big picture ends up taking care of itself.

For organizations who don’t take a complex systems approach to SOA, however, the risks are enormous. As traditional systems scale, they become less agile. Ask any architect who’s attempted to hardwire several disparate pieces of middleware together in a large enterprise — yes, maybe you can get such a rat’s nest to work, but it will be expensive and inflexible. If you want to scale your SOA implementation so that it continues to deliver business agility even on the enterprise scale, then the complex systems approach is absolutely essential.

Discussion
50 comments for “Scaling SOA with Complex Systems Engineering”
John Brondum Avatar

If you are still perplexed then try try this blog post :)
http://emergentarchitecture.net/2010/02/24/more-isnt-just-more/

Posted by John Brondum | February 25, 2010
seanpdoyle Avatar

Excellent article! The comment link by John Brondum is also excellent as well in fleshing this out.

Coming from a traditional engineering perspective, much effort is often applied in designing in ever more precise details to avoid or dampen emergent behavior. What's far more effective, especially when integrating increasingly more complex frameworks and network enabled applications, is to embrace emergence and actually use it to ones advantage. This requires a sea change in thinking from a traditional engineering approach to a more adaptive and collaborative approach focusing on flexible, adaptive interactions.

Posted by seanpdoyle | April 9, 2010
Ron Wolf Avatar

A terrific read, thx! In particular, your one sentence characterization of emergence is brilliant. And your examples are a lot of fun.

However, I'm perplexed at how this fine post could only have garnered two comments?? And also how I missed it when it first came out? I would have paid attention for sure with a title such as "Emergent Systems Thinking". Could I suggest that CSE is, um, not the most sellable or understandable terminology? Indeed as you point out, complexity is not even a necessary attribute of these Complex Systems....

Posted by Ron Wolf | May 22, 2012
Are Frameworks Too Constraining and is Chaos the Natural State? | Software Architecture Zen Avatar

[...] have recently published a number of good articles on complex systems, the death of Enterprise Architecture and the dangers of ‘checklist architecture’. [...]

life insurance claim lawyer attorney law firm Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Webdesign Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Webdesign | August 3, 2016
man with a van London Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by man with a van London | August 4, 2016
Buy Drugs Without a Prescription Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Buy Drugs Without a Prescription | August 13, 2016
семейные ценности Avatar

... [Trackback]

[...] Find More Informations here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by семейные ценности | August 17, 2016
satta matka Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by satta matka | August 22, 2016
satta matka Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by satta matka | August 22, 2016
Joseph de Saram#Rhodium Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Joseph de Saram#Rhodium | August 26, 2016
Agen Bandarq Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Agen Bandarq | September 1, 2016
yandex ankara Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by yandex ankara | September 1, 2016
forlimpopoli.biancheria.best Avatar

... [Trackback]

[...] Read More: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by forlimpopoli.biancheria.best | September 2, 2016
economics tuition Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by economics tuition | September 3, 2016
clenbuterol sopharma Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by clenbuterol sopharma | September 3, 2016
yandex ankara Avatar

... [Trackback]

[...] Read More: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by yandex ankara | September 10, 2016
طراحی وب سایت Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by طراحی وب سایت | September 18, 2016
web design services Melbourne Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by web design services Melbourne | September 29, 2016
can you make money with udemy Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by can you make money with udemy | October 1, 2016
Entertainment and Movie reviews with tips on how to get Website Traffic and Make Money Online. Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Best Newspaper in India Avatar

... [Trackback]

[...] There you will find 41462 more Infos: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Best Newspaper in India | November 2, 2016
para para dinle Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by para para dinle | November 3, 2016
computer scientist Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by computer scientist | November 4, 2016
best steroid sites to buy from Avatar

... [Trackback]

[...] There you will find 95395 more Infos: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by best steroid sites to buy from | November 4, 2016
joseph de saram Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by joseph de saram | November 5, 2016
joseph s r de saram Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by joseph s r de saram | November 5, 2016
joe de saram Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by joe de saram | November 5, 2016
joseph de saram Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by joseph de saram | November 6, 2016
Sandra balan Avatar

... [Trackback]

[...] Read More: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Sandra balan | November 11, 2016
computer kopen winterswijk Avatar

... [Trackback]

[...] Read More: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by computer kopen winterswijk | November 14, 2016
New Life infertility treatment Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by New Life infertility treatment | November 15, 2016
palliet-kjole.dk Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by palliet-kjole.dk | November 17, 2016
erotik outlet Avatar

... [Trackback]

[...] There you will find 38413 more Infos: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by erotik outlet | November 17, 2016
une bite Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by une bite | November 21, 2016
what is dnp steroid Avatar

... [Trackback]

[...] There you will find 26807 more Infos: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by what is dnp steroid | November 23, 2016
attorney Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by attorney | November 27, 2016
http://www.godwinsremovals.co.uk/international-removals/northern-ireland Avatar

... [Trackback]

[...] Read More: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

ATM Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by ATM | December 12, 2016
android games Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by android games | December 17, 2016
android apps Avatar

... [Trackback]

[...] Find More Informations here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by android apps | December 18, 2016
kid fights at school Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by kid fights at school | December 19, 2016
my link Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by my link | December 22, 2016
Skrota bilen Avatar

... [Trackback]

[...] There you will find 78073 more Infos: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Skrota bilen | December 28, 2016
testo p Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by testo p | December 28, 2016
Skrota bilen Göteborg Avatar

... [Trackback]

[...] Read More here: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Skrota bilen Göteborg | December 28, 2016
trump for children Avatar

... [Trackback]

[...] Read More: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by trump for children | December 29, 2016
basement waterproofing rochester ny Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by basement waterproofing rochester ny | December 31, 2016
Link M88 Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2010/02/11/scaling-soa-with-complex-systems-engineering/ [...]

Posted by Link M88 | January 4, 2017