The Agile development model should not be seen as a panacea or identified with specific methodologies. Instead it should be understood as a default option to be applied whenever phased solutions can be factored out.
Alternative: When conditions cannot be met, i.e when phased solutions are required, model-based system engineering frameworks should be used to integrate business-driven projects (agile) with architecture oriented ones (phased).
“The little reed, bending to the force of the wind, soon stood upright again when the storm had passed over”
The consequences of digital environments go well beyond a simple adjustment of business processes and call for an in-depth transformation of enterprise architectures.
To begin with, the generalization of digital environments bears out the Symbolic System modeling paradigm: to stay competitive, enterprises have to manage a relevant, accurate, and up-to-date symbolic representation of their business context and concerns.
With regard to architectures, it means a seamless integration of systems and knowledge architectures.
With regard to processes it means a built-in ability to learn from environments and act accordingly.
Such requirements for resilience and adaptability in unsettled environments are characteristic of the Pagoda architecture blueprint.
As can be observed wherever high buildings are being erected on shaking grounds, Pagoda-like architectures set successive layers around a central pillar providing intrinsic strength and resilience to external upsets while allowing the floors to move with the whole or be modified independently. Applied to enterprise architectures in digital environments, that blueprint can be much more than metaphoric.
That blueprint puts a new light on model based approaches to systems engineering (MBSE):
Conceptual models, targeting enterprises organization and business independently of supporting systems.
Logical models, targeting the symbolic objects managed by supporting systems as surrogates of business objects and activities.
Physical models, targeting the actual implementation of symbolic surrogates as binary objects.
Weaving together enterprises and knowledge architectures would greatly enhance the traceability of transformations induced by the immersion of enterprises in digital environments.
Systems & Knowledge Architectures
If digitized business flows are to pervade enterprise systems and feed decision-making processes, systems and knowledge architectures are to be merged into a single nervous system as materialized by the Pagoda central pillar:
Ubiquitous, massive, and unrelenting digitized business flows cannot be dealt with lest a clear distinction is maintained between raw data acquired across platforms, and the information (previously data) models which ensure the continuity and consistency of systems. .
Once structured and refined, business data flows must be blended with information models sustaining systems functionalities.
A comprehensive and business driven integration of organization and knowledge could then support strategic and operational decision-making at enterprise level.
Rounding off this nervous system with a brain, ontologies would provide the conceptual frame for models representing enterprises and their environments.
Agile Architectures & Homeostasis
Homeostasis is the ability of a viable organism to learn from their environment and adapt their behavior and structures according to changes. As such homeostasis can be understood as the eextension of enterprise agility set in digital environments, ensuring:
Integrated decision-making processes across concerns (business, systems, platforms), and time-frames (tactical, operational, strategic, … ).
Integrated information processing, from data-mining to knowledge management.
To that end, changes should be differentiated with regard to source (business or technology environment, organization, systems) and flows (data, information, knowledge); that would be achieved with a pagoda blueprint.
Threads of operational and strategic decision-making processes could then be weaved together, combining OODA loops at process level and economic intelligence at enterprise level.
Humans often expect concepts to come with innate if vague meanings before being compelled to withstand endless and futile controversies around definitions. Going the other way would be a better option: start with differences, weed out irrelevant ones, and use remaining ones to advance.
Concerning enterprise, it would start with the difference between business and architecture, and proceed with the wholeness of data, information, and knowledge.
Business Architecture is an Oxymoron
Business being about time and competition, success is not to be found in recipes but would depend on particularities with regard to objectives, use of resources, and timing. These drives are clearly at odds with architectures rationales for shared, persistent, and efficient structures and mechanisms. As a matter of fact, dealing with the conflicting nature of business and architecture concerns can be seen as a key success factor for enterprise architects, with information standing at the nexus.
Data as Resource, Information as Asset, Knowledge as Service
Paradoxically, the need of a seamless integration of data, information, and knowledge means that the distinction between them can no longer be overlooked.
Data is captured through continuous and heterogeneous flows from a wide range of sources.
Information is built by adding identity, structure, and semantics to data. Given its shared and persistent nature it is best understood as asset.
Knowledge is information put to use through decision-making. As such it is best understood as a service.
Ensuring the distinction as well as the integration must be a primary concern of enterprise architects.
Sustainable Success Depends on a Balancing Act
Success in business is an unfolding affair, on one hand challenged by circumstances and competition, on the other hand to be consolidated by experience and lessons learnt. Meeting challenges while warding off growing complexity will depend on business agility and the versatility and plasticity of organizations and systems. That should be the primary objective of enterprise architects.
“For things to remain the same, everything must change”
Lampedusa, “The Leopard”
Whatever the understanding of the discipline, most EA schemes implicitly assume that enterprise architectures, like their physical cousins, can be built from blueprints. But they are not because enterprises have no “Pause” and “Reset” buttons: business cannot be put on stand-by and must be carried on while work is in progress.
Systems & Enterprises
Systems are variously defined as:
“A regularly interacting or interdependent group of items forming a unified whole” (Merriam-Webster).
“A set of connected things or devices that operate together” (Cambridge Dictionary).
“A way of working, organizing, or doing something which follows a fixed plan or set of rules” (Collins Dictionary)
“A collection of components organized to accomplish a specific function or set of functions” (TOGAF from ISO/IEC 42010:2007)
While differing in focus, most understandings mention items and rules, purpose, and the ability to interact; none explicitly mention social structures or interactions with humans. That suggests where the line should be drawn between systems and enterprises, and consequently between corresponding architectures.
Architectures & Changes
Enterprises are live social entities made of corporate culture, organization, and supporting systems; their ultimate purpose is to maintain their identity and integrity while interacting with environments. As a corollary, changes cannot be carried out as if architectures were just apparel, but must ensure the continuity and consistency of enterprises’ structures and behaviors.
That cannot be achieved by off-soil schemes made of blueprints and step-by-step processes detached from actual organization, systems, and processes. Instead, enterprise architectures must be grown bottom up from actual legacies whatever their nature: technical, functional, organizational, business, or cultural.
Insofar as enterprise architectures are concerned, legacies are usually taken into account through one of three implicit assumptions:
No legacy assumptions ignore the issue, as if the case of start-ups could be generalized. These assumptions are logically flawed because enterprises without legacy are like embryos growing their own inherent architecture, and in that case there would be no need for architects.
En Bloc legacy assumptions take for granted that architectures as a whole could be replaced through some Big Bang operation without having a significant impact on business activities. These assumptions are empirically deceptive because, even limited to software architectures, Big Bang solutions cannot cope with the functional and generational heterogeneity of software components characterizing large organizations. Not to mention that enterprise architectures are much more that software and IT.
Piecemeal legacies can be seen as the default assumption, based on the belief that architectures can be re-factored or modernized step by step. While that assumption may be empirically valid, it may also miss the point: assuming that all legacies can be dealt with piecemeal rubs out the distinction pointed above between systems and enterprises.
So, the question remains of what is to be changed, and how ?
EA as a Work In Progress
As with leopard’s spots and identity, the first step would be to set apart what is to change (architectures) from what is to carry on (enterprise).
Maps and territories do provide an overview of spots’ arrangement, but they are static views of architectures, whereas enterprises are dynamic entities that rely on architectures to interact with their environment. So, for maps and territories to serve that purpose they should enable continuous updates and adjustments without impairing enterprises’ awareness and ability to compete.
That shift from system architecture to enterprise behavior implies that:
The scope of changes cannot be fully defined up-front, if only because the whole enterprise, including its organization and business model, could possibly be of concern.
Fixed schedules are to be avoided, lest each and every unit, business or otherwise, would have to be shackled into a web of hopeless reciprocal commitments.
Different stakeholders may come as interested parties, some more equal than others, possibly with overlapped prerogatives.
So, instead of procedural and phased approaches supposed to start from blank pages, EA ventures must be carried out iteratively with the planning, monitoring, assessment, and adjustment of changes across enterprises’ businesses, organizations, and systems. That can be represented as an extension of the OODA (Observation, Orientation, Decision, Action) loop:
Actual observations from operations (a)
Data analysis with regard to architectures as currently documented (b).
Changes in business processes (c).
Changes in architectures (d).
Moreover, due to the generalization of digital flows between enterprises and their environment, decision-making processes used to be set along separate time-frames (operational, tactical, strategic, …), must now be weaved together along a common time-scale encompassing internal (symbolic) as well as external (actual) events.
It ensues that EA processes must not only be continuous, but they also must deal with latency constraints.
Changes & Latency
Architectures are by nature shared across organizational units (enterprise level) and business processes (system level). As a corollary, architecture changes are bound to introduce mismatches and frictions across business-specific applications. Hence the need of sorting out the factors affecting the alignment of maps and territories:
Elapsed time between changes in territories and maps updates (a>b) depends on data analytics and operational architecture.
Elapsed time between changes in maps and revised objectives (b>c) depends on business analysis and organization.
Elapsed time between changes in objectives and their implementation (c>d) depends on engineering processes and systems architecture.
Elapsed time between changes in systems and changes in territories (d>a) depends on applications deployment and technical architectures.
On that basis it’s possible to define four critical lags:
Operational: data analytics can be impeded by delayed, partial, or inaccurate feedback from processes.
Mapping: business analysis can be impeded by delays or discrepancies in data analytics.
Engineering: development of applications can be impeded by delays or discrepancies in business analysis.
Processes: deployment of business processes can be impeded by delays in the delivery of supporting applications.
These lags condition the whole of EA undertakings because legacy structures, mechanisms, and organizations are to be continuously morphed into architectures without introducing misrepresentations that would shackle activities and stray decision-making.
EA Latency & Augmented Reality
Insofar as architectural changes are concerned, discrepancies and frictions are rooted in latency, i.e the elapsed time between actual changes in territories and the updating of relevant maps.
As noted above, these lags have to be weighted according to time-frames, from operational days to strategic years, so that the different agents could be presented with the relevant and up-to-date views befitting to each context and concerns.
That could be achieved if enterprises architectures were presented through augmented reality technologies.
Compared to virtual reality (VR) which overlooks the whole issue of reality and operates only on similes and avatars, augmented reality (AR) brings together virtual and physical realms, operating on apparatuses that weaves actual substrates, observations, and interventions with made-up descriptive, predictive, or prescriptive layers.
On that basis, users would be presented with actual territories (EA legacy) augmented with maps and prospective territories.
Composition and dynamics of maps and territories (actual and prospective) could be set and edited appropriately, subject to latency constraints.
Given the digitization of enterprises environments, engineering processes have to be entwined with business ones while kept in sync with enterprise architectures. That calls for new threads of collaboration taking into account the integration of business and engineering processes as well as the extension to business environments.
Whereas models are meant to support communication, traditional approaches are already straining when used beyond software generation, that is collaboration between humans and CASE tools. Ontologies, which can be seen as a higher form of models, could enable a qualitative leap for systems collaborative engineering at enterprise level.
Systems Engineering: Contexts & Concerns
To begin with contents, collaborations should be defined along three axes:
Requirements: business objectives, enterprise organization, and processes, with regard to systems functionalities.
Feasibility: business requirements with regard to architectures capabilities.
Architectures: supporting functionalities with regard to architecture capabilities.
Since these axes are usually governed by different organizational structures and set along different time-frames, collaborations must be supported by documentation, especially models.
In order to support collaborations across organizational units and time-frames, models have to bring together perspectives which are by nature orthogonal:
Contexts, concerns, and languages: business vs engineering.
Time-frames and life-cycle: business opportunities vs architecture stability.
That could be achieved if engineering models could be harnessed to enterprise ones for contexts and concerns. That is to be achieved through the integration of processes.
As already noted, the integration of business and engineering processes is becoming a key success factor.
For that purpose collaborations would have to take into account the different time-frames governing changes in business processes (driven by business value) and engineering ones (governed by assets life-cycles):
Business requirements engineering is synchronic: changes must be kept in line with architectures capabilities (full line).
Software engineering is diachronic: developments can be carried out along their own time-frame (dashed line).
Application-driven projects usually focus on users’ value and just-in-time delivery; that can be best achieved with personal collaboration within teams. Architecture-driven projects usually affect assets and non-functional features and therefore collaboration between organizational units.
Collaboration: Direct or Mediated
Collaboration can be achieved directly or through some mediation, the former being a default option for applications, the latter a necessary one for architectures.
Both can be defined according to basic cognitive and organizational mechanisms and supported by a mix of physical and virtual spaces to be dynamically redefined depending on activities, projects, locations, and organisation.
Direct collaborations are carried out between individuals with or without documentation:
Immediate and personal: direct collaboration between 5 to 15 participants with shared objectives and responsibilities. That would correspond to agile project teams (a).
Delayed and personal: direct collaboration across teams with shared knowledge but with different objectives and responsibilities. That would tally with social networks circles (c).
Mediated collaborations are carried out between organizational units through unspecified individual members, hence the need of documentation, models or otherwise:
Direct and Code generation from platform or domain specific models (b).
Model transformation across architecture layers and business domains (d)
Depending on scope and mediation, three basic types of collaboration can be defined for applications, architecture, and business intelligence projects.
As it happens, collaboration archetypes can be associated with these profiles.
Agile development model (under various guises) is the option of choice whenever shared ownership and continuous delivery are possible. Application projects can so be carried out autonomously, with collaborations circumscribed to team members and relying on the backlog mechanism.
Projects set across enterprise architectures cannot be carried out without taking into account phasing constraints. While ill-fated Waterfall methods have demonstrated the pitfalls of procedural solutions, phasing constraints can be dealt with a roundabout mechanism combining iterative and declarative schemes.
Engineering vs Business Driven Collaborations
With collaborative engineering upgraded at enterprise level, the main challenge is to iron out frictions between application and architecture projects and ensure the continuity, consistency and effectiveness of enterprise activities. That can be achieved with roundabouts used as a collaboration mechanism between projects, whatever their nature:
Shared models are managed at roundabout level.
Phasing dependencies are set in terms of assertions on shared models.
Depending on constraints projects are carried out directly (1,3) or enter roundabouts (2), with exits conditioned by the availability of models.
Moreover, with engineering embedded in business processes, collaborations must also bring together operational analytics, decision-making, and business intelligence. Here again, shared models are to play a critical role:
Enterprise descriptive and prescriptive models for information maps and objectives
Environment predictive models for data and business understanding.
Whereas both engineering and business driven collaborations depend on sharing information and knowledge, the latter have to deal with open and heterogeneous semantics. As a consequence, collaborations must be supported by shared representations and proficient communication languages.
Ontologies & Representations
Ontologies are best understood as models’ backbones, to be fleshed out or detailed according to context and objectives, e.g:
Thesaurus, with a focus on terms and documents.
Systems modeling, with a focus on integration, e.g Zachman Framework.
Classifications, with a focus on range, e.g Dewey Decimal System.
Meta-models, with a focus on model based engineering, e.g models transformation.
Conceptual models, with a focus on understanding, e.g legislation.
Knowledge management, with a focus on reasoning, e.g semantic web.
As such they can provide the pillars supporting the representation of the whole range of enterprise concerns:
Taking a leaf from Zachman’s matrix, ontologies can also be used to differentiate concerns with regard to architecture layers: enterprise, systems, platforms.
Last but not least, ontologies can be profiled with regard to the nature of external contexts, e.g:
Institutional: Regulatory authority, steady, changes subject to established procedures.
Professional: Agreed upon between parties, steady, changes subject to established procedures.
Corporate: Defined by enterprises, changes subject to internal decision-making.
Social: Defined by usage, volatile, continuous and informal changes.
Personal: Customary, defined by named individuals (e.g research paper).
Ontologies & Communication
If collaborations have to cover engineering as well as business descriptions, communication channels and interfaces will have to combine the homogeneous and well-defined syntax and semantics of the former with the heterogeneous and ambiguous ones of the latter.
With ontologies represented as RDF (Resource Description Framework) graphs, the first step would be to sort out truth-preserving syntax (applied independently of domains) from domain specific semantics.
On that basis it would be possible to separate representation syntax from contents semantics, and to design communication channels and interfaces accordingly.
That would greatly facilitate collaborations across externally defined ontologies as well as their mapping to enterprise architecture models.
To summarize, the benefits of ontological frames for collaborative engineering can be articulated around four points:
A clear-cut distinction between representation semantics and truth-preserving syntax.
A common functional architecture for all users interfaces, humans or otherwise.
Modular functionalities for specific semantics on one hand, generic truth-preserving and cognitive operations on the other hand.
Profiled ontologies according to concerns and contexts.
A critical fifth benefit could be added with regard to business intelligence: combined with deep learning capabilities, ontologies would extend the scope of collaboration to explicit as well as implicit knowledge, the former already framed by languages, the latter still open to interpretation and discovery.
Knowledge graphs, which have become a key component of knowlege management, are best understood as a reincarnation of ontologies.
Repeated announces of looming software apocalypse may take some edge off vigilance, but repeated systems failures should be taken seriously, if only because they appear to be rooted in a wide array of causes, from wrongly valued single parameters (e.g 911 threshold or Apple’s free pass for “root” users) to architecture obsolescence (e.g reservation systems.)
Yet, if alarms are not to be ignored, prognoses should go beyond syndromes and remedies beyond sticking plaster: contrary to what is suggested by The Atlantic’s article, systems are much more than piles of code, and programming is probably where quality has been taken the most seriously.
Programs vs Systems
Whatever programmers’ creativity and expertise, they cannot tackle complexity across space, time, and languages: today’s systems are made of distributed interacting components, specified and coded in different languages, and deployed and modified across overlapping time-frames. Even without taking into account continuous improvements in quality, apocalypse is not to loom in the particulars of code but on their ways in the world.
Solutions should therefore be looked for at system level, and that conclusion can only be bolstered by the ubiquity of digitized business flows.
Systems are the New Babel
As illustrated by the windfalls benefiting Cobol old timers, language is arguably a critical factor, for the maintenance of legacy programs as well as for communication between stakeholders, users, and engineers.
So if problems can be traced back to languages, it’s where solutions are to be found: from programming languages (for code) to natural ones (for systems requirements), everything can be specified as symbolic representations, i.e models.
Model in the Loop
Models are generally understood as abstractions, and often avoided for that very reason. That shortsighted mind-set is made up for by concrete employs of abstractions, as illustrated by the Automotive industry and the way it embeds models in engineering processes.
Summarily, the Automotive’s Model in Loop (MiL) can be explained through three basic ideas:
Systems are to be understood as the combination of physical and software artifacts.
Insofar as both can be implemented as digits, they can be uniformly described as models.
As a consequence, analysis, design and engineering can be carried out through the iterative building, simulating, testing, and adjusting various combinations of hardware and software.
By bringing together physical components and code into a seamless digitized whole, MiL brings down the conceptual gap between actual elements and symbolic representations, aka models. And that leap could be generalized to a much wider range of systems.
Models are the New Code
Programming habits and the constraints imposed by the maintenance of legacy systems have perpetuated the traditional understanding of systems as a building-up of programs; hence the focus put on the quality of code. But when large, distributed, and perennial systems are concerned, that bottom-up mind-set falls short and brings about:
An exponential increase of complexity at system level.
Opacity and discontinued traceability at application level between current use and legacy code.
Both flaws could be corrected by combining top-down modeling and bottom-up engineering. That could be achieved with iterative processes carried out from both directions.
Model in the Loop meets Enterprise Architecture
From a formal perspective models are of two sorts: extensional ones operate bottom-up and associate sets of individuals with categories, intensional ones operate top-down and specify the features meant to be shared by all instances of a type. based on that understanding, the former can be used to simulate the behaviors of targeted individuals depending on categories, and the latter to prescribe how to create instances of types meant to implement categories.
As it happens, Model-in-loop combines the two schemes at component level:
Any combination of manual and automated solution can be used as a starting point for analysis and simulation (a).
Given the outcomes of simulation and tests, the architecture is revisited (b) and corresponding artifacts (software and hardware) are designed (c).
The new combination of artifacts are developed and integrated, ready for analysis and simulation (d).
Assuming that MiL bottom-up approach could be tallied with top-down systems engineering processes, it would enable a seamless and continuous integration of changes in software components and systems architectures.
All too often Enterprise Architecture (EA) is planned as a big bang project to be carried out step by step until completion. That understanding is misguided as it confuses EA with IT systems and implies that enterprises could change their architectures as if they were apparel.
But enterprise architectures are part and parcel of enterprises, a combination of culture, organization, and systems; whatever the changes, they must keep the continuity, integrity, and consistency of the whole.
Who: enterprise roles, system users, platform entry points.
What: business objects, symbolic representations, objects implementation.
How: business logic, system applications, software components.
When: processes synchronization, communication architecture, communication mechanisms.
Where: business sites, systems locations, platform resources.
These capabilities are set across architecture layers and support business, engineering, and operational processes.
Enterprise architects are to continuously assess and improve these capabilities with regard to current weaknesses (organizational bottlenecks, technical debt) or future developments (new business, M&A, new technologies).
Given the increased dependencies between business, engineering, and operations, defining EA workflows in terms of work units defined bottom-up from capabilities is to provide clear benefits with regard to EA versatility and plasticity.
Contrary to top-down (aka activity based) ones, bottom-up schemes don’t rely on one-fits-all procedures; as a consequence work units can be directly defined by capabilities and therefore mapped to engineering workshops:
Moreover, dependency constraints can be directly defined as declarative assertions attached to capabilities and managed dynamically instead of having to be hard-wired into phased processes.
That approach is to ensure two agile conditions critical for the development of architectural features:
Shared ownership: lest the whole enterprise be paralyzed by decision-making procedures, work units must be carried out under the sole responsibility of project teams.
Continuous delivery: architecture driven developments are by nature transverse but the delivery of building blocs cannot be put off by the decision of all parties concerned; instead it should be decoupled from integration.
Enterprise architecture projects could then be organized as a merry-go-round of capabilities-based work units to be set up, developed, and delivered according to needs and time-frames.
Enterprise architecture is about governance more than engineering. As such it has to ensure continuity and consistency between business objectives and strategies on one side, engineering resources and projects on the other side.
Assuming that capability-based work units will do the job for internal dependencies (application contents and engineering), the problem is to deal with external ones (business objectives and enterprise organization) without introducing phased processes. Beyond differences in monikers, such dependencies can generally be classified along three reasoned categories:
Operational: whatever can be observed and acted upon within a given envelope of assets and capabilities.
Tactical: whatever can be observed and acted upon by adjusting assets, resources and organization without altering the business plans and anticipations.
Strategic: decisions regarding assets, resources and organization contingent on anticipations regarding business environments.
The role of enterprise architects will then to manage the deployment of updated architecture capabilities according to their respective time-frames.
As noted before, EA workflows by nature can seldom be carried out in isolation as they are meant to deal with functional features across business domains. Instead, a portfolio of architecture (as opposed to development) work units should be managed according to their time-frame, the nature of their objective, and the kind of models to be used:
Strategic features affect the concepts defining business objectives and processes. The corresponding business objects and processes are primarily defined with descriptive models; changes will have cascading effects for engineering and operations.
Tactical features affect the definition of artifacts, logical or physical. The corresponding engineering processes are primarily defined with prescriptive models; changes are to affect operational features but not the strategic ones.
Operational features affect the deployment of resources, logical or physical. The corresponding processes are primarily defined with predictive models derived from descriptive ones; changes are not meant to affect strategic or tactical features.
Architectural projects could then be managed as a dynamic backlog of self-contained work units continuously added (a) or delivered (b).