“The little reed, bending to the force of the wind, soon stood upright again when the storm had passed over”
The consequences of digital environments go well beyond a simple adjustment of business processes and call for an in-depth transformation of enterprise architectures.
To begin with, the generalization of digital environments bears out the Symbolic System modeling paradigm: to stay competitive, enterprises have to manage a relevant, accurate, and up-to-date symbolic representation of their business context and concerns.
With regard to architectures, it means a seamless integration of systems and knowledge architectures.
With regard to processes it means a built-in ability to learn from environments and act accordingly.
Such requirements for resilience and adaptability in unsettled environments are characteristic of the Pagoda architecture blueprint.
As can be observed wherever high buildings are being erected on shaking grounds, Pagoda-like architectures set successive layers around a central pillar providing intrinsic strength and resilience to external upsets while allowing the floors to move with the whole or be modified independently. Applied to enterprise architectures in digital environments, that blueprint can be much more than metaphoric.
That blueprint puts a new light on model based approaches to systems engineering (MBSE):
Conceptual models, targeting enterprises organization and business independently of supporting systems.
Logical models, targeting the symbolic objects managed by supporting systems as surrogates of business objects and activities.
Physical models, targeting the actual implementation of symbolic surrogates as binary objects.
Pagoda Blueprint & Digital Environments
The Pagoda blueprint gets a new relevance in the context of digital transformation.
Moreover, the blueprint is not limited to enterprise architectures and can be applied to every kind of systems:
Devices associated to physical platforms supporting analog communication through the Internet of Things (a).
Equipements associated to physical platforms controlled by systems, supporting digital communication (b) and functional alignment (c) .
That would greatly enhance the traceability and transparency of transformations induced by the immersion of enterprises in digital environments.
Systems & Knowledge Architectures
If digitized business flows are to pervade enterprise systems and feed business intelligence (BI), systems and knowledge architectures are to be merged into a single nervous system as materialized by the Pagoda central pillar:
Ubiquitous, massive, and unrelenting digitized business flows cannot be dealt with lest a clear distinction is maintained between raw data acquired across platforms, and the information (previously data) models which ensure the continuity and consistency of systems. .
Once structured and refined, business data flows must be blended with information models sustaining systems functionalities.
A comprehensive and business driven integration of organization and knowledge could then support strategic and operational decision-making at enterprise level.
Rounding off this nervous system with a brain, ontologies would provide the conceptual frame for models representing enterprises and their environments.
Agile Architectures & Homeostasis
Homeostasis is the ability of a viable organism to learn from their environment and adapt their behavior and structures according to changes. As such homeostasis can be understood as the eextension of enterprise agility set in digital environments, ensuring:
Integrated decision-making processes across concerns (business, systems, platforms), and time-frames (tactical, operational, strategic, … ).
Integrated information processing, from data-mining to knowledge management.
To that end, changes should be differentiated with regard to source (business or technology environment, organization, systems) and flows (data, information, knowledge); that would be achieved with a pagoda blueprint.
Threads of operational and strategic decision-making processes could then be weaved together, combining OODA loops at process level and economic intelligence at enterprise level.
Given the digitization of enterprises environments, engineering processes have to be entwined with business ones while kept in sync with enterprise architectures. That calls for new threads of collaboration taking into account the integration of business and engineering processes as well as the extension to business environments.
Whereas models are meant to support communication, traditional approaches are already straining when used beyond software generation, that is collaboration between humans and CASE tools. Ontologies, which can be seen as a higher form of models, could enable a qualitative leap for systems collaborative engineering at enterprise level.
Systems Engineering: Contexts & Concerns
To begin with contents, collaborations should be defined along three axes:
Requirements: business objectives, enterprise organization, and processes, with regard to systems functionalities.
Feasibility: business requirements with regard to architectures capabilities.
Architectures: supporting functionalities with regard to architecture capabilities.
Since these axes are usually governed by different organizational structures and set along different time-frames, collaborations must be supported by documentation, especially models.
In order to support collaborations across organizational units and time-frames, models have to bring together perspectives which are by nature orthogonal:
Contexts, concerns, and languages: business vs engineering.
Time-frames and life-cycle: business opportunities vs architecture stability.
That could be achieved if engineering models could be harnessed to enterprise ones for contexts and concerns. That is to be achieved through the integration of processes.
As already noted, the integration of business and engineering processes is becoming a key success factor.
For that purpose collaborations would have to take into account the different time-frames governing changes in business processes (driven by business value) and engineering ones (governed by assets life-cycles):
Business requirements engineering is synchronic: changes must be kept in line with architectures capabilities (full line).
Software engineering is diachronic: developments can be carried out along their own time-frame (dashed line).
Application-driven projects usually focus on users’ value and just-in-time delivery; that can be best achieved with personal collaboration within teams. Architecture-driven projects usually affect assets and non-functional features and therefore collaboration between organizational units.
Collaboration: Direct or Mediated
Collaboration can be achieved directly or through some mediation, the former being a default option for applications, the latter a necessary one for architectures.
Both can be defined according to basic cognitive and organizational mechanisms and supported by a mix of physical and virtual spaces to be dynamically redefined depending on activities, projects, locations, and organisation.
Direct collaborations are carried out between individuals with or without documentation:
Immediate and personal: direct collaboration between 5 to 15 participants with shared objectives and responsibilities. That would correspond to agile project teams (a).
Delayed and personal: direct collaboration across teams with shared knowledge but with different objectives and responsibilities. That would tally with social networks circles (c).
Mediated collaborations are carried out between organizational units through unspecified individual members, hence the need of documentation, models or otherwise:
Direct and Code generation from platform or domain specific models (b).
Model transformation across architecture layers and business domains (d)
Depending on scope and mediation, three basic types of collaboration can be defined for applications, architecture, and business intelligence projects.
As it happens, collaboration archetypes can be associated with these profiles.
Agile development model (under various guises) is the option of choice whenever shared ownership and continuous delivery are possible. Application projects can so be carried out autonomously, with collaborations circumscribed to team members and relying on the backlog mechanism.
Projects set across enterprise architectures cannot be carried out without taking into account phasing constraints. While ill-fated Waterfall methods have demonstrated the pitfalls of procedural solutions, phasing constraints can be dealt with a roundabout mechanism combining iterative and declarative schemes.
Engineering vs Business Driven Collaborations
With collaborative engineering upgraded at enterprise level, the main challenge is to iron out frictions between application and architecture projects and ensure the continuity, consistency and effectiveness of enterprise activities. That can be achieved with roundabouts used as a collaboration mechanism between projects, whatever their nature:
Shared models are managed at roundabout level.
Phasing dependencies are set in terms of assertions on shared models.
Depending on constraints projects are carried out directly (1,3) or enter roundabouts (2), with exits conditioned by the availability of models.
Moreover, with engineering embedded in business processes, collaborations must also bring together operational analytics, decision-making, and business intelligence. Here again, shared models are to play a critical role:
Enterprise descriptive and prescriptive models for information maps and objectives
Environment predictive models for data and business understanding.
Whereas both engineering and business driven collaborations depend on sharing information and knowledge, the latter have to deal with open and heterogeneous semantics. As a consequence, collaborations must be supported by shared representations and proficient communication languages.
Ontologies & Representations
Ontologies are best understood as models’ backbones, to be fleshed out or detailed according to context and objectives, e.g:
Thesaurus, with a focus on terms and documents.
Systems modeling, with a focus on integration, e.g Zachman Framework.
Classifications, with a focus on range, e.g Dewey Decimal System.
Meta-models, with a focus on model based engineering, e.g models transformation.
Conceptual models, with a focus on understanding, e.g legislation.
Knowledge management, with a focus on reasoning, e.g semantic web.
As such they can provide the pillars supporting the representation of the whole range of enterprise concerns:
Taking a leaf from Zachman’s matrix, ontologies can also be used to differentiate concerns with regard to architecture layers: enterprise, systems, platforms.
Last but not least, ontologies can be profiled with regard to the nature of external contexts, e.g:
Institutional: Regulatory authority, steady, changes subject to established procedures.
Professional: Agreed upon between parties, steady, changes subject to established procedures.
Corporate: Defined by enterprises, changes subject to internal decision-making.
Social: Defined by usage, volatile, continuous and informal changes.
Personal: Customary, defined by named individuals (e.g research paper).
Ontologies & Communication
If collaborations have to cover engineering as well as business descriptions, communication channels and interfaces will have to combine the homogeneous and well-defined syntax and semantics of the former with the heterogeneous and ambiguous ones of the latter.
With ontologies represented as RDF (Resource Description Framework) graphs, the first step would be to sort out truth-preserving syntax (applied independently of domains) from domain specific semantics.
On that basis it would be possible to separate representation syntax from contents semantics, and to design communication channels and interfaces accordingly.
That would greatly facilitate collaborations across externally defined ontologies as well as their mapping to enterprise architecture models.
To summarize, the benefits of ontological frames for collaborative engineering can be articulated around four points:
A clear-cut distinction between representation semantics and truth-preserving syntax.
A common functional architecture for all users interfaces, humans or otherwise.
Modular functionalities for specific semantics on one hand, generic truth-preserving and cognitive operations on the other hand.
Profiled ontologies according to concerns and contexts.
A critical fifth benefit could be added with regard to business intelligence: combined with deep learning capabilities, ontologies would extend the scope of collaboration to explicit as well as implicit knowledge, the former already framed by languages, the latter still open to interpretation and discovery.
Knowledge graphs, which have become a key component of knowlege management, are best understood as a reincarnation of ontologies.
Interestingly, variants of MBSE/MDSE acronyms put the focus on the duality of the concept, software on one side, systems on the other.
As that duality operates for models, systems, and organizations, MBSE offers a holistic view on enterprise architecture.
Models and Software
Models are symbolic representations of actual contexts in line with specific purposes: requirements analysis, simulation, software design, etc. Software is a subset of models characterized by target (computer code) and language (executable instructions). Based on that understanding, MBSE should not be limited to DSLs silos and code generation but employed to bring together and manage the whole range of concerns and artifacts.
Systems and Applications
The hapless track record of Waterfall and the parallel ascent of Agile have clouded the grounds for phased development processes. But whereas agile schemes are the default option when applications can be developed independently, external dependencies prevent their scaling up to system level. That’s when system engineering takes precedence on applications development, with MBSE introduced to manage shared models and support collaboration between teams.
Organization and Projects
As epitomized by agile development models, projects can be driven by specific business needs or shared architecture capabilities. Whereas the former are best carried out iteratively by autonomous teams sharing skills and responsibility, the latter entail collaboration between organizational units along time. MBSE provides the link between standalone projects, phased processes, and enterprise organization.
By providing a holistic view of changes in organizations, systems, and software, MBSE should be a key component of enterprise architecture.
Views can take different meanings, from windows opening on specific data contexts (e.g DB relational theory), to assortments of diagrams dedicated to particular concerns (e.g UML).
Models for their part have also been understood as views, on DB contents as well as systems’ architecture and components, the difference being on the focus put on engineering. Due to their association with phased processes, models has been relegated to a back-burner by agile approaches; yet it may resurface in terms of granularity with model-based engineering frameworks.
Yet, whatever the terminology (layers vs levels), what is at stake is the alignment of two basic scales:
Architectures: enterprise (concepts), systems (functionalities), and platforms (technologies).
Process view: captures the concurrency and synchronization aspects.
Physical view: describes the mapping(s) of software artifacts onto hardware.
Development view: describes the static organization of software artifacts in development environments.
A fifth is added for use cases describing the interactions between systems and business environments.
Whereas these views have been originally defined with regard to UML diagrams, they may stand on their own meanings and merits, and be assessed or amended as such.
Apart from labeling differences, there isn’t much to argue about use cases (for requirements), process (for operations), and physical (for deployment) views; each can be directly associated to well identified parts of systems engineering that are to be carried out independently of organizations, architectures or methods.
Logical and development views raise more questions because they imply a distinction between design and implementation. That implicit assumption induces two kinds of limitations:
They introduce a strong bias toward phased approaches, in contrast to agile development models that combine requirements, development and acceptance into iterations.
They classify development processes with regard to predefined activities, overlooking a more critical taxonomy based on objectives, architectures and life-cycles: user driven and short-term (applications ) vs data-based and long-term (business functions).
These flaws can be corrected if logical and development views are redefined respectively as functional and application views, the former targeting business objects and functions, the latter business logic and users’ interfaces.
That make views congruent with architecture levels and consequently with engineering workshops. More importantly, since workshops make possible the alignment of products with work units, they are a much better fit to model-based engineering and a shift from procedural to declarative paradigm.
Model-based Systems Engineering & Granularity
At least in theory, model-based systems engineering (MBSE) should free developers from one-fits-all procedural schemes and support iterative as well as declarative approaches. In practice that would require matching tasks with outcomes, which could be done if responsibilities on the former can be aligned with models granularity of the latter.
With coarse-grained phased schemes like MDA’s CIM/PIM/PSM (a), dependencies between tasks would have to be managed with regard to a significantly finer artifacts’ granularity.
For agile schemes, assuming conditions on shared ownership and continuous deliveries are met, projects would put locks on “models” at both ends (users’ stories and deliveries) of development cycles (b), with backlogs items defining engineering granularity.
From the enterprise perspective it would be possible to unify the management of changes in architectures across layers and responsibilities: business concepts and organization, functional architecture, and systems capabilities:
From the engineering perspective it would be possible to unify the management of changes in artifacts at the appropriate level of granularity: static and explicit using milestones (phased), dynamic and implicit using backlogs (agile).
UML (Unified Modeling Language) and MDA (Model Driven Architecture) epitomize the lack of focus and consistency of the OMG’s strategy. As it’s safe to assume that there can be no architectures without models, MDA and UML arguably bring sensible (if not perfect) schemes without significant competition.
Unfortunately, not much has been made to play on their obvious complementarity and to exploit their synergies.
Computation independent models (CIMs) describe organization and business processes independently of the role played by supporting systems.
Platform independent models (PIMs) describe the functionalities supported by systems independently of their implementation.
Platform specific models (PSMs) describe systems components depending on implementation platforms.
Engineering can then be managed along architecture layers (a), or carried out as a whole for each application (b).
It’s important to note that the MDA framework is completely neutral with regard to methods: engineering processes can be organized as phased activities (procedural), iterations (agile), or artifacts transformation (declarative).
Logic & The Matter of Models
Whatever the idiosyncrasies and fuzziness of business concerns and contexts, at the end of the day requirements will have to be coerced into the strict logic of computer systems. That may be a challenging task to be carried out directly, but less so if upheld by models.
As it happens, a fact all too often ignored, models come with sound logical foundations that can be used to formalize the mapping of requirements into specifications; schematically, models are to be set in two formal categories:
Descriptive (aka extensional) ones try to classify actual objects, events, and processes into categories.
Prescriptive (aka intensional) ones specify what is expected of systems components and how to develop them.
Interestingly, that distinction provides a formal justification to the one between analysis and design models, the former for the consolidation of requirements across business domains and enterprise organization, the latter for systems and software designs. Such logical foundations could help to manage the mapping of business processes and systems architectures.
UML & the Anatomy of Models
Except scientific computation, there is no reason to assume a-priori congruence between the description of business objects and processes and the specification of the software components. As a corollary, their respective structures and features are better to be dealt with separately.
But that’s not the case at architecture level, where domains and identities have to be managed continuously and consistency on the two sides of the business/system divide. At this level (aka enterprise architecture), responsibilities and identification and communication mechanisms must be defined uniformly.
Compared to MDA set at architecture level, UML describes the corresponding artifacts for business, systems, and platform layers. Regardless of the confusing terminology (layers or levels), that puts MDA and UML along orthogonal dimensions: the former (layers) deals with the nature of contents, the latter (levels) with their structures and features.
Using the same unified modeling language across business, systems, and platform layers is to clearly and directly enhance transparency and traceability; but the full extent of MDA/UML cross-benefits is to appear when models logic is taken into account.
Models & Systems Evolution
As illustrated by the increasing number of systemic crashes, systems obsolescence is no longer a matter of long-term planning but of operational continuity: change has become the rule and as far as complex and perennial systems are concerned, architectures are to evolve while supporting their functional duties seamlessly. If that is to be achieved, modularity and a degree of consistency are necessary between the nature of changes and their engineering. That’s where MDA is to help.
As pointed to above, modularity is best achieved with regard to level (architecture, element) and models contents (business, systems, platforms).
At architecture level, changes in domains, identification, and categories must be aligned between descriptive (enterprise) and prescriptive (systems) models. That will be best achieved with UML models across all MDA layers.
The constraints of continuity and consistency can be somewhat eased at element level: if descriptive (business) and prescriptive (systems) models of structures and features are to be consistent, they are not necessarily congruent. On component (prescriptive/design) side, UML and object-oriented design (OOD) are to keep them encapsulated. As for the business (descriptive/analysis) side, since structures and features can be modeled separately (and OOD not necessarily the best option), any language (UML, BPMN, DSL,etc.) can be used. In between, the structure (aka signature) of messages passed at architecture level is to be specified depending on communication framework.
Considering the new challenges brought about by the comprehensive interoperability of heterogeneous systems, the OMG should reassess the full range of latent possibilities to be found in its engineering portfolio.
The recent paralysis of British Airways world operations (due to a power failure, if officials are to be believed), following the crash of Delta Airlines’ reservation system and a number of similar incidents, once again points to the reliability of large and critical IT systems.
Particularly at risk are airlines or banking systems, whose seasoned infrastructures, at the cutting edge when introduced half a century ago, have been strained to their limit by waves of extensive networked new functionalities. Confronted to the magnitude and complexity of overall modernization, most enterprises have preferred piecemeal updates to architectural leaps. Such policies may bring some respite, but they may also turn into aggravating factors, increasing stakes and urgency as well as shortening odds.
Assuming some consensus about stakes, hazards, and options, the priority should be to overcome jumping fears by charting a reassuring perspective in continuity with current situation. For that purpose models may provide heartening parachutes.
Models: Intents & Doubts
Models can serve two kinds of purposes:
Describe business contexts according to enterprise objectives, foretell evolution, and simulate policies.
Prescribe the architecture of supporting systems and the design of software components.
Frameworks were supposed to combine the two perspectives, providing a comprehensive and robust basis to systems governance. But if prescriptive models do play a significant role in engineering processes, in particular for code generation, they are seldom fed by their descriptive counterpart.
Broadly speaking, the noncommittal attitudes toward descriptive models comes from a rooted mistrust in non executable models: as far as business analysts and software engineers are concerned, such models can only serve as documentary evidence. And since prescriptive models are by nature grounded to systems’ inner making, there is no secure conceptual apparatus linking systemic changes with their technical consequences. Hence the jumping frights.
Overcoming those frights could be achieved by showing the benefits of secure and soft landings.
Models for Secure Landings
As any tools, models must be assessed with regard to their purpose: prescriptive ones with regard to feasibility and reliability of architectures and design, descriptive ones with regard to correctness and consistency. As already noted, compared to what has been achieved for the former, nothing much has been done about the validity of the latter.
Yet, and contrary to customary beliefs, the rigorous verification of descriptive (aka extensional) models is not a dead-end. Of course these models can never be proven true because there is no finite scope against which they could be checked; but it doesn’t mean that nothing can be done to improve their reliability:
Correctness: How to verify that all the relevant individuals and features are taken into account. That can only be achieved empirically by building models open to falsification.
Consistency: How to verify that the symbolic descriptions (categories and connectors) are complete, coherent and non redundant across models and abstraction levels. That can be formally verified.
Alignment: How to verify that current and required business processes are to be seamlessly and effectively supported by systems architectures. That can be managed by introducing a level of indirection, as illustrated by MDA with platform independent models (PIMs) set between computation independent (CIMs) and platform specific (PSMs) ones.
Once established on secure grounds, models can be used to ensure soft landings.
Models for Soft Landings
Set within model based system engineering frameworks, models will help to replace piecemeal applications updates by seamless architectures modernization:
Systems: using models shift the focus of change from hardware to software.
Enterprise: models help to factor out the role of organization and regulations.
Project management: models provide the necessary hinge between agile and phased projects, the former for business driven applications, the latter for architecture oriented ones. Combining both approaches will ensure than lean and just-in-time processes will not be sacrificed to system modernization.
More generally, and more importantly, models are the option of choice (if not the only one) for enterprise knowledge management:
Business: Computation independent models (CIMs), employed to trace, justify and rationalize business strategies and processes portfolios.
Systems: Platform specific models (PSMs), employed to trace, justify and rationalize technical alternatives and decisions.
Decision-making and learning: Platform independent models (PIMs), employed to align business and systems and support enterprise architecture governance.
And knowledge management is arguably the primary factor for successful comprehensive modernization.
Strategic Decision-making: Cash or Crash
Governance is all about risks and decision-making, but investing on truly fail-safe systems for airlines or air traffic control can be likened to a short bet on the Armageddon, and that cannot be easily framed in a neat cost-benefit analysis. But that may be the very nature of strategic decision-making: not amenable to ROI but aiming at risks assessment and the development of the policies apt to contain and manage them. That would be impossible without models.
After a promising inception twenty years ago by the OMG (Object Management Group), UML (Unified Modeling Language) seems to have lost part of its steam and fallen short of initial expectations.
On a general perspective that may be due to the primacy given to the agendas of tools providers, at the expense of users concerns. More specifically, UML practices have been hamstrung by two opposed misconceptions: on one side UML has been associated to OO methods and as a result demoted by non-devotees; on the other side it has been taken as a substitute for programming languages, and so confined to class diagrams and code generation. Hopefully, two major advances in methods and development frameworks, respectively agile and MBSE, could correct those biases and renew UML appeal.
UML Core Artifacts
UML diagrams target five primary artifacts:
Use cases, events, and actors describe what happens between systems and active agents in their environments.
Activities describe what systems are supposed to do when use cases are triggered.
Classes describe systems components.
Sequences describe how systems components collaborate to perform activities.
States and transitions describe the behaviors and synchronization of systems components.
While those diagrams cover the whole of enterprise systems, UML is all too often limited to the description of software components.
UML and Enterprise Systems
As noted above, the lack of clear users guidelines can be seen as a main cause of UML piecemeal and biased adoption. That can be illustrated by the use of activity and class diagrams:
Whereas activity and state diagrams could have been tailored to fully and consistently describe business processes, an alternative notation (BPMN) with arguable benefits is often preferred by business analysts.
Whereas UML is meant to deal with the whole of systems, successful implementations like domain specific languages (DSL) focus on class diagrams and code generation.
But both negative trends could be reversed if changes in methodological or technical environments could put UML users on sounder grounds and give them clearer guidelines.
Methods: Use Cases & Agile
As it happened, use cases have been the main UML innovation, other artifacts having been already introduced by modeling languages. And more to the point, they were meant to be the cornerstone of the “unified” construction, a modeling bridge between business processes and supporting systems. That clearly didn’t happen with traditional (and failed) methods like waterfall, but agile could be more welcoming.
Agile principles put the focus on collaboration and iterative code development, with only sparse mentions of models or processes. But as requirements don’t always come as clear-cut short stories told by well identified business units, use cases may help agile teams to deal with organizational or architectural dependencies:
Open minded and Versatile: use cases are not limited to users because actors (aka roles) are meant to hide the actual agents involved: people, devices, or other systems. As a consequence, the scope of UCs is not limited to dialog with users but may also includes batch (as one-step interactions) and real-time transactions.
Modular and inter-operable: given their simplicity and clarity of purpose, use cases can be easily processed by a wide array of modeling tools on both sides of the business/engineering divide, e.g BPM and UML.
Iterative: given their modularity, use cases can be easily tailored to the iterative paradigm. Once a context is set by business process or user’s story, development iterations can be defined with regard to invariants (use case), iterations (extension points and scenarii), backlog units (activities), and exit condition.
Scalable: use cases provide a sound and pragmatic transition between domain specific stories and architectural features.
That could be the basis of an open relationship between agile development models and UML.
Frameworks: Classes & MBSE
Beyond the various labels (based/driven, system/software, development/engineering), all model based approaches to systems engineering and software development try to replace procedural approaches by declarative ones, and to redefine processes in terms of artifacts transformation.
The benefits of that approach have already been demonstrated at design level through the use of domain specific languages (DSLs). Since effective DSLs usually rely on class diagrams, UML could be used to leverage their success beyond code generation and across the whole of the engineering process.
With artifacts seamlessly and consistently defined and managed within a single engineering framework, procedural schemes could be replaced by declarative ones reusing, editing, or transforming artifacts depending on constraints.
Given their inclination to fall, phases may be a recurring bane of development projects. Agile solutions have emerged as a default option providing that projects can be fully and firmly put under shared ownership and their outcome delivered continuously. But even when such conditions cannot be met lean processes may still be achieved with the help of model based engineering frameworks.
Phased vs Procedural
Not all applications can stand alone and therefore be developed by a cohesive team of business analysts and software engineers delivering a continuous stream of programs. Among the reasons for that:
Stakeholders and decisions-making may spread across organizational units.
Engineering resources may not be available simultaneously and continuously.
The schedules of decisions or deliveries may depend on expected but not forecast changes in technical, business, or regulatory environments.
Traditional approaches to phasing constraints (notoriously Waterfall) have suffered from a bureaucratic bias as they have tried to coerce every project into predefined tasks and procedures targeting standard outcomes and deliverables. And with regard of phasing concerns, the lack of flexibility and built-in mechanisms has been counterproductive: instead of making room for phased decision-making, procedural solutions have resulted in fixed requirements set upfront.
Procedural vs Declarative
Engineering processes are to be considered when, whatever the reasons, activities cannot be performed simultaneously. Whereas procedural approaches deal with the problem with predefined sequences of generic activities, declarative ones directly consider associated input and output flows and set conditions on their processing.
Instead of one-fits-all predefined tasks, work-units can be defined with regard to their impact on development flows. As a consequence processes can be freed of bureaucratic shackles and combine iterative schemes with phasing engineering constraints.
Model Based Engineering
Agile development models are meant to epitomize lean engineering processes as development flows are seamlessly and continuously delivered to customers without the need of intermediate products. Assuming that good schemes have to provide good-enough options, the aim is to build cut-to-the-bone processes for sub-optimal conditions, namely even if agile constraints of shared ownership and continuous delivery cannot be fully satisfied. And that can be achieved with the help of MBSE built-in mechanisms:
The challenge of distributed and differentiated ownership can be dealt with by adjusting the granularity of artifacts with regard to business domains, functional architecture, and platform deployment.
The constraints regarding discrete and phased development and delivery are not to be confronted upfront through planning but dealt with dynamically by balancing users’ drive with artifacts’ phasing constraints.
Lean processes can then be achieved by anchoring model based frameworks to their environment.
Phased Yet Lean Processes
Lean processes can be defined by timed delivery without the use of unnecessary intermediate resources or assets.
Agile development models do the job by putting users’ needs on the driver’s seat and doing away with intermediate artifacts other than code. But shouldn’t backlog items likened to intermediate artifacts ? And in any case compromises may be necessary if users don’t speak with a single organizational voice and milestones are needed to synchronize development flows. For that purpose model based engineering processes have to be coupled with their organizational and technical environments:
Downstream automated transformation are to ensure just-in-time delivery without undue or redundant intermediate documents and models (aka development inventories).
Upstream conceptual (or meta-) models are to ensure semantic consistency across projects.
From a broader perspective that will demonstrate the ultimate benefits of both agile and model based engineering approaches.
Models transformation is generally recognized as the basic mechanism of model based systems engineering (MBSE). Yet, the actual scope of transformations is somewhat limited to design-to-code, and its sequential bias puts MBSE at odds with agile development approaches. Could a revisited understanding help to figure out this apparent discrepancy ?
Traditional transformation paradigm involves ordered sequences of models obtained by applying rules to their immediate predecessor(s). That organizational scheme has three critical consequences, for applicability, economics of reuse, and development processes.
Applicability: the effectiveness of transformations is conditioned by (a) an executable language for the description of targets, and (b) a closed and compact set of unambiguous patterns. Those conditions can only be satisfied for the downstream part of the development process.
Reuse: given the sequencing constraints, models are to be managed and reused along tree-like structures with duplicates introduced at branching points.
Development processes: sequenced models brings forth phased options and leaves out agile solutions.
Assuming those issues are not conclusive, they may be overcame by revisiting the nature of transformations.
Transformation vs Inheritance & Composition
Most of the proposed taxonomies (see references below) put the focus on languages and mechanisms (e.g rules) of sequential transformation without paying enough attention to the nature and the semantics of models contents. Even when abstraction levels are taken into account, the respective contents of each level remain undefined. As it happens, that issue may be the key to a better understanding of models transformation.
To begin with, rule-based transformation has to be compared to inheritance and composition:
Structural inheritance can be used to refine models as to take into account business scenarii previously ignored; e.g special conditions for good customers.
Functional inheritance can be used to introduce new capabilities; e.g new authentication procedures.
Functional composition can be used to apply capabilities across different scenarii; e.g customized authentication procedures.
Rules-based solutions can be used by any kind of transformation.
That taxonomy implies a clear distinction between operations executed within the same level of abstraction and those targeting artifacts defined at different levels: contrary to rules-based transformations, inheritance and composition can only be applied to artifacts sharing common semantics.
While that would clearly prevent their use for models organized along abstraction levels, semantic pitfalls could be mastered for models built from artifacts from different abstraction levels.
Releasing models from (still to be defined) abstraction levels would bring two critical benefits:
Whatever the terminology (abstract, conceptual, functional, etc.), abstraction semantics are much easier to define for artifacts than for models.
That would remove a chunk of restrictions on the design of transformation processes.
In that case transformation rules could be turned into combination ones and sequential transformation turned into cross-breeding.
Mendel, Models, Mongrels
Taking a cue from Gregor Mendel’s use of cross-fertilization, the aim of a revisited transformation paradigm would be three-fold:
To refine the granularity of reuse, from models to artifacts
To substitute combination for sequential transformation whenever possible.
To substitute graphs for trees, with models organized along two basic layers, final (aka mongrels) or reusable (aka blueprints).
As far as MBSE is concerned, the genetics metaphor helps to clarify the nature of abstraction. Conceptually, it introduces a distinction between artifacts and models:
With regard to artifacts, abstraction layers are defined by scope: enterprise, systems, platforms.
With regard to models, abstraction layers are defined by capabilities: reusable (stable traits), or final (recessive traits).
That taxonomy is corroborated by its functional counterpart: artifacts transformation is carried out with inheritance and composition, models transformation relies on combination.
More important, that understanding goes a long way solving the issues regarding scope, reuse, and development processes.
Scope: Weaving Analysis & Design Traits
Definitions and taxonomies should always be assessed with regard of their applicability. On that account there isn’t much to say for abstraction layers applied to models: they don’t fit because too many traits can be defined across different layers, e.g: business rules, authentication, encryption, etc.
That difficulty can be neatly and consistently removed by models built from artifacts defined at different levels.
Models Reuse: Blueprints vs Mongrels
Reuse is all too often seen as a contentious objective with inconclusive ROI. One one hand it requires significant overheads to manage the resources, on the other hand the outcomes can introduce regressive traits. The distinction between sound reusable models and final ones significantly reduces both the costs of the former and the risks of the latter.
Processes Organization: MBSE & Agile
Model based systems engineering and the agile development model are arguably two of the most conclusive approaches to software engineering. Unfortunately they are often seen as difficult bedfellows, principally (but not uniquely) because the former insists on the importance of models with some bias toward phased processes, while the latter is all for iterative processes with models mentioned as an afterthought, if at all. Yet, both approaches could be made complementary on condition that models could be processed iteratively. And that could be achieved if sequenced transformations of homogeneous models would be replaced by the combination of heterogeneous ones.
Within such a framework an agile team could, e.g, iteratively develop new business requirements, taking into account existing functionalities (a) and business rules (b), and generate code (c).