Category: Software Engineering

Models Truth and Disproof

“If you cannot find the truth right where you are, where else do you expect to find it?”

Dōgen Zenji

Summary

Software engineering models can be regrouped in two categories depending on their target: analysis models represent business context and concerns, design ones represent systems components. Whatever the terminologies, all models are to be verified with regard to their intrinsic qualities, and validated with regard to their domain of discourse, respectively business objects and activities (analysis models), or software artifacts (design models).

(Chris Engman)
Internal & External Consistency (Chris Engman)

Checking for internal consistency is arguably straightforward as proofs can be built with regard to the syntax and semantics of modeling (or programming) languages. Things are more complicated for external consistency because hypothetical proofs would have to rely on what is known of the business domains, whose knowledge is by nature partial and specific, if not hypothetical. Nonetheless, even if general proofs are out of reach, the truth of models can still be disproved by counter examples found among the instances under consideration.

Domains of Discourse: Business vs Engineering

With regard to systems engineering, domains of discourse cover artifacts which are, by “construct”, fully defined. Conversely, with regard to business context and objectives, domains of discourse have to deal with instances whose definitions are a “work in progress”.

vvvv
Domains of Discourse: Business vs Engineering

That can be illustrated by analysis models, which are meant to translate requirements into system functionalities, as opposed to design ones, which specify the corresponding software artifacts. Since software artifacts are supposed to be built from designs, checking the consistency of the mapping is arguably a straightforward undertaking. That’s not the case when the consistency of analysis models has to be checked against objects and activities identified by business’ domains of discourse, possibly with partial, ambiguous, or conflicting descriptions. For that purpose some logic may help.

Flat Models & Logic

Business requirements describe objects, events, and activities, and the purpose of modeling is to identify those instances and regroup them into subsets built according to their features and relationships.

Building descriptions for targeted instances business objects & activities
How to organize instances of business objects & activities into subsets

As far as models make no use of abstractions (“flat” models), instances can be organized using basic set operators and epistemic (i.e relating to the degree of validation) constraints with regard to existence (m/d), uniqueness (x/o), and change (f/m):

cccc
Notation for epistemic constraints

Using the EU-Rent Car example:

  • Rental cars are exclusively and definitively partitioned according to models (mxf).
  • Models are exclusively partitioned according to rental group (mxm), and exclusively and definitively according body style (mxf).
  • Rental cars are partitioned by derivation (/) according to group and body style.
cccc
Flat model using basic set operators for exclusive (cross) and final (grey) partitions (2)

Such models are deemed to be consistent if all instances are consistently taken into account.

Flat Models External Consistency

Assuming that models backbone can be expressed logically, their consistency can be formally verified using a logical language, e.g Prolog.

To begin with, candidate subsets are obtained by combing requirements for core modeling artifacts expressed as predicates (21 for descriptions of actual objects, 121 for descriptions of actual locations, 20 for descriptions of symbolic ones, 22 for descriptions of symbolic partitions), e.g:

  • type(20, manufacturer).
  • type(21, rentalCar).
  • type(22,  carModel).
  • type(22, rentalGroup).
  • type(22,  bodyStyle).
  • type(121, depot).

Partitions and functional connectors (220 for symbolic reference, 222 for partitions, 221 for actual connection), e.g:

  • connect(222, rentalCar,carModel, mxf).
  • connect(222, carModel, group, mxm).
  • connect(222, carModel, bodyStyle,mxf).
  • connect(220, manufacturer_, carModel, manufacturer, mof).
  • connect(121, location, rentalCar, depot, mxt)

Finally, features and structures (320 for properties, 340 for operations), e.g:

  • feature(340, move_to, depot).
  • feature(320, address).
  • feature(320, location).
  • member(manufacturer,address,mom).
  • member(rentalCar,location,mxm).
  • member(rentalCar,move_to,mxm).

Those candidate descriptions are to be assessed and improved by applying them to sets of identified occurrences taken from requirements. The objective being to map each instance to a description: instance(name, term()), e.g:

  • instance(sedan,carModel(f1(F1),f2(F2))).
  • instance(coupe,carModel(f1(F1),f2(F2))).
  • instance(ford, manufacturer(f6(F6),f7(F7))).
  • instance(focus, rentalCar(f6(F6),f7(F7))).
  • instance_(manufacturer_,focus,ford).

Using a logical interpreter, validation can then be carried out iteratively by looking for counter examples that could disprove the truth of the representations:

  • All instances are taken into account: there is no instance N without instance(N,Structure).
  • Logical consistency: there is no instance N with conflicting partitioning (native and derived).
  • Completeness: there is no instance type(X,N,T(f1,f2,..)) with undefined feature fi.
  • Functional consistency: there is no instance of relation R (native and derived) without a consistent type relation(X, R, Origin, Destination, Epistemic) .

It must be noted that the approach is not limited to objects and is meant to encompass the whole scope of requirements: actual objects, symbolic representations, business logic, and processes execution.

Multilevel Models: From Partitions to Sub-types

Flat models fall short when specific features are to be added to the elements of partitions subsets, and in that case sub-types must be introduced. Yet, and contrary to partitions, sub-types come with abstractions: set within a flat model (i.e without sub-types), Car model fully describes all instances, but when sub-types sedan, coupe, and convertible are introduced, the Car model base type is nothing more than a partial (hence abstract) description.

ccc
From partition to sub-types: subsets descriptions are supplemented with specific features.

Whereas that difference may seem academic, it has direct and practical consequences when validation is considered because consistency must then be checked for every level, concrete or abstract.

LSP & External Consistency

As it happens, the corresponding problem has been tackled by Barbara Liskov for software design: the so-called Liskov substitution principle (LSP) states that if S is a sub-type of T, then instances of T may be replaced with instances of S without altering any of the desirable properties of the program.

Translated to analysis models, the principle would state that, given a set of instances, a model must keep its consistency independently of the level of abstraction considered. As a corollary, and assuming a model abides by the substitution principle, it would be possible to generalize the external consistency of a detailed level to the whole model whatever the level of abstraction. Hence the importance of compliance with the substitution principle when introducing sub-types in analysis models.

vvv
All instances must be accounted for whatever the level of abstraction

Applying the Substitution Principle to Analysis Models

Abstraction is arguably the essence of requirements modeling as its purpose is to bring specific and changing concerns under a common, consistent, and lasting conceptual roof. Yet, the two associated operations of specialization and generalization often receive very little scrutiny despite the fact that most of the related pitfalls can be avoided if the introduction of sub-types (i.e levels of abstraction) is explicitly justified by partitions. And that can be achieved by the substitution principle.

First of all, and as far as requirements analysis is concerned, sub-types should only to be introduced for specific features, properties or operations. Then, epistemic constraints can be used to tally the number of specialized instances with the number of generalized ones, and check for the possibility of functional discrepancies:

  • Discretionary (or conditional or non exhaustive) partitions (d__) may bring about more instances for the base type (nb >= ∑nbi).
  • Overlapping (or duplicate or non isolated) partitions (_o_) may bring about less instances for the base type (nb <= ∑nbi).
  • Assuming specific features, mutable (or reversible) partitions (__m) means that features may differ between level; otherwise (same features) sub-types are not necessary.
vvv
Epistemic constraints on partitions can be used to enforce the LSP

Using a Prolog-like language, the only modification will concern the syntax of predicates, with structures replaced by lists of features:

  • type(20, manufacturer,[f6,f7]).
  • type(21, rentalCar,[f5]).
  • type(22,  carModel,[f1,f2]).
  • type(22, rentalGroup,[f9]).
  • type(22,  bodyStyle,[f8]).
    • type(20, bodyStyle:sedan, [f11,f12]).
    • type(20, bodyStyle:coupe, [f13]).
    • type(20, bodyStyle:convertible, [f14]).
  • type(121, depot,[f10]).

The logical interpreter could then be used to map the sub-types to partitions and check for substitution.

Further Reading

Further Readings

2015: A Scraps-Free New Year

As chances are for new years to come with the legacies of past ones, that would be a good time to scrub the scraps.

Sifting through requirements subhodGupta
How to scrub last year scraps (Subhod Gupta)

Legacies as Forced Reuses

As far as enterprise architectures are concerned, sanguine resolutions or fanciful wishes notwithstanding, new years seldom open the door to brand new perspectives. More often than not, they bring new constraints and further curb the possible courses of action by forcing the reuse of existing assets and past solutions. That will call for a review and assessment of the irrelevancies or redundancies in processes and data structures lest they clog the organization and systems, raise entropy, and degrade governance capability.

Architectures as Chosen Reuses

Broadly defined, architectures combine assets (physical or otherwise) and mechanisms (including organizational ones) supporting activities which, by nature, must be adaptable to changing contexts and objectives. As such, the primary purpose of architectures is to provide some continuity to business processes, across locations and between business units on one hand, along time and business cycles on the other hand. And that is mainly to be achieved through reuse of assets and mechanisms.

Balancing Changes & Reuse

It may be argued that the main challenge of enterprise architects is to maintain the right balance between continuity and agility, the former to maintain corporate identity and operational effectiveness, the latter to exploit opportunities and keep an edge ahead of competitors. That may turn to be an oxymoron if architects continuously try to discard or change what was designed to be kept and reused. Yet that pitfall can be avoided if planting architectures and pruning offshoots are carried out independently.

Seasons to Plant & to Prune

Enterprises life can be set along three basic time-scales:

  • Operational: for immediate assessment and decision-making based on full information.
  • Tactical: for periodic assessment and decision-making based on partially reliable information. Periodicity is to be governed by production cycles.
  • Strategic: for planned assessment and decision-making based on unknown or unavailable information. Time-frames are to be governed by business models and risks assessment.

Whereas architecture life-cycles are by nature strategic and meant to span an indefinite, but significant, number of production cycles, trimming redundancies can be carried on periodically providing it doesn’t impact processes execution. So why not doing the house cleaning with the beginning of new years.

Further Reading

How to Mind a Tree Story

Summary

Depending on devotees or dissenters, the Agile development model is all too often presented as dead-end or end-of-story. Some of that unfortunate situation can be explained, and hopefully pacified, by comparing users’ stories to plants, with their roots, trunks, and branches. Assuming that agility calls for sound footings and good springboards, it may be argued that many problems arise with stories barking at the wrong tree (application level) or getting lost in the woods (architecture level).

How to Mind/Mend a True/Tree Story
How to Mind/Mend a True/Tree Story

Application level: Trees, Bushes and Hedges

As Aristotle first stated, good stories have to follow the three unities: one course of action, located in a single space, run along continuous time.

That rule is clearly satisfied  by stories that can be developed like plants growing from clearly identified roots.

Yet, stories like bushes may grow too many offshoots to be accounted for by a single action narrative; in that case it may be possible to single out a primary trunk and a set of forking branches along which different scenarii could be developed.

More serious difficulties may appear with thickets mixing offshoots from different bushes sharing the same space. That situation will first require some ground work in order to single out individual roots, and then use them to extricate each bush separately. When, like offshoots that actually mingle, story-lines cross and share actions, the description of such actions (aka features) is to be factored out and separated from the contexts of their enactment in the different story-lines.

Finally, like bushes in hedges, stories may chronicle repeated activities serving some collective purpose. That configuration is both easy to recognize and dealt with effectively by introducing a stereotyped story feature for collections and loops management.

Architecture level: Groves, Woods and Plantations

Contrary to hedges which are built on the similarity of their constituents, groves are based on their functional differences, and that can also be seen as a critical distinction between containers and architectures.

In agile parlance, that is best compared to the difference between stories and epics, the former telling what happens between users and applications, the latter taking a bird’s view of the relationships between business processes and systems.

In most of the cases the question will arise for sizable stories deemed too large for development purposes. When dealing with that situation the first step should be to look for thickets and bushes, respectively to be set apart as individual bushes or refined as scenarii. When still confronted with multiple roots, the question would be to decide between hedges and groves, that is between repeated activities and collaboration. And that decision would be critical because collaborations call for a different kind of story (aka epics or themes) set at a higher level, namely architecture.

Scaling Ups and Downs

Assuming the three-units rule cannot be met, two alternative approaches are possible, depending on whether the story has to be broken down or upgraded to an epic, and the undoing of the rule can be used to make a decision:

  1. When the course of actions, once started, is to be contingent on subsequent business (aka external) events the story should be upgraded to an epic, as it will often refer to a part or whole of a business process.
  2. Otherwise: when activities are set along different periods of time (i.e contingent on time-events) the story can be broken down depending on size, functional architecture, or development constraints.
  3. Otherwise: when activities are distributed across locations it may be necessary to factor out architecture-dependent features dealing with shared address spaces and synchronization mechanisms

Applying those guidelines to stories will put the whole development processes on rails and help to align requirements with their architectural footprint: business logic, system functionalities, or platform technologies.

Further Readings

External Links

Alignment for Dummies

Summary

The emergence of Enterprise Architecture as a discipline of its own has put the light on the necessary distinction between actual (aka business) and software (aka system) realms. Yet, despite a profusion of definitions for layers, tiers, levels, views, and other modeling perspectives, what should be a constitutive premise of system engineering remains largely ignored, namely: business and systems concerns are worlds apart and bridging the gap is the main challenge of architects and analysts, whatever their preserve.

jBaldessari_900x450
(Alignment with Dummies (J. Baldessari)

The consequences of that neglect appear clearly when enterprise architects consider the alignment of systems architectures and capabilities on one hand, with enterprise organization and business processes on the other hand. Looking into the grey zone in between, some approaches will line up models according to their structure, assuming the same semantics on both sides of the divide; others will climb up the abstraction ladder until everything will look alike. Not surprisingly, with the core interrogation (i.e “what is to be aligned ?”) removed from the equation, models will be turned into dummies enabling alignment to be carried out by simple pattern matching.

Models & Views

The abundance of definitions for layers, tiers or levels often masks two different understandings of models:

  • When models are understood as symbolic descriptions of sets of instances, each layer targets a different context with a different concern. That’s the basis of the Model Driven Architecture (MDA) and its distinction between Computation Independent Models (CIMs), Platform Independent Models (PIMs), and Platform Specific Models (PSMs)
  • When models are understood as symbolic descriptions built from different perspectives, all layers targets the same context, each with a different concern. Along that understanding each view is associated to a specific aspect or level of abstraction: processes view, functional view, conceptual view, technical view, etc.

As it happens, many alignment schemes use, implicitly or explicitly, the second understanding without clarifying the underlying assumptions regarding the backbone of artifacts. That neglect is unfortunate because, to be of any significance, views will have to be aligned with regard to those artifacts.

What is to be aligned

From a general perspective, and beyond lexical controversies, alignment has to be managed with regard to two basic scales:

  • Architectures: enterprise (concepts), systems (functionalities), and platforms (technologies).
  • Models: conceptual (business context and organization), analysis (symbolic representations), design (physical implementation).

From a practical point of view, alignment is meant to deal with two main problems: how business processes are supported by systems functionalities, and how those functionalities are to be implemented. Given that the latter can be fully dealt with at system level, the focus can be put on the alignment of business processes and functional architectures.

A naive solution could be to assume services on both processes and systems sides. Yet, the apparent symmetry covers a tautology: while aiming for services oriented architectures on the systems side would be legitimate, if not necessarily realistic, taking for granted that business processes also tally with services would presume some prior alignment, in other words that the problem has already been solved.

The pragmatic and logically correct approach is therefore to map business processes to system functionalities using whatever option is available, models (CIMs vs PIMs), or views (processes vs functions). And that is where the distinction between business and software semantics is critical: assuming the divide can be overlooked, some “shallow” alignment could be carried out directly providing the models can be translated into some generic language; but if the divide is acknowledged a “deep” alignment will have to be supported by a semantics bridge built across.

Shallow Alignment

Just like models are meant to describe sets of instances, meta-models are meant to describe instances of models independently of their respective semantics. Assuming a semantic continuity between business and systems models, meta-models like OMG’s KDM (Knowledge Discovery Meta-model) appear to provide a very practical solution to the alignment problem.

From a practical point of view, one may assume that no model of functional architecture is available because otherwise it would be aligned “by design” and there would be no problem. So something has to be “extracted” from existing software components:

  1. Software (aka design) models are translated into functional architectures.
  2. Models of business processes are made compatible with the generic language used for system models.
  3. Associations are made based on patterns identified on each side.

While the contents of the first and third steps are well defined and understood, that’s not the case for the second step which take for granted the availability of some agreed upon modeling semantics to be applied to both functional architecture and business processes. Unfortunately that assumption is both factually and logically inconsistent:

  • Factually inconsistent: it is denied by the plethora of candidates claiming for the role, often with partial, overlapping, ambiguous, or conflicting semantics.
  • Logically inconsistent: it simply dodges the question (what’s the meaning of alignment between business processes and supporting systems) either by lumping together the semantics of the respective contexts and concerns, or by climbing up the ladder of abstraction until all semantic discrepancies are smoothed out.

Alignments built on that basis are necessarily shallow as they deal with artifacts disregarding of their contents, like dummies in test plans. As a matter of fact the outcome will add nothing to traceability, which may be enough for trivial or standalone processes and applications, but is to be meaningless when applied at architecture level.

Deep Alignment

Compared to the shallow one, deep alignment, instead of assuming a wide but shallow commonwealth, tries to identify the minimal set of architectural concepts needed to describe alignment’s stake. Moreover, and contrary to the meta-modelling approach, the objective is not to find some higher level of abstraction encompassing the whole of models, but more reasonably to isolate the core of architecture concepts and constructs with shared and unambiguous meanings to be used by both business and system analysts.

That approach can be directly set along the MDA framework:

Basicam_languages
Deep alignment makes a distinction between what is at stake at architecture level (blue), from the specifics of process or domain (green), and design (brown).
  • Contexts descriptions (UML, DSL, BPM, etc) are not meant to distinguish between architectural constructs and specific ones.
  • Computation independent models (CIMs) describe business objects and processes combining core architectural constructs (using a generic language like UML), with specific business ones. The former can be mapped to functional architecture, the latter (e.g rules) directly transformed into design artifacts.
  • Platform independent models (PIMs) describe functional architectures using core constructs and framework stereotypes, possibly enriched with specific artifacts managed separately.
  • Platform specific models (PSMs) can be obtained through transformation from PIMs, generated using specific languages, or refactored from legacy code.

Alignment can so focus on enterprise and systems architectural stakes leaving the specific concerns dealt with separately, making the best of existing languages.

Alignment & Traceability

As mentioned above, comparing alignment with traceability may help to better understand its meaning and purpose.

  • Traceability is meant to deal with links between development artifacts from requirements to software components. Its main objective is to manage changes in software architecture and support decision-making with regard to maintenance and evolution.
  • Alignment is meant to deal with enterprise objectives and systems capabilities. Its main objective is to manage changes in enterprise architecture and support decision-making with regard to organization and systems architecture.

cccc

As a concluding remark, reducing alignment to traceability may counteract its very purpose and make it pointless as a tool for enterprise governance.

Further readings

EA Documentation: Taking Words for Systems

In so many words

Given the clear-cut and unambiguous nature of software, how to explain the plethora of  “standard” definitions pertaining to systems, not to mention enterprises, architectures ?

Documents and architectures, which grows on the other (Gilles Barbier).
Documents and Systems: which ones nurture the others (Gilles Barbier).

Tentative answers can be found with reference to the core functions documents are meant to support: instrument of governance, medium of exchange, and content storage.

Instrument of Governance: the letter of the law

The primary role of documents is to support the continuity of corporate identity and activities with regard to their regulatory and business environments. Along that perspective documents are to receive legal tender for the definitions of parties (collective or individuals), roles, and contracts. Such documents are meant to support the letter of the law, whether set at government, industry, or corporate level. When set at corporate level that letter may be used to assess the capability and maturity of architectures, organizations, and processes. Whatever the level, and given their role for legal tender or assessment, those documents have to rely on formal textual definitions, possibly supplemented with models.

Medium of Exchange: the spirit of the law

Independently of their formal role, documents are used as medium of exchange, across corporate entities as well as internally between their organizational units. When freed from legal or governance duties, such documents don’t have to carry authorized or frozen interpretations and assorted meanings can be discussed and consolidated in line with the spirit of the law. That makes room for model-based documents standing on their own, with textual definitions possibly set in the background. Given the importance of direct discussions in the interpretation of their contents, documents used as medium of (immediate) exchange should not be confused with those used as means of storage (exchange along time).

Means of Storage: letter only

Whatever their customary functions, documents can be used to store contents to be reinstated at a later stage. In that case, and contrary to direct (aka immediate) exchange, interpretations cannot be consolidated through discussion but have to stand on the letter of the documents themselves. When set by regulatory or organizational processes, canonical interpretations can be retrieved from primary contexts, concerns, or pragmatics. But things can be more problematic when storage is performed for its own purpose, without formal reference context. That can be illustrated by legacy applications with binary code can be accompanied by self-documented source code, source with documentation, source with requirements, generated source with models, etc.

Documentation and Enterprise Architecture

Assuming that the governance of structured social organizations must be supported by comprehensive documentation, documents must be seen as a necessary and intrinsic component of enterprise architectures and their design should be aligned on concerns and capabilities.

As noted above, each of the basic functionalities comes with specific constraints; as a consequence a sound documentation policy should not mix functionalities. On that basis, documents should be defined by mapping purposes with users across enterprise architecture layers:

  • With regard to corporate environment, documentation requirements are set by legal constraints, directly (regulations and contracts) or indirectly (customary framework for transactions, traceability and audit).
  • With regard to organization, documents have to met two different objectives. As a medium of exchange they are meant to support the collaboration between organizational units, both at business level (processes) and across architecture levels. As an instrument of governance they are used to assess architecture capabilities and processes performances. Documents supporting those objectives are best kept separate if negative side effects are to be avoided.
  • With regard to systems functionalities, documents can be introduced for procurements (governance), development (exchange), and change (storage).
  • Within systems, the objective is to support operational deployment and maintenance of software components.
Documents’ purposes and users
Documents’ purposes and users

The next step will be to integrate documents pertaining to actual environments and organization (brown background) with those targeting symbolic artifacts (blue background).

EmergA_ActSmb
Models are used to describe actual or symbolic objects and behaviors

That could be achieved with MBE/MDA approaches.

Further readings

 

Agile between Space & Time

Continuity of delivery and locus

While the scope of agile principles extends far beyond the eponymous methods, some of them are more specific and their applicability contingent on contexts and objectives; two of them are especially important as they entail very specific assumptions with regard to time and space:

  • Continuous delivery of valuable software.
  • Direct collaboration between software users and developers all along the development process.
Material inconsistency (H. Zamora)
Continuity and collaboration  (H. Zamora)

Time Capsules vs Time Scales

If time is seen as a discrete gauge introduced to sequence events, it ensues that continuity indicates that no external events are to be taken into account. In other words, once its objectives have been set, software development must be carried out according its own pace, whatever happens in business context and enterprise organization. That’s a pretty strong assumption that should be explicitly endorsed by stakeholders and users.

Interestingly, this assumption can be set against fixed requirements and upfront design associated with waterfall approaches, as if development policies were to be set between two alternative options:

  1. Time capsules: projects deal with changing requirements subject to frozen business context and organization.
  2. Time scales: projects deal with frozen requirements subject to planned changes in business context and organization.

A pragmatic approach should take the best of each option: put limited and self-contained objectives into capsules and organize capsules along scales.

Collaborative vs Procedural Spaces

The reason for processes is that tasks cannot be carried out simultaneously, and the reason for human contribution is that some tasks involve decision-making based on circumstances and provisos that cannot be determined upfront.

Those decisions may concern different domains of concerns with their respective objectives and roles: business requirements, systems functionalities, or technical implementations. Agile and phased approaches clearly disagree about how those decisions are to be taken and conveyed along  processes and across organizational spaces,  the former  ruling that all the parties should deal with them jointly, the latter opting for a separation of concerns. As a counterpart to time continuity, agile recommendation implies some continuity across organizational spaces that cannot be taken for granted. Whether that proviso can be met will determine the choice of a development policy:

  • Collaborative: problems are solved and decisions taken through collaboration between parties within a single organizational space.
  • Procedural: parties deal separately with problems and decision-making within their respective organizational spaces, and communication between those spaces is carried out through prearranged channels.

Those options clearly depend on organizational and technical environments, hence the benefits of charting projects with regard to constraints on ownership and delivery.

Charting projects: Ownership and Delivery

Given that choosing the right development model is a primary factor of success, decision-making should rely on simple and robust criteria. First, if decisions are to be taken and implemented collectively, shared governance has to be secured. Second,  if deliveries are to be carried out continuously, projects shouldn’t be dependent on external events.

When charted with regard to ownership and delivery, projects can be regrouped into four basic categories:

  • Business requirements with distinct stakeholders and objectives directly associated with business value (a).
  • Business requirements with objectives and business value involving separate stakeholders (b).
  • Functional requirements targeting features used by different applications (c).
  • Non functional requirements targeting features used by different applications across different systems (d).
vvv
Charting projects with regard to ownership and delivery constraints

That straightforward taxonomy provide clear guidelines for project managers: options (a) and (d) associated respectively with agile and phased approaches, and options (b) and (c) to be decided with regard to actual projects, the ability of organizations to accomodate particular development schemes, and long term objectives regarding engineering processes.

Further Readings

The Scope of Agile Principles

Objective

The Agile development model as pioneered by the eponymous Manifesto is based both on universal principles meant to be applied in any circumstances, and on more specific ones subject to some prerequisites. Sorting out the provisos may help to extend and improve the agile footprint.

The flexibility of Agile principles (E. de Souza)
The flexibility of Agile principles (E. de Souza)

Checklist

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

  • Scope: Specific to agile development model.
  • Requisite: Iterative and incremental development.

2. Welcome changing requirements, even late in  development. Agile processes harness change for  the customer’s competitive advantage.

  • Scope: Universal.
  • Requisite: Requirements traceability and modular design.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

  • Scope: Universal.
  • Requisite: Modular design and incremental development.

4. Business people and developers must work together daily throughout the project.

  • Scope: Specific to agile development model.
  • Requisite: Shared ownership, no external dependencies.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

  • Scope: Universal.
  • Requisite: Dedicated organization and human resources policy.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

  • Scope: Universal.
  • Requisite: Corporate culture.

7. Working software is the primary measure of progress.

  • Scope: Universal.
  • Requisite: Quality management and effective assessment of returns and costs.

8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

  • Scope: Universal.
  • Requisite: Dedicated project management and human resources policy.

9. Continuous attention to technical excellence and good design enhances agility.

  • Scope: Universal.
  • Requisite: Corporate culture and human resources policy.

10. Simplicity–the art of maximizing the amount of work not done–is essential.

  • Scope: Universal.
  • Requisite: Quality management and corporate culture.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

  • Scope: Universal.
  • Requisite: Shared ownership and no external dependencies.

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

  • Scope: Universal.
  • Requisite: Dedicated organization and corporate culture

Assessment

Perhaps surprisingly, only two (#1 and #4) out of twelve principles cannot be applied universally as they clearly conflict with phased processes and tasks differentiation. In other words ten out of twelve of the agile principles could bring benefits to all and every type of project.

Further Reading

External Links

Ergonomy, Fingertips Errors & Automated Testing

Objective

When interacting with systems, users do things they aren’t supposed to do and walk along irrelevant, even unthinkable, paths that can put tests designers at a loss. This apparent chink between users’ conscious self and their fingertips can be explained by the way humans assess situations and make decisions. Curtailing it is the aim of ergonomics.

Errors at fingerstips (Rembrandt)
Anatomy of Errors: from brain to fingers (Rembrandt)

Taking a leaf from A. Tversky and D. Kahneman (who received the 2002 Nobel Prize in Economics), decision-making relies on two cognitive mechanisms:

  1. The first one “operates automatically and quickly, with little or no effort and no sense of voluntary control”. It’s put in use when actual situations must be assessed and decisions taken rapidly if not instantly.
  2. The second one “allocates attention to the effortful mental activities that demand it, including complex computations”. It’s put in use when situations can be assessed with regard to past experience in order to support informed decisions making.

That distinction can be directly applied to users’ behaviors interacting with systems:

  1. Intuitive behavior: decisions are taken on the basis of the visual context and options as presented by users interfaces before taking into account underlying business contents and logic.
  2. Rational behavior: decisions are taken on the basis of business contents and logic disregarding supporting systems interfaces.

Set in context, that distinction can be put in parallel (but not confused) with the one between domain and functional requirements, the former dealing rationally with business objects and logic, the latter putting the former to use through interactions with supporting systems.

Functional requirements describe the part played by supporting systems
Functional requirements describe the part played by supporting systems

Assuming that business logic should not be contingent on supporting systems interfaces, the best option would be to test its implementation independently of users interactions; moreover, tests targeting intuitive behaviors (i.e not directly based on domain specific contents), could then be generated automatically.

Looking for Errors

Given that testing is meant to find flaws in deliverables, tests are certainly more effective when designers know what they are looking for.

For that purpose phased approaches rely on sequences of differentiated tests dealing successively with programming (unit tests), functional requirements (integration tests), and business requirements (acceptance tests).  The unfortunate downside of those policies is that the most wide-ranging flaws are the last to be looked for, with the risk of being found after cascading and costly consequences for functionalities and programs.

Phased and Iterative approaches to tests
Phased and Iterative approaches to tests

Conversely, agile approaches follow iterative policies, with each development cycle combining the definition, programming, and tests of software products. When properly implemented those policies significantly improve the early detection and correction of errors whatever their origin. Yet, since there is no explicit management of intermediate outcomes, it’s difficult to differentiate the tests according the kind of errors to look for, e.g faulty business rules implementation or flawed user interface.

Architecture driven approaches may provide an answer, with requirements unambiguously sorted out depending on their architectural footprint: business contents or system functionalities. As a corollary, tests could also be designed along the same lines, targeting business rationale or human behavior.

Errors in Mirrors

Acceptance tests being performed with regard to requirements, they should be designed along requirements taxonomy, respectively for business logic, users’ interactions, quality of services, and components implementation. Being aligned on requirements, those tests can be neatly defined with regard to closed sets of specifications, functional or otherwise.

Functional tests have to expect the unexpected
Functional tests have to expect the unexpected

But that’s not the case for users’ interactions because people behaviors are not fully predictable; hence, while tests can be systematically designed with regard to the set of users’ actions framed by business and functional requirements, there is no way to comprehensively and unambiguously check for all and every possible behavioral contingencies. That will make for three levels of functional tests:

  1. Implementation of business logic: tests should be designed directly from business requirements, independently of interactions with users.
  2. Implementation of scenarii: while interactions are defined in reference to business logic, their validation should focus on the presentation of contents and dialog control.
  3. Users exceptions: in addition to inputs validity, already checked with business logic, and users’ actions, supposedly secured by interaction scenarii, it is necessary to check that unexpected behaviors have been properly considered .
How to check that unexpected behaviors have been properly considered ?
How to check that unexpected behaviors have been properly considered ?

In other words, functional tests will have to look simultaneously for errors in software (defined with regard to a finite set of requirements), and for users’ mistakes (set in an open range of behaviors). As if tests designers were to mirror users errors in order to look for software ones. So, assuming that errors in business logic and interactions have been considered, what should still be checked, and how ?

Fingertips Errors

When faced with choices, users bank on mental maps combining graphical and business layers, with the implicit assumption that maps’ contexts and concerns are kept up to date. Those maps combine three communication mechanisms:

  • Languages, natural or specific, use syntax and semantics to define business contents, logic, and operations.
  • Icons use similarity for the visual representation of business operations or functional primitives (e.g create, delete, etc).
  • Signals use proximity to draw users’ attention to predefined events (e.g sounds for operations completion or incoming emails).

While language-based interactions are supposedly fully covered by business and functional tests, icons and signals make room for “fingertips” reactions which cannot be directly framed within business logic or functional scenarii, and therefore cannot be comprehensively checked for erroneous behaviors.

Icons and signal based communication can trigger unexpected behaviors.
Icons and signal based communication can trigger unexpected behaviors.

Yet, if instinctive reactions preclude rational considerations, decisions may be swayed by analogies and associations before being informed by the relevant business contents. To prevent that risk, test scenarii built on business logic and functional interactions should be extended in order to take into account the intuitive aspects of users’ behaviors.

Mental Maps & Automated Tests

As noted above, mental maps are built on three layers, one deep (language semantics) and two shallow (icons and signals). While the shallow layers are supposed to reference the deep one, icons and signals may induce instinctive behaviors independently of the referenced business logic. Those behaviors can be triggered by two kinds of mechanisms:

  • Analogy: users will look for similarities and familiar configurations.
  • Proximity: users will look for continuity with regard to scope and operations.

Clearly, lapses in such behaviors will normally escape tests designed for business and functional requirements; yet, by being driven by self-contained mechanisms, intuitive behaviors can be checked independently of references to business contents. And that may open the door to automated tests generation.

With regard to similarities, tests should look for possible confusion between:

  • Objects with common representation but specific features (inheritance).
  • Operations with shared semantics but different scope (polymorphism).
  • Sequences with shared operations but different timing .

With regard to proximity, tests should look for possible confusion between:

  • Objects and their parts, or between their parts (structural proximity).
  • Operations usually associated into the same activity (functional proximity).
  • Operations usually executed successively (chronological proximity).

Scripts for such tests could be generated through pattern-matching and run by wizard applications.

Further Reading

External Links

2014: AAA+ for a New Year

Continuity

As illustrated by the millennium bug, systems perseverance often defy human expectations, as if continuity was a self-acquired characteristic of systems once set free from their unknowing designers. Hence the benefit of keeping the focus on primary issues regarding software artifacts, development processes, and enterprise architecture.

Happy New Year
Where to look  for AAA issues (Maurizio Cattelan)

Software Artifacts

As for any engineering activity, software engineering must be clear about its end products as well as intermediate ones. If  methodologies and technologies are set aside, three types of artifacts should be considered:

  • Requirements provide the necessary starting point. How they are expressed will essentially depend on the languages familiar to users.
  • Models are introduced as intermediate products when the continuity of processes cannot be achieved and some communication is necessary.
  • Software components are the end products.

Model based engineering approaches provide a comprehensive and consistent framework for the analysis of engineering problems and the choice of development solutions.

Enterprise Architecture

Software artifacts are the building blocks of information systems whose aims is to support enterprise objectives. Hence the need of a shared framework for enterprise (business), systems (applications), and platforms (technology) concerns. As attempts to force all problems and solutions into a single formalism have been inconclusive (to say the least), on should instead maintain the separation of concerns and build a bridge between those that must be kept aligned.

Agile Processes

While enterprise architectures are meant to deal with stable and shared assets and mechanisms, business processes are supposed to deal with changes and to take advantage of opportunities. Hence the conundrum when the latter are to be supported by the former; that’s the challenge taken on by agile approaches. While that may be an easy task for low-hanging fruits of well circumscribed projects, innovative solutions are needed if agile principles are to be “scaled” up to architecture level.

Further Reading

External Links

MDA & EA: Is The Tail Wagging The Dog ?

Making Heads or Tails

OMG’s Model Driven Architecture (MDA) is a systems engineering framework set along three model layers:

  • Computation Independent Models (CIMs) describe business objects and activities independently of supporting systems.
  • Platform Independent Models (PIMs) describe systems functionalities independently of platforms technologies.
  • Platform Specific Models (PSMs) describe systems components as implemented by specific technologies.

Since those layers can be mapped respectively to enterprise, functional, and technical architectures, the question is how to make heads or tails of the driving: should architectures be set along model layers or should models organized according architecture levels.

(judy Kensley McKie)
A Dog Making Head or Tail (Judy Kensley McKie)

In other words, has some typo reversed the original “architecture driven modeling” (ADM) into “model driven architecture” (MDA) ?

Wrong Spelling, Right Concepts

A confusing spelling should not mask the soundness and relevance of the approach: MDA model layers effectively correspond to a clear hierarchy of problems and solutions:

  • Computation Independent Models describe how business processes support enterprise objectives.
  • Platform Independent Models describe how systems functionalities support business processes.
  • Platform Specific Models describe how platforms implement systems functionalities.
MDA layers correspond to a clear hierarchy of problems and solutions
MDA layers correspond to a clear hierarchy of problems and solutions

That should leave no room for ambiguity: regardless of the misleading “MDA” moniker,  the modeling of systems is meant to be driven by enterprise concerns and therefore to follow architecture divides.

Architectures & Assets Reuse

As it happens, the “MDA” term is doubly confusing as it also blurs the distinction between architectures and processes. And that’s unfortunate because the reuse of architectural assets by development processes is at the core of the MDA framework:

  • Business objects and logic (CIM) are defined independently of the functional architectures (PIM) supporting them.
  • Functional architectures (PIM) are defined independently of implementation platforms (PSM).
  • Technical architecture (PSM) are defined independently of deployment configurations.
MDA layers clearly coincide with reusable assets
MDA layers coincide with categories of reusable assets

Under that perspective the benefits of the “architecture driven” understanding (as opposed to the “model driven” one) appear clearly for both aspects of enterprise governance:

  • Systems governance can be explicitly and transparently aligned on enterprise organization and business objectives.
  • Business and development processes can be defined, assessed, and optimized with regard to the reuse of architectural assets.

With the relationship between architectures and processes straightened out and architecture reinstated as the primary factor, it’s possible to reexamine the contents of models used as hinges between them.

Languages & Model Purposes

While engineering is not driven by models but by architectures, models do describe architectures. And since models are built with languages, one should expect different options depending on the nature of artifacts being described. Broadly speaking, three basic options can be considered:

  • Versatile and general modeling languages like UML can be tailored to different contexts and purposes, along development cycle (requirements, analysis, design) as well as across perspectives (objects, activities, etc) and domains (banking, avionics, etc)
  • Non specific business modeling languages like BPM and rules-based languages are meant to be introduced upfront, even if their outcome can be used further down the development cycle.
  • Domain specific languages, possibly built with UML, are also meant to be introduced early as to capture domains complexity. Yet, and contrary to languages like BPM, their purpose is to provide an integrated solution covering the whole development cycle.
Languages: general purpose (blue), process or domain specific (green), or design.
Languages: general purpose (blue), process or domain specific (green), or design (brown).

As seen above for reuse and enterprise architecture, a revised MDA perspective clarifies the purpose of models and consequently the language options. With developments “driven by models”, code generation is the default option and nothing much is said about what should be shared and reused, and why. But with model contents aligned on architecture levels, purposes become explicit and modeling languages have to be selected accordingly, e.g:

  • Domain specific languages for integrated developments (PSM-centered).
  • BPM for business specifications to be implemented by software packages (CIM-centered).
  • UML for projects set across system functional architecture (PIM-centered).

The revised perspective and reasoned association between languages and architectures can then be used to choose development processes: projects that can be neatly fitted into single boxes can be carried out along a continuous course of action,  others will require phased development models.

Enterprise Architecture & Engineering Processes

Systems engineering has to meet different kinds of requirements: business goals, system functionalities, quality of service, and platform implementations. In a perfect (model driven engineering) world there would be one stakeholder, one architecture, and one time-frame. Unfortunately, requirements are usually set by different stakeholders, governed by different rationales, and subject to changes along different time-frames. Hence the importance of setting forth the primary factors governing engineering processes:

  • Planning: architecture levels (business, systems, platforms) are governed by different time-frames and engineering projects must be orchestrated accordingly.
  • Communication: collaboration across organizational units require traceability and transparency.
  • Governance: decisions across architecture levels and business units cannot be made upfront and options and policies must be assessed continuously.

Those objectives are best supported when engineering processes are set along architecture levels:

Enterprise Architecture & Processes
Enterprise Architecture & Processes
  1. Requirements: at enterprise level requirements deal with organization and business processes (CIMs). The enterprise requirements process starts with portfolio management, is carried on with systems functionalities, and completed with platforms operational requirements.
  2. Problems Analysis: at enterprise level analysis deals with symbolic representations of enterprise environment, objectives, and activities (PIMs). The enterprise analysis process starts with the consolidation of symbolic representations for objects (power-types) and activities (scenarii), is carried on with functional architectures, and completed with platforms non-functional features. Contrary to requirements, which are meant to convey changes and bear adaptation (dashed lines), the aim of analysis at enterprise level is to consolidate symbolic representations and guarantee their consistency and continuity. As a corollary, analysis at system level must be aligned with its enterprise counterpart before functional (continuous lines) requirements are taken into account.
  3. Solutions Design: at enterprise level design deals with operational concerns and resources deployment. The enterprise design process starts with locations and resources, is carried on with systems configurations, and completed with platforms deployments. Part of it is to be supported by systems as designed (PSMs) and implemented as platforms. Yet, as figured by dashed arrows, operational solutions designed at enterprise level bear upon the design of systems architectures and the configuration of their implementation as platforms.

When engineering is driven by architectures, processes can be devised depending on enterprise concerns and engineering contexts. While that could come with various terminologies, the partitioning principles will remain unchanged, e.g:

  • Agile processes will combine requirements with development and bypass analysis phases (a).
  • Projects meant to be implemented by Commercial-Off-The-Shelf Software (COTS) will start with business requirements, possibly using BPM, then carry on directly to platform implementation, bypassing system analysis and design phases (b).
  • Changes in enterprise architecture capabilities will be rooted in analysis of enterprise objectives, possibly but not necessarily with inputs from business and operational requirements, continue with analysis and design of systems functionalities, and implement the corresponding resources at platform level (c).
  • Projects dealing with operational concerns will be conducted directly through systems design of and platform implementation (d).
Processes should be devised according enterprise concerns and engineering contexts
Examples of process templates depending on objectives and contexts.

To conclude, when architecture is reinstated as the primary factor, the MDA paradigm becomes a pivotal component of enterprise architecture as it provides a clear understanding of architecture divides and dependencies on one hand, and their relationship with engineering processes on the second hand.

Postscript

MDA illustrates a contrario the straying agenda of the OMG: despite its soundness as a foundation of enterprise architecture as well as its complementarity with the Unified Modeling Language (UML), MDA three-tiers framework has been left unattended, to be replaced by an open-ended (more than two hundred, and counting) hotpotch of amorphous and overlapping meta-models.

Nonetheless, the paradigm is not to be forsaken and can be found behind the Zachman framework and its revamping by Caminao.

https://caminao.blog/wp-content/uploads/2019/01/symbotransfo_pagoda.jpg?w=600

MDA reincarnation as the Pagoda Blueprint

Further Reading

External Links