Depending on context and purpose requirements can be understood as customary documents, contents, or work in progress.
Given that requirements are the entry point of engineering processes, nothing should be assumed regarding their form (natural, formal, or specific language), or content (business, functional, non-functional, or technical).
Depending on the language used, requirements can be directly analyzed and engineered, or may have to be first formatted (aka captured).
Since requirements reflect biased and partial agendas, taxonomy, in particular the distinction between functional and non functional concerns, is in the eyes of the stakeholders.
Depending on content and context, requirements can be engineered as a whole (agile development models), or set apart as to tally with external dependencies (phased development models).
Agile and phased development solutions are meant to solve different problems and therefore differ with artifacts and activities; that can be illustrated by requirements, understood as words in progress for the former, etched statements for the latter.
Ignoring that distinction is to make stories stutter from hiccupped iterations, or phases sputter along ripped milestones.
Agile & Phased Tell Different Stories Differently
As illustrated by ill-famed waterfall, assuming that requirements can be fully set upfront often put projects at the hazards of premature commitments; conversely, giving free rein to expectations could put requirements on collision courses.
That apparent dilemma can generally be worked out by setting apart business outlines from users’ stories, the latter to be scripted and coded on the fly (agile), the former analysed and documented as a basis for further developments (phased). To that end project managers must avoid a double slip:
Mission creep: happens when users’ stories are mixed with business models.
Jump to conclusions: happens when enterprise business cases prevail over the specifics of users’ concerns.
Interestingly, the distinction between purposes (users concerns vs business functions) can be set along one between language semantics (natural vs modeling).
Semantics: Capture vs Analysis
Beyond methodological contexts (agile or phased), a clear distinction should be made between requirements capture (c) and modeling (m): contrary to the former which translates sequential specifications from natural to programming (p) languages without breaking syntactic and semantic continuity, the latter carries out a double translation for dimension (sequence vs layout) and language (natural vs modeling.)
The continuity between natural and programming languages is at the root of the agile development model, enabling users’ stories to be iteratively captured and developed into code without intermediate translations.
That’s not the case with modeling languages, because abstractions introduce a discontinuity. As a corollary, requirements analysis is to require some intermediate models in order to document translations.
The importance of discontinuity can be neatly demonstrated by the use of specialization and generalization in models: the former taking into account new features to characterize occurrences (semantic continuity), the latter consolidating the meaning of features already defined (semantic discontinuity).
As noted above, users’ stories can be continuously developed into code because a semantic continuity can be built between natural and programming languages statements. That necessary condition is not a sufficient one because users’ stories have also to stand as complete and exclusive basis for applications.
Such a complete and exclusive mapping to application is de-facto guaranteed by continuous and incremental development, independently of the business value of stories. Not so with intermediate models which, given the semantic discontinuity, may create back-doors for broader concerns, e.g when some features are redefined through generalization. Hence the benefits of a clarity of purpose:
Users’ stories stand for specific requirements meant to be captured and coded by increments. Documentation should be limited to application maintenance and not confused with analysis models.
Use cases should be introduced when stories are to be consolidated or broader concerns factored out , e.g the consolidation of features or business cases.
Sorting out the specifics of users concerns while keeping them in line with business models is at the core of business analysts job description. Since that distinction is seldom directly given in requirements, it could be made easier if aligned on modeling options: stories and specialization for users concerns, models and generalization for business features.
From Stories to Cases
The generalization of digital environments entails structural and operational adjustments within enterprise architectures.
At enterprise level the integration of homogeneous digital flows and heterogeneous symbolic representations can be achieved through enterprise architectures and profiled ontologies. But that undertaking is contingent on the way requirements are first dealt with, namely how the specifics of users’ needs are intertwined with business designs.
As suggested above, modeling schemes could help to distinguish as well as consolidate users narratives and business outlooks, capturing the former with users’ stories and the latter with use cases models.
That would neatly align means (part played by supporting systems) with ends (users’ stories vs business cases):
Users’ stories describe specific objectives independently of the part played by supporting systems.
Use cases describe the part played by systems taking into account all supported stories.
It must be stressed that this correspondence is not a coincidence: the consolidation of users’ stories into broader business objectives becomes a necessity when supporting systems are taken into account, which is best done with use cases.
Aligning Stories with Cases
Stories and models are orthogonal descriptions, the former being sequenced, the latter laid out; it ensues that correspondences can only be carried out for individuals uniformly identified (#) at enterprise and systems level, specifically: roles (aka actors), events, business objects, and execution units.
It must be noted that this principle is supposed to apply independently of the architectures or methodologies considered.
With continuity and consistency of identities achieved at architecture level, the semantic discontinuity between users’ stories and models (classes or use cases) can be managed providing a clear distinction is maintained between:
Modeling abstractions, introduced by requirements analysis and applied to artifacts identified at architecture level.
The semantics of attributes and operations, defined by users’ stories and directly mapped to classes or use cases features.
Finally, stories and cases need to be anchored to epics and enterprise architecture.
Business Cases & Enterprise Stories
Likening epics to enterprise stories would neatly frame the panoply of solutions:
At process level users’ stories and use cases would be focused respectively on specific business concerns and supporting applications.
At architecture level epics and business cases would deal respectively with business models and objectives, and supporting systems capabilities.
That would provide a simple yet principled basis for enterprise architectures governance.
Digital environments and the ubiquity of software in business processes introduces a new perspective on value chains and the assessment of supporting applications.
At the same time, as software designs cannot be detached of architectures capabilities, the central question remains of allocating costs and benefits between primary and support activities .
Value Chains & Activities
The concept of value chain introduced by Porter in 1985 is meant to encompass the set of activities contributing to the delivery of a valuable product or service for the market.
Taking from Porter’s generic model, various value chains have been refined according to business specific categories for primary and support activities.
Whatever their merits, these approaches are essentially static and fall short when the objective is to trace changes induced by business developments; and that flaw may become critical with the generalization of digital business environments:
Given the role and ubiquity of software components (not to mention smart ones), predefined categories are of little use for impact analysis.
When changes in value chains are considered, the shift of corporate governance towards enterprise architecture puts the focus on assets contribution, cutting down the relevance of activities.
Hence the need of taking into account changes, software development, and enterprise architectures capabilities.
Value Added & Software Development
While the growing interest for value chains in software engineering is bound to agile approaches and business driven developments, the issue can be put in the broader perspective of project planning.
With regard to assessment,stakeholders, start with business opportunities and look at supporting systems from a black box perspective; in return, software providers are to analyze requirements from a white box perspective, and estimate corresponding development effort and time delivery.
Assuming transparency and good faith, both parties are meant to eventually align expectations and commitments with regard to features, prices, and delivery.
With regard to policies, stakeholders put the focus on returns on investment (ROI), obtained from total cost of ownership, quality of service, and timely delivery. Providers for their part try to minimize development costs while taking into account effective use of resources and costs of opportunities. As it happens, those objectives may be carried on as non-zero sum games:
Business stakeholders foretell the actualized returns (a) to be expected from the functionalities under consideration (b).
Providers consider the solutions (b’) and estimate actualized costs (a’).
Stakeholders and providers agree on functionalities, prices and deliveries (c).
Assuming that business and engineering environments are set within different time-frames, there should be room for non-zero-sum games winding up to win-win adjustments on features, delivery, and prices.
Continuous vs Phased Alignments
Notwithstanding the constraints of strategic planning, business processes are by nature opportunistic, and their ability to be adjusted to circumstances is becoming all the more critical with the generalization of digital business environments.
Broadly speaking, the squaring of supporting applications to business value can be done continuously or by phases:
Phased alignments start with some written agreements with regard to features, delivery, and prices before proceeding with development phases.
Continuous alignment relies on direct collaboration and iterative development to shape applications according to business needs.
Beyond sectarian controversies, each approach has its use:
Continuous schemes are clearly better at harnessing value chains, providing that project teams be allowed full project ownership, with decision-making freed of external dependencies or delivery constraints.
Phased schemes are necessary when value chains cannot be uniquely sourced as they take roots in different organizational units, or if deliveries are contingent on technical constraints.
In any case, it’s not a black-and-white alternative as work units and projects’ granularity can be aligned with differentiated expectations and commitments.
Work Units & Architecture Capabilities
While continuous and phased approaches are often opposed under the guises of Agile vs Waterfall, that understanding is misguided as it extends the former to a motley of self-appointed agile schemes and reduces the latter to an ill-famed archetype.
Instead, a reasoned selection of a development models should be contingent on the problems at hand, and that can be best achieved by defining work-units bottom-up with regard to the capabilities targeted by requirements:
Development patterns could then be defined with regard to architecture layers (organization and business, systems functionalities, platforms implementations) and capabilities footprint:
Phased: work units are aligned with architecture capabilities, e.g : business objects (a), business logic (b), business processes (c), users interfaces (d).
Iterative: work units are set across capabilities and defined dynamically according to development problems.
That would provide a development framework supporting the assessment of iterative as well as phased projects, paving the way for comprehensive and integrated impact analysis.
Value Chains & Architecture Capabilities
As far as software engineering is concerned, the issue is less the value chain itself than its change, namely how value is to be added along the chain.
To summarize, the transition to this layout is carried out in two steps:
Conceptually, Zachman’s original “Why” column is translated into a line running across column capabilities.
Graphically, the five remaining columns are replaced by embedded pentagons, one for each architecture layer, with the new “Why” line set as an outer layer linking business value to architectures capabilities:
That apparently humdrum transformation entails a significant shift in focus and practicality:
The focus is put on organizational and business objectives, masking the ones associated to systems and platforms layers.
It makes room for differentiated granularity in the analysis of value, some items being anchored to specific capabilities, others involving cross dependencies.
Value chains can then be charted from business processes to supporting architectures, with software applications in between.
As pointed above, the crumbling of traditional fences and the integration of enterprise architectures into digital environments undermine the traditional distinction between primary and support activities.
To be sure, business drive is more than ever the defining factor for primary activities; and computing more than ever the archetype of supporting ones. But in between the once clear-cut distinctions are being blurred by a maze of digital exchanges.
In order to avoid a spaghetti heap of undistinguished connections, value chains are to be “colored” according to the nature of links:
Between architectures capabilities: business and organization (enterprise), systems functionalities, or platforms and technologies.
Between architecture layers: engineering processes.
When set within that framework, value chains could be navigated in both directions:
For the assessment of applications developed iteratively: business value could be compared to development costs and architecture assets’ depreciation.
For the assessment of features (functional or non functional) to be shared across business applications: value chains will provide a principled basis for standard accounting schemes.
Combined with model based system engineering that could significantly enhance the integration of enterprise architecture into corporate governance.
Computation independent models (CIMs) describe organization and business processes independently of the role played by supporting systems.
Platform independent models (PIMs) describe the functionalities supported by systems independently of their implementation.
Platform specific models (PSMs) describe systems components depending on platforms and technologies.
Engineering processes can then be phased along architecture layers (a), or carried out iteratively for each application (b).
When set across activities value chains could be engraved in CIMs and refined with PIMs and PSMs(a). Otherwise, i.e with business value neatly rooted in single business units, value chains could remain implicit along software development (b).
“For things to remain the same, everything must change”
Lampedusa, “The Leopard”
Whatever the understanding of the discipline, most EA schemes implicitly assume that enterprise architectures, like their physical cousins, can be built from blueprints. But they are not because enterprises have no “Pause” and “Reset” buttons: business cannot be put on stand-by and must be carried on while work is in progress.
Systems & Enterprises
Systems are variously defined as:
“A regularly interacting or interdependent group of items forming a unified whole” (Merriam-Webster).
“A set of connected things or devices that operate together” (Cambridge Dictionary).
“A way of working, organizing, or doing something which follows a fixed plan or set of rules” (Collins Dictionary)
“A collection of components organized to accomplish a specific function or set of functions” (TOGAF from ISO/IEC 42010:2007)
While differing in focus, most understandings mention items and rules, purpose, and the ability to interact; none explicitly mention social structures or interactions with humans. That suggests where the line should be drawn between systems and enterprises, and consequently between corresponding architectures.
Architectures & Changes
Enterprises are live social entities made of corporate culture, organization, and supporting systems; their ultimate purpose is to maintain their identity and integrity while interacting with environments. As a corollary, changes cannot be carried out as if architectures were just apparel, but must ensure the continuity and consistency of enterprises’ structures and behaviors.
That cannot be achieved by off-soil schemes made of blueprints and step-by-step processes detached from actual organization, systems, and processes. Instead, enterprise architectures must be grown bottom up from actual legacies whatever their nature: technical, functional, organizational, business, or cultural.
Insofar as enterprise architectures are concerned, legacies are usually taken into account through one of three implicit assumptions:
No legacy assumptions ignore the issue, as if the case of start-ups could be generalized. These assumptions are logically flawed because enterprises without legacy are like embryos growing their own inherent architecture, and in that case there would be no need for architects.
En Bloc legacy assumptions take for granted that architectures as a whole could be replaced through some Big Bang operation without having a significant impact on business activities. These assumptions are empirically deceptive because, even limited to software architectures, Big Bang solutions cannot cope with the functional and generational heterogeneity of software components characterizing large organizations. Not to mention that enterprise architectures are much more that software and IT.
Piecemeal legacies can be seen as the default assumption, based on the belief that architectures can be re-factored or modernized step by step. While that assumption may be empirically valid, it may also miss the point: assuming that all legacies can be dealt with piecemeal rubs out the distinction pointed above between systems and enterprises.
So, the question remains of what is to be changed, and how ?
EA as a Work In Progress
As with leopard’s spots and identity, the first step would be to set apart what is to change (architectures) from what is to carry on (enterprise).
Maps and territories do provide an overview of spots’ arrangement, but they are static views of architectures, whereas enterprises are dynamic entities that rely on architectures to interact with their environment. So, for maps and territories to serve that purpose they should enable continuous updates and adjustments without impairing enterprises’ awareness and ability to compete.
That shift from system architecture to enterprise behavior implies that:
The scope of changes cannot be fully defined up-front, if only because the whole enterprise, including its organization and business model, could possibly be of concern.
Fixed schedules are to be avoided, lest each and every unit, business or otherwise, would have to be shackled into a web of hopeless reciprocal commitments.
Different stakeholders may come as interested parties, some more equal than others, possibly with overlapped prerogatives.
So, instead of procedural and phased approaches supposed to start from blank pages, EA ventures must be carried out iteratively with the planning, monitoring, assessment, and adjustment of changes across enterprises’ businesses, organizations, and systems. That can be represented as an extension of the OODA (Observation, Orientation, Decision, Action) loop:
Actual observations from operations (a)
Data analysis with regard to architectures as currently documented (b).
Changes in business processes (c).
Changes in architectures (d).
Moreover, due to the generalization of digital flows between enterprises and their environment, decision-making processes used to be set along separate time-frames (operational, tactical, strategic, …), must now be weaved together along a common time-scale encompassing internal (symbolic) as well as external (actual) events.
It ensues that EA processes must not only be continuous, but they also must deal with latency constraints.
Changes & Latency
Architectures are by nature shared across organizational units (enterprise level) and business processes (system level). As a corollary, architecture changes are bound to introduce mismatches and frictions across business-specific applications. Hence the need of sorting out the factors affecting the alignment of maps and territories:
Elapsed time between changes in territories and maps updates (a>b) depends on data analytics and operational architecture.
Elapsed time between changes in maps and revised objectives (b>c) depends on business analysis and organization.
Elapsed time between changes in objectives and their implementation (c>d) depends on engineering processes and systems architecture.
Elapsed time between changes in systems and changes in territories (d>a) depends on applications deployment and technical architectures.
On that basis it’s possible to define four critical lags:
Operational: data analytics can be impeded by delayed, partial, or inaccurate feedback from processes.
Mapping: business analysis can be impeded by delays or discrepancies in data analytics.
Engineering: development of applications can be impeded by delays or discrepancies in business analysis.
Processes: deployment of business processes can be impeded by delays in the delivery of supporting applications.
These lags condition the whole of EA undertakings because legacy structures, mechanisms, and organizations are to be continuously morphed into architectures without introducing misrepresentations that would shackle activities and stray decision-making.
EA Latency & Augmented Reality
Insofar as architectural changes are concerned, discrepancies and frictions are rooted in latency, i.e the elapsed time between actual changes in territories and the updating of relevant maps.
As noted above, these lags have to be weighted according to time-frames, from operational days to strategic years, so that the different agents could be presented with the relevant and up-to-date views befitting to each context and concerns.
That could be achieved if enterprises architectures were presented through augmented reality technologies.
Compared to virtual reality (VR) which overlooks the whole issue of reality and operates only on similes and avatars, augmented reality (AR) brings together virtual and physical realms, operating on apparatuses that weaves actual substrates, observations, and interventions with made-up descriptive, predictive, or prescriptive layers.
On that basis, users would be presented with actual territories (EA legacy) augmented with maps and prospective territories.
Composition and dynamics of maps and territories (actual and prospective) could be set and edited appropriately, subject to latency constraints.
Requirements is what to feed engineering processes. As such they are to be presented under a wide range of forms, and nothing should be assumed upfront about forms or semantics.
Answering the question of reuse therefore depends on what is to be reused, and for what purpose.
Documentation vs Reuse
Until some analysis can be carried out, requirements are best seen as documents; whether such documents are to be ephemeral or managed would be decided depending on method (agile or phased), contents (business, supporting systems, implementation, or quality of services), or purpose (e.g governance, regulations, etc).
Setting apart external conditions, requirements documentation could be justified by:
Traceability of decision-making linking initial requests with actual implementation.
Maintenance of deliverables during their life-cycle.
Assuming that requirements have been properly formatted, e.g as analysis models (with technical ones managed internally at system level), reuse could be justified by changes in business, functional, or quality of services requirements:
Business processes are meant to change with opportunities. With requirements available as analysis models, changes would be more easily managed (a) if they could be fine-grained. Business rules are a clear example, but that could also be the case for new features added to business objects.
Functional requirements may change even without change of business ones, e.g if new channels and users are introduced addressing existing business functions. In that case reusable business requirements (b) would dispense with a repeat of business analysis.
Finally, quality of service could be affected by operational changes like localization, number of users, volumes, or frequency. Adjusting architecture capabilities would be much easier with functional (c) and business (d) requirements properly documented as analysis models.
Along that perspective, requirements reuse appears to revolve around two pivots, documents and analysis models. Ontologies could be used to bind them.
Requirements & Ontologies
Reusing artifacts means using them in contexts or for purposes different of native ones. That may come by design, when specifications can anticipate on shared concerns, or as an afterthought, when initially unexpected similarities are identified later on. In any case, reuse policies have to overcome a twofold difficulty:
Visibility: business and functional analysts must be made aware of potential reuse without having to spend too much time on research.
Overheads: ensuring transparency, traceability, and consistency checks on requirements (documents or analysis models) cannot be achieved without costs.
Ontologies could help to achieve greater visibility with acceptable overheads by framing requirements with regard to nature (documents or models) and context:
With regard to nature, the critical distinction is between document management and model based engineering systems. When framed as ontologies, the former is to be implemented as thesaurus targeting terms and documents, the latter as ontologies targeting categories specific to organizations and business domains.
With regard to context the objective should be to manage reusable requirements depending on the kind of jurisdiction and stability of categories, e.g:
Institutional: Regulatory authority, steady, changes subject to established procedures.
Professional: Agreed upon between parties, steady, changes subject to accord.
Corporate: Defined by enterprises, changes subject to internal decision-making.
Social: Defined by usage, volatile, continuous and informal changes.
Personal: Customary, defined by named individuals (e.g research paper).
Combined with artificial intelligence, ontology archetypes could crucially extend the benefits of requirements reuse, notably through the impact of deep learning for visibility.
On a broader perspective requirements should be seen as a source of knowledge, and their reuse managed accordingly.
Repeated announces of looming software apocalypse may take some edge off vigilance, but repeated systems failures should be taken seriously, if only because they appear to be rooted in a wide array of causes, from wrongly valued single parameters (e.g 911 threshold or Apple’s free pass for “root” users) to architecture obsolescence (e.g reservation systems.)
Yet, if alarms are not to be ignored, prognoses should go beyond syndromes and remedies beyond sticking plaster: contrary to what is suggested by The Atlantic’s article, systems are much more than piles of code, and programming is probably where quality has been taken the most seriously.
Programs vs Systems
Whatever programmers’ creativity and expertise, they cannot tackle complexity across space, time, and languages: today’s systems are made of distributed interacting components, specified and coded in different languages, and deployed and modified across overlapping time-frames. Even without taking into account continuous improvements in quality, apocalypse is not to loom in the particulars of code but on their ways in the world.
Solutions should therefore be looked for at system level, and that conclusion can only be bolstered by the ubiquity of digitized business flows.
Systems are the New Babel
As illustrated by the windfalls benefiting Cobol old timers, language is arguably a critical factor, for the maintenance of legacy programs as well as for communication between stakeholders, users, and engineers.
So if problems can be traced back to languages, it’s where solutions are to be found: from programming languages (for code) to natural ones (for systems requirements), everything can be specified as symbolic representations, i.e models.
Model in the Loop
Models are generally understood as abstractions, and often avoided for that very reason. That shortsighted mind-set is made up for by concrete employs of abstractions, as illustrated by the Automotive industry and the way it embeds models in engineering processes.
Summarily, the Automotive’s Model in Loop (MiL) can be explained through three basic ideas:
Systems are to be understood as the combination of physical and software artifacts.
Insofar as both can be implemented as digits, they can be uniformly described as models.
As a consequence, analysis, design and engineering can be carried out through the iterative building, simulating, testing, and adjusting various combinations of hardware and software.
By bringing together physical components and code into a seamless digitized whole, MiL brings down the conceptual gap between actual elements and symbolic representations, aka models. And that leap could be generalized to a much wider range of systems.
Models are the New Code
Programming habits and the constraints imposed by the maintenance of legacy systems have perpetuated the traditional understanding of systems as a building-up of programs; hence the focus put on the quality of code. But when large, distributed, and perennial systems are concerned, that bottom-up mind-set falls short and brings about:
An exponential increase of complexity at system level.
Opacity and discontinued traceability at application level between current use and legacy code.
Both flaws could be corrected by combining top-down modeling and bottom-up engineering. That could be achieved with iterative processes carried out from both directions.
Model in the Loop meets Enterprise Architecture
From a formal perspective models are of two sorts: extensional ones operate bottom-up and associate sets of individuals with categories, intensional ones operate top-down and specify the features meant to be shared by all instances of a type. based on that understanding, the former can be used to simulate the behaviors of targeted individuals depending on categories, and the latter to prescribe how to create instances of types meant to implement categories.
As it happens, Model-in-loop combines the two schemes at component level:
Any combination of manual and automated solution can be used as a starting point for analysis and simulation (a).
Given the outcomes of simulation and tests, the architecture is revisited (b) and corresponding artifacts (software and hardware) are designed (c).
The new combination of artifacts are developed and integrated, ready for analysis and simulation (d).
Assuming that MiL bottom-up approach could be tallied with top-down systems engineering processes, it would enable a seamless and continuous integration of changes in software components and systems architectures.