Views can take different meanings, from windows opening on specific data contexts (e.g DB relational theory), to assortments of diagrams dedicated to particular concerns (e.g UML).
Deconstructing the Universe along Contexts and Concerns (Depero Fortunato)
Models for their part have also been understood as views, on DB contents as well as systems’ architecture and components, the difference being on the focus put on engineering. Due to their association with phased processes, models has been relegated to a back-burner by agile approaches; yet it may resurface in terms of granularity with model-based engineering frameworks.
Yet, whatever the terminology (layers vs levels), what is at stake is the alignment of two basic scales:
Architectures: enterprise (concepts), systems (functionalities), and platforms (technologies).
Process view: captures the concurrency and synchronization aspects.
Physical view: describes the mapping(s) of software artifacts onto hardware.
Development view: describes the static organization of software artifacts in development environments.
A fifth is added for use cases describing the interactions between systems and business environments.
Whereas these views have been originally defined with regard to UML diagrams, they may stand on their own meanings and merits, and be assessed or amended as such.
Apart from labeling differences, there isn’t much to argue about use cases (for requirements), process (for operations), and physical (for deployment) views; each can be directly associated to well identified parts of systems engineering that are to be carried out independently of organizations, architectures or methods.
Logical and development views raise more questions because they imply a distinction between design and implementation. That implicit assumption induces two kinds of limitations:
They introduce a strong bias toward phased approaches, in contrast to agile development models that combine requirements, development and acceptance into iterations.
They classify development processes with regard to predefined activities, overlooking a more critical taxonomy based on objectives, architectures and life-cycles: user driven and short-term (applications ) vs data-based and long-term (business functions).
These flaws can be corrected if logical and development views are redefined respectively as functional and application views, the former targeting business objects and functions, the latter business logic and users’ interfaces.
Architecture based views
That make views congruent with architecture levels and consequently with engineering workshops. More importantly, since workshops make possible the alignment of products with work units, they are a much better fit to model-based engineering and a shift from procedural to declarative paradigm.
Model-based Systems Engineering & Granularity
At least in theory, model-based systems engineering (MBSE) should free developers from one-fits-all procedural schemes and support iterative as well as declarative approaches. In practice that would require matching tasks with outcomes, which could be done if responsibilities on the former can be aligned with models granularity of the latter.
With coarse-grained phased schemes like MDA’s CIM/PIM/PSM (a), dependencies between tasks would have to be managed with regard to a significantly finer artifacts’ granularity.
Managing changes at architecture (a) or application (b) level.
For agile schemes, assuming conditions on shared ownership and continuous deliveries are met, projects would put locks on “models” at both ends (users’ stories and deliveries) of development cycles (b), with backlogs items defining engineering granularity.
Backlogs mechanism can be used to manage customized granularity and hierarchical dependencies across model layers
From the enterprise perspective it would be possible to unify the management of changes in architectures across layers and responsibilities: business concepts and organization, functional architecture, and systems capabilities:
Functional architecture as symbolic bridge between business needs and system capabilities.
From the engineering perspective it would be possible to unify the management of changes in artifacts at the appropriate level of granularity: static and explicit using milestones (phased), dynamic and implicit using backlogs (agile).
Fine grained model based frameworks could support phased as well as agile development solutions
Such a declarative repository would greatly enhance exchanges and integration across projects and help to align heterogeneous processes independently of the methodologies used.
Uses cases are meant to describe how users interact with systems, classes are meant to describe software components, including those executing use cases. It ensues that classes are introduced with the realization of use cases but are not supposed to appear as such in their definition.
Users are not supposed to know about surrogates
The Case for Use Cases
Use cases (UCs) are the brain child of Ivar Jacobson and often considered as the main innovation introduced by UML. Their success, which largely outdoes UML’s footprint, can be explained by their focus and simplicity:
Focus: UCs are meant to describe what happens between users and systems. As such they are neatly bounded with regard to their purpose (UCs are the detailed parts of business processes supported by systems) and realization (UCs are implemented by software applications).
Simplicity: while UCs may eventually include formal (e.g pre- and post-conditions) and graphical (e.g activity diagrams) specifications, they can be fully defined and neatly circumscribed using stick actors (for the roles played by users or any other system) and ellipses (for system behaviors).
Use Cases & UML diagrams
As it often happens to successful innovations, use cases have been widely interpreted and extended; nonetheless, the original concepts introduced by Ivar Jacobson remain basically unaltered.
The Point of Use Cases
Whereas focus and simplicity are clearly helpful, the primary success factor is that UCs have a point, namely they provide a conceptual bridge between business and system perspectives. That appears clearly when UCs are compared to main alternatives like users’ stories or functional requirements:
Users’ stories are set from business perspective and lack explicit constructs for the parts supported by systems. As a consequence they may flounder to identify and describe business functions meant to be shared across business processes.
Conversely, functional requirements are set from system perspective and have no built-in constructs linking business contexts and concerns to their system counterparts. As a consequence they may fall short if business requirements cannot be set upfront or are meant to change with business opportunities.
Along that understanding, nothing should be done to UCs that could compromise their mediating role between business value and system capabilities, the former driven by changes in business environment and enterprise ability to seize opportunities, the latter by the continuity of operations and the effective use of technical or informational assets.
Business Objects vs Software Components
Users’ requirements are driven by concrete, partial, and specific business expectations, and it’s up to architects to weld those diverse and changing views into the consistent and stable functional abstractions that will be implemented by software components.
Users’ requirements are driven by concrete, partial, changing and specific concerns, but supported by stable and fully designed software abstractions.
Given that double discrepancy of objectives and time-scales, business analysts should not try to align their requirements with software designs, and system analysts should not try to second-guess their business counterparts with regard to future business objects. As a consequence, respective outcomes would be best achieved through a clear separation of concerns:
Use cases deal with the business value of applications, mapping views on business objects to aspects of classes.
Functional architectures deal with assets, in particular the continuous and consistent representation of business objects by software components as described by classes.
How to get best value from assets
As it happens, that double classification with regard to scope and purpose should also be used to choose a development model: agile when scope and purpose can be united, phased approach otherwise.
Depending on devotees or dissenters, the Agile development model is all too often presented as dead-end or end-of-story. Some of that unfortunate situation can be explained, and hopefully pacified, by comparing users’ stories to plants, with their roots, trunks, and branches. Assuming that agility calls for sound footings and good springboards, it may be argued that many problems arise with stories barking at the wrong tree (application level) or getting lost in the woods (architecture level).
How to Mind/Mend a True/Tree Story
Application level: Trees, Bushes and Hedges
As Aristotle first stated, good stories have to follow the three unities: one course of action, located in a single space, run along continuous time.
That rule is clearly satisfied by stories that can be developed like plants growing from clearly identified roots.
Yet, stories like bushes may grow too many offshoots to be accounted for by a single action narrative; in that case it may be possible to single out a primary trunk and a set of forking branches along which different scenarii could be developed.
More serious difficulties may appear with thickets mixing offshoots from different bushes sharing the same space. That situation will first require some ground work in order to single out individual roots, and then use them to extricate each bush separately. When, like offshoots that actually mingle, story-lines cross and share actions, the description of such actions (aka features) is to be factored out and separated from the contexts of their enactment in the different story-lines.
Finally, like bushes in hedges, stories may chronicle repeated activities serving some collective purpose. That configuration is both easy to recognize and dealt with effectively by introducing a stereotyped story feature for collections and loops management.
Architecture level: Groves, Woods and Plantations
Contrary to hedges which are built on the similarity of their constituents, groves are based on their functional differences, and that can also be seen as a critical distinction between containers and architectures.
In agile parlance, that is best compared to the difference between stories and epics, the former telling what happens between users and applications, the latter taking a bird’s view of the relationships between business processes and systems.
In most of the cases the question will arise for sizable stories deemed too large for development purposes. When dealing with that situation the first step should be to look for thickets and bushes, respectively to be set apart as individual bushes or refined as scenarii. When still confronted with multiple roots, the question would be to decide between hedges and groves, that is between repeated activities and collaboration. And that decision would be critical because collaborations call for a different kind of story (aka epics or themes) set at a higher level, namely architecture.
Scaling Ups and Downs
Assuming the three-units rule cannot be met, two alternative approaches are possible, depending on whether the story has to be broken down or upgraded to an epic, and the undoing of the rule can be used to make a decision:
When the course of actions, once started, is to be contingent on subsequent business (aka external) events the story should be upgraded to an epic, as it will often refer to a part or whole of a business process.
Otherwise: when activities are set along different periods of time (i.e contingent on time-events) the story can be broken down depending on size, functional architecture, or development constraints.
Otherwise: when activities are distributed across locations it may be necessary to factor out architecture-dependent features dealing with shared address spaces and synchronization mechanisms
Applying those guidelines to stories will put the whole development processes on rails and help to align requirements with their architectural footprint: business logic, system functionalities, or platform technologies.
“Since words are only names for things, it would be more convenient for all men to carry about them such things as were necessary to express a particular business they are to discourse on.”
Jonathan Swift, Gulliver’s Travels
Objective
Modeling languages are meant to support the description of contexts and concerns from specific standpoints. And because those different perspectives have to be mapped against shared references, language must also provide constructs for ironing out variants and factoring out constants.
Ironing out differences in features and behaviors (Erwitt Elliott)
Yet, while most languages generally agree on basic definitions of objects and behaviors, many distinctions are ignored or subject to controversial understanding; such shortcomings might be critical when architecture capabilities are concerned:
Actual entities and their symbolic counterpart.
Actual entities and their roles.
Business logic and business operations
External events and system time.
Architecture Capabilities
As those distinctions set the backbone of functional architectures, languages should be assessed according their ability to express them unambiguously using the least possible set of constructs.
Business objects vs Symbolic Surrogates
As far as symbolic systems are concerned, the primary purpose of models is to distinguish between targets and representations. Hence the first assessment yardstick: modeling languages must support a clear and unambiguous distinction between objects and behaviors of concern on one hand, symbolic system surrogates on the other hand.
A primer on representation: objects of concerns and surrogates
Yet, if objects and behaviors are to be consistently and continuously managed, modeling languages must also provide common constructs mapping identities and structures of actual entities to their symbolic counterpart.
Agents vs Roles
Given that systems are built to support business processes, modeling languages should enable a clear distinction between physical entities able to interact with systems on one hand, roles as defined by enterprise organization on the other hand.
Agents and Roles
In that case the mapping of actual entities to systems representations is not about identities but functionalities: a level of indirection has to be introduced between enterprise organization and system technology because the same roles can be played,simultaneously or successively, by people, devices, or other systems.
Business logic vs Business operations
Just like actual objects are not to be confused with their symbolic description, modeling languages must make a clear distinction between business logic and processes execution, the former defining how to process symbolic objects and flows, the latter with the coupling between process execution and changes in actual contexts. That distinction is of a particular importance if business and organizational decisions are to be made independently of supporting systems technology.
Business logic and operational contingencies
Language constructs must also support the consolidation of functional and operational units, the former being defined by integrity constraints on symbolic flows and roles authorizations on objects and operations, the latter taking into account synchronization constraints between the state of actual contexts and the state of their system symbolic counterpart. And for that purpose languages must support the distinction between external and internal events.
External vs Internal Events
Paraphrasing Albert Einstein, time is what happens between events. As a corollary, external time is what happens between context events, and internal time is what happens between system ones. While both can coincide for single systems and locations, no such assumption should be made for distributed systems set in different locations. In that case modeling language should support a clear distinction between external events signalling changes set in actual locations, and internal events signalling changes affecting system surrogates.
Time scales are set by events and concerns
Along that perspective synchronization is to be achieved through the consolidation of time scales. For single locations that can be done using system clocks, across distributed locations the consolidation will entail dedicated time frames and mechanisms set in reference to some initial external event.
Conclusion: How to share differences across perspectives
Somewhat paradoxically, multiple modeling languages erase differences by paring down shared descriptions to some uniform lump that can be understood by all. Conversely, agreeing on a set of distinctions that should be supported by every language could provide an antidote to the Babel syndrome.
That approach can be especially effective for the alignment of enterprise and systems architectures as the four distinctions listed above are equally meaningful in both perspectives.
Development processes start with requirements and wind up in code; in between there isn’t much of a consensus among the software engineering community about how to define the scope (spaces), how to sequence the tasks (paths), and how to time deliveries (paces). On one side of the debate phased approaches hope for fixed spaces and ordered paths but often get entangled in moving lines. On the other side of the debate agile teams try to find their space by increments but risk losing the path while still on their way.
Maze revisited: finding a path while building the space (Ibrahim El-Salahi)
This lack of agreed upon concepts and principles entrusts personal skills and best practices as primary success factors. Conversely, that could explain the rate of failures for software projects, significantly higher than for “hard” engineering ones; given the quasi absence of physical constraints, the opposite would have been expected, which would suggest some critical intrinsic flaw.
With the “benefits” of hindsight and agile assessment of waterfall flaws, the focus has been put on fixed scope and schedule, in particular with regard to requirements and quality management:
Fixed requirements set upfront: since there is an inverse relationship between the level of details and the reliability and stability of requirements, staking the whole project on requirements fully defined at such an early time is arguably a very hazardous policy.
Quality as an afterthought: given that finding defects is not very gratifying when undertaken in isolation, delegating the task will offer few guarantees if not associated with rewards commensurate to findings; moreover, quality as a detached concern may easily turn into a collateral damage when set along mounting costs and scheduling constraints. Alternatively, quality checks may change into a more positive endeavor when conducted as an intrinsic part of development.
Agile answer to those failings has been to conduct specifications, development, and quality assurance into integrated iterations. As a consequence, the definition of scope becomes a byproduct of development cycles, with requirements itemized as features in order to be developed progressively. Moreover, with specifications and schedules managed dynamically, timetables become impracticable and deliveries can only be carried out by shuttles.
The agile “reformation” has open new perspectives and beget many fruitful practices, and the objective here is to see how those approaches of scope and schedule can be reformulated within the perspective of architecture layers. This part examines the congruence between the alternate flows of use cases and the backlog of users’ stories, and considers their complementarity as path-finders. The second part will focus on the role of time-boxes as pace-makers and the benefits for quality assurance.
Architectures and Projects Scope
Whatever the compass, agile or phased, projects footprint can be set across three architecture layers:
Enterprise architectures describe business environments and objectives, resources and regulatory constraints.
System architectures describe enterprises in terms of functional entities of human agents and physical and software assets.
Technical architectures describe the platforms supporting functional entities.
Architecture layers vs Processes
Projects are meant to carry out changes within architectures initiated by business, engineering, or services management processes:
Business processes are defined by business environment and objectives. Changes may have to deal with domains and activities, organization and supported operations, and quality of service as experienced by users.
Engineering processes focus on the development of software systems supporting business processes: business domains and applications, system functionalities, platform implementations.
Services management stand between engineering deliveries and operational concerns: location of assets, access to services, releases deployment, and systems configuration.
While development projects may (and will usually) cross architecture layers, their roots and stakes should nonetheless be clearly positioned if projects are to be planned within the respective time-spans, governed by the relevant authority, and their products accepted by the right stakeholders.
Development Project, from Requirements to Deployments
That put projects governance at crossroads between (a) business objectives set by market opportunities, (b) the deployment of features into functional architectures, and (c) the deployment of releases according to changes in technical architecture. With phased developments, scope and schedules are fixed upfront, which means that the business layer forces its time-frame over the ones of system and technical layers, which may introduce frictions regarding scope as well as quality:
With regard to scope, frictions stem from features and schedules set fully and definitively at enterprise level, independently of any feedback from functional and technical layers.
With regard to quality, frictions stem from tests performed at technical layer when scope and schedules can no longer be revised in case of negative outcomes.
The consequences are all too easy to observe, with business needs partially satisfied and software quality sacrificed. Hence the need of a balanced approach that would consolidate the different maps and time-frames in order to minimize frictions between layers.
Mapping Project Scope
Projects scope can be described along two dimensions, one set by business logic, the other by system functionalities:
First, one have to circumscribe the business variants to be taken into consideration. For that purpose the project footprint, first introduced as users’ stories, will have to be documented by activity or business process diagrams.
Then, the project scope will have to mark out the subset of business requirements to be supported by system functionalities. That will usually be done with use cases describing interactions between system and users.
Complementary descriptions of projects footprints: use cases (interactions between users and system) and activity diagrams (business logic).
That makes those descriptions both orthogonal and complementary: orthogonal because use cases cut across activity diagrams, complementary because use cases are meaningless without targeted activities.
More importantly, they are associated with different architecture layers and governed by different concerns:
At business level (business processes or activities), the perimeter and granularity of requirements must be congruent with the continuity and consistency constraints of business objects and operations.
At functional level (use cases), the span and granularity of interactions between system and users must coincide with execution paths. But the rationale governing users interactions is not the same as the one governing the integrity of business processes. As a consequence, the paths considered for development may pick sequences of operations defined by business processes but should not define them anew based upon interaction constraints.
Finally, assuming that use cases see systems as black-boxes, their footprint should not depend on decisions taken at technical level.
Those concerns can be dealt with separately if projects scope is explored iteratively, e.g using activity diagrams for business logic and use case diagrams for users interactions.
Iterative Mapping of Project Footprint
Iterative development is not just about increments but, first and foremost, about exploring development spaces. That is especially useful when projects overlap architecture layers and cannot rely on fully fledged requirements.
Such projects have to deal with two challenges:
They must identify and manage work units according to the state of requirements and the nature of dependencies (business, organization, or technology, …).
They must carry on with developments based on incomplete specifications while exploring alternatives and deferring decisions until the “last responsible moment” when further delay would limit the options at hand.
Taking a leaf out of the agile book, projects should be driven by users’ value, with requirements first introduced as users’ stories. From that springboard, as informal and incomplete as could be, stories must be fleshed out and organized in order to support the reasoned exploration of project scope.
At inception stories are no more than a user, an objective, and an activity, all set at business level independently of the part played by systems. Scope exploration must therefore begin with activities backbone and be furthered with variants, aka scenarii.
Adding execution paths to project scope
Given a set of business scenarii, the candidates for system support must be ordered and mapped to system features, actual or planned. That should provide a blueprint for development paths. Unfortunately, as variants are added to plots, narratives can easily turn messy, mixing features and capabilities across architecture layers. And that’s where the benefits of use cases are to be found.
Development Paths: From Users’ Stories to Use Cases
Whatever the context, iterations are formal constructs defined by invariants, increments, and exit conditions. When applied to development spaces, iterations are defined by:
Invariants: conditions on architectural assets supporting the scenario under consideration.
Increments: features or variants added to scenarii.
Exit condition: no more features or variants (empty backlog) or time-out.
Applied to architecture layers, invariants provide for reasoned iterations and backlogs:
Enterprise layer are the first to be considered: cycles are set for persistency and execution units and bound by domains (identification mechanisms, integrity constraints, and semantics); within cycles, increments target attributes, operations and variants.
Functional layer come second: cycles are set for interaction units (aka use cases), and bound by the continuity and consistency business objects and activities; increments target transient attributes and operations.
Technical layer come last: cycles are set for platforms and bound by functional units.
But there is a catch: while users’ stories and activity (or business process) diagrams are set at enterprise level, development projects are considered at system level; because systems functionalities are not supposed to appear in users’ stories or activity diagrams, there could be a gap between business and functional requirements. As it happens, use cases provide a bridge: on one hand they focus on the interactions between users and systems, on the other hand their basic and alternate flows can be directly mapped to the paths in activity diagrams.
From alternative flows to Development Paths
That provides a clear and sound basis for the definition of development paths: on one hand alternate flows can be ranked according users’ priorities; on the other hand they determine the sequence of use cases that will have to be developed.
Backlogs and Pathfinders
While the objective of users’ stories is to tie up projects in business value, the objective of use cases is to anchor them in the context of system functionalities. That perspective, and the role of models, may be ignored for standalone projects, but it is necessary when project development paths are to be governed both by business and functional dependencies, described respectively by users’ stories and use cases.
In that case the exploration of development paths should be guided by invariants set along MDA model layers: computation independent (business processes), platform independent (systems functionalities), and platform specific (technology platforms).
Projects are rooted in use cases but development paths are governed by users’ stories.
When projects are rooted in business activities (a), e.g the possibility of upgrading a customer, stories describe execution paths and are ranked according business priorities. Iterations will proceed with development and new cycles added for alternative paths to the basic one.
Depending on context dependencies, development projects can be directly initiated from given sequences of activities (b) or conducted in parallel with users’ stories. Use cases remain the option of choice when the features supporting users’ stories are meant to be shared (e.g checkout). In that case the development paths are governed both by users’ value and functional dependencies. When features are deemed specific (e.g upgrade), use cases can be bypassed and development paths explored simultaneously according users’ and development concerns.
Backlog organization is more complex when development paths cross the divide between functional and technical concerns. Ideally, one would expect a clear separation of concerns, with use cases defined independently of technical options, just like business logic doesn’t depend on system functionalities. But alternatives may be blurred due to the dependencies between interactions design and platform capabilities, the risk being to associate technical options with functional variants, e.g specialized use cases.
Entangled development paths: self check out depends on technical platform.
That’s the case when features can only be implemented on specific platforms. If those features are also specific the corresponding development cycle can be managed as a whole. Otherwise the relevant decisions should be factored out. The same principle applies for features supporting different business processes.
Squaring the Circles: From Epics to Releases
Iterations run within boundaries set by invariants, and with regard to projects scope, those invariants are set by architecture capabilities: enterprise on one hand, systems on the other hand.
From the enterprise perspective, development projects (b) are meant to support business objectives (a), not to define them. As a consequence, users’ stories must remain within borders set upfront. That can be achieved by introducing business projects (aka strategies, aka epics) and portfolios of development ones.
Development Paths: Portfolio of business objectives (a), associated backlogs of users’ stories (b), targeted features (c), architectural capabilities (d), and releases (e).
From the system perspective, a clear distinction should be maintained between projects supported by platforms capabilities (b), and projects targeting platforms capabilities (d). Eventually, those different levels of explorations will have to be consolidated as releases (e), and that is where one may find the agile answer to waterfall.
A Time for Every Purpose: Time-boxes as Pace-Makers
As Einstein famously said, “The only reason for time is so that everything doesn’t happen at once.” In other words time is what happens between events, and the use of a single time-frame will put all events under the same rationale.
But architectures are best understood as shearing layers whose events are governed by different rationales, respectively: business opportunities, engineering constraints, or operational needs.
That is arguably the critical flaw of waterfall solutions as they force business, development, and operations under the same set of strictures. And that’s why agile’s solution to components release may be its pivotal innovation as it establishes the autonomy of those three layers and introduces time-boxes as their pace-makers.
As previously noted, embarking for sizable and lengthy project on the assumption that detailed scope and schedules can be set upfront is a very hazardous policy. Alternatively, agile development models carry out specifications, development, and quality assurance into integrated iterations, making room for a progressive exploration of problem spaces and solution paths, and consequently for informed decision-making and better risk management.
Paths & Paces (S.Dali)
Yet, whatever the method, at some point, scope and features will have to be committed; and with targeted features set dynamically, planned schedules are no more an option. Hence the need to reconsider the way time is taken into account.
Dependencies: Playing for Time
Paraphrasing Einstein, one may say that the only reason for processes is so that everything doesn’t happen at once. Why is that ?
First, processes are meant to support informed decisions. If problem spaces and solution paths cannot be settled upfront they must be done progressively. And that will clearly introduce informational dependencies supporting:
Decisions about what is to be done: alternative paths and their priorities
Decisions about how it should be done: supported features and their priorities
Decisions about how it can be done: quality assurance and acceptance.
In that context the objective of development processes is to do as much of definition, building, and acceptance, and to delay decisions in order to gather the most of the relevant information until the “last responsible moment”, i.e without preempting any of the initial set of alternative options.
Assuming informed decisions are supported by architecture knowledge, processes must take into account engineering constraints regarding:
Technical dependencies associated to the nature of development flows and environments.
Functional dependencies associated to products functionalities and supporting systems.
Organizational dependencies associated to business units and localization.
Iterating across architecture layers
While those dependencies are defined within architecture layers, their impact on the organization of work units may have to be consolidated when projects are carried out across layers.
Schedules: Running for Time
Whether development cycles stay within or run across architecture layers, there will be no way to decide about deliveries and schedules at project inception. In other words dependencies will have to be sorted out and planning settled along the road.
Project planning means consolidating overlapping time-frames. That may not be a problem for projects set within single architecture layers (e.g migration), but that should definitively be taken into account when heterogeneous time-frames govern changes across architecture layers. With regard to changes managed at project level (endogenous events), the consolidation may be done within project time-frame; but that will not be possible for changes occurring in non managed environments (exogenous events).
Those events are set in time-frames governed by their own rationale (e.g business, organization, or technology) that cannot be subsumed into the engineering timetable:
At enterprise level, time is set by business context. Both business objectives and business process solutions are meant to be decided with regard to actual (aka exogenous) business opportunities (a).
At system level, time is set by project planning (endogenous events). Given functional requirements (e.g users’ stories or use cases), architects and designers have to decide about the scheduling of systems functionalities and services, and the release of corresponding applications. Since those decisions are not directly exposed to exogenous events, they can be made according to engineering constraints and resources availability (b).
At platform level, time is set by business and operational objectives (endogenous events), and technical contexts (exogenous events). Assuming that risks are evenly set, the problem is to align endogenous (managed by project) with exogenous (anticipated from contexts) events: too early releases may preclude later but more useful ones; too late releases may hamper operations (c).
The texture of time differs across architecture layers (yellow for endogenous, red for exogenous)
Whereas those time-scales are to be synchronized, there is no reason they would be congruent. Hence the need of some mechanism, static (e.g milestones) or dynamic (e.g backlogs) supporting the scheduling of work units.
From Milestones to Backlogs
As considered elsewhere, phased models of development may offer some benefits when dependencies originate from different environments or involve different authorities; yet they may impose still bigger penalties when requirements cannot be fully settled upfront. Conversely, agile approaches are the option of choice when complex problem spaces and solution paths are to be explored and refined progressively; but may prove ineffective in dealing with business, organizational, or technical dependencies set from outside projects. That difference of perspective is reflected by the mechanisms used to manage dependencies: milestones or backlogs.
Sorting out dependencies means consolidating informational and engineering constraints across architecture layers. When problem spaces and solution paths cannot be managed under the same authority (heterogeneous dependencies) phased development models use milestones to consolidate expectations and commitments across layers and time-frames.
Otherwise (homogeneous dependencies) agile development models use backlogs to explore problem spaces and solution paths, whatever the architecture layer and nature of dependencies. With the benefit of shared ownership, business events are propagated all along development paths, governing backlogs of stories (business requirements), features (engineering constraints), and releases (operational requirements).
Backlogs can be used to consolidate organizational, functional, and technical dependencies.
Along that perspective backlogs and milestones can be seen as mirrored solutions of the same problem, namely how to deal with the functional and timing dimension of dependencies: backlogs deal only with the functional dimension as they consider the dependencies between tasks without taking into account their actual timing; milestones look from the opposite direction, anchoring aggregate tasks to timetables before considering dependencies between items. Depending on the point of view:
Backlogs may appear as stacks of stones: the milestones are fragmented and the initial sequence translated into queues.
Milestones may appear as flattened backlogs: dependencies are frozen and stories are ironed out before being merged into heaps.
Milestones can be seen as “flattened” backlogs, or, alternatively, backlogs as stacked milestones
Hence, with regard to the ranking of dependencies, the difference between backlogs and milestones is of granularity, finer for the former, coarser for the latter. But with regard to execution, the difference is of nature: contrary to milestones, backlogs don’t fix any timing.
That distinction between sequence (functional ranking) and schedule (time ranking) may be critical when iterative development has to be combined with heterogeneous dependencies.
Collaboration Levels
As noted above, the planning of work units must take into account three criteria:
The dependencies between tasks, engineering or informational, determine the sequencing independently of actual timing. They are derived from technical or organizational constraints.
The schedules anchor the start and completion of tasks to time-frames. Some time-frames are set within the enterprise (e.g resources availability), others are set from outside (e.g regulations or business opportunities).
Pace determine the elapsed time taken to complete a task. It may be seen as a metronome used to adjust the throughput depending on resources availability on one hand, quality requirements on the other hand.
Milestones and Backlogs Capabilities
At first sight, comparison shows backlogs to outdo milestones on all three accounts: dependencies are managed at finer granularity, which enables dynamic scheduling and releases management; last but not least, due to iteration cycles and time-boxing, development paces can be aligned with resources and quality requirements. Yet, that edge can be misleading if dependencies translate into unmanageable or multiple backlogs.
Collaboration is at the core of iteration cycles, and it can only be achieved through shared and dynamic management of backlogs. Assuming a set of stories, each new cycle has to be preceded by decisions regarding priorities, refinements, and responsibilities; and if informational dependencies have to be taken into account, these decisions must be taken collectively and directly:
Collectively: decisions about priorities and refinements must be negotiated between team members from both sides of the business/system (or users/developers) divide.
Directly: all decisions must be taken by the team itself, without any organizational mediation or external interference.
But those conditions could be thwarted if different projects have to develop stories with shared features implemented on different platforms.
Shared features and cross implementations could thwart collective and direct decision-making.
In that case consolidation mechanisms could bring back fixed scope/schedule configurations:
Different teams, each with its own specific users and developers concerns, will have to commit to agreed features and timetables.
By involving independent organizational units, those decisions will entail some procedural mediation carried out independently of iteration cycles.
Yet, that shouldn’t necessarily be the end of the stories, providing some mechanism could be found to synchronize iteration cycles. And that could be achieved with blackboards.
From Backlogs to Blackboards
Blackboards can be understood as shared backlogs stripped from their ranking mechanism so that items can be consulted and dealt with by different project teams. Alternatively, they can also be seen as timetables stripped from scheduling mechanism. Either way, those views illustrate how blackboards may support shared dependencies without forcing them into time-frames:
Shared features are posted on a single blackboard where their status can be consulted and updated by the teams in charge of the stories concerned.
Development teams post their releases on blackboards according to targeted platforms.
Blackboards are used to manage shared dependencies and differentiated stories’ contents
As it happens, that approach also answers the recurring question of stories’ nature and granularity: with the benefit of blackboards, fine-grained stories can be indexed as business cases, use cases, or new releases and cohabit in backlogs.
Combining backlogs with blackboards provides a collaboration mechanisms supporting external dependencies without impairing teams ownership of iteration cycles. Yet it is not a substitute for schedules as it doesn’t deal with the alignment of cycles with enterprise time-frames.
Time boxes as Pacemakers
Milestones and backlogs are synchronization mechanisms. The former, combined with timetables, coordinate teams along time-spans; the latter, combined with time boxes, coordinate tasks between cycles. Hence, while both aim at the same objective, namely that everything doesn’t happen at once, their understanding of time is very different: timetables are bound to an external measure of time, time boxes are just arbitrarily fixed intervals that can be tethered to any time-frame.
As a consequence, time boxes can be applied at different levels. At system level they are associated to project backlogs and used to set the tempo of iteration cycles. At enterprise level they are associated to business objectives and anchored to strategic plans. Assuming those plans are described by epics, it would be possible to use different tempos depending on level (enterprise or systems) or even applications.
Time boxes can be combined with blackboards to synchronize tempos between enterprise level (T) and interrelated projects (T/x and T/y).
Ideally, differentiated tempos should foster an emerging harmony between melody (users’ stories) and accompaniment (supporting systems) converging into the fulfillment of strategic objectives. Practically, the alignment of releases with epics cannot be taken for granted.
Squaring the Circles: Project Planning
Agile approaches are based upon the dynamic exploration of problem spaces and the iterative development of solution paths, the objective being to maximize the value of functional requirements under the constraints of technical ones. Yet, with spaces and paths defined dynamically, standard exploration procedures like breath or depth-first traversal are useless because the ranking of paths is part of the solution, which means that priorities must be revised at the end of each cycle.
Nonetheless, explorations cannot be everlasting and there must be some end to the story, when stakeholders get what they expect (or accept what they get) and users can begin to reap the benefits. Fixed scope and schedules being ruled out, the problem is to align iterations outcomes (b) with business objectives (a). Solutions can be positioned between two archetypal approaches, one driven by business objectives, the other by development tasks.
How to align tasks (b) with objectives (a)
The first approach would see development teams take commitments with regard to broadly defined objectives: with problem spaces represented by trees progressively refined and explored, commitments can be made collectively on sets of solution paths within still undefined sub-trees (b1). The team will then take responsibility for the details of iteration cycles and will adjust its throughput as to align releases with business objectives.
Alternatively, and providing a finer granularity can be obtained and managed, stories could be broken down into tasks for which commitments will be made individually by team members (b2). Assuming that the tasks workloads can be assessed, iterations could then be planned and releases scheduled on the basis of time boxes parameters.
Commitments can be made collectively with regard to objectives (b1) or individually with regard to tasks (b2).
Not surprisingly, the choice of a planning policy is to be conditioned by the granularity of work units:
Task based planning requires finer grained stories and comes with a phased flavor by reintroducing analysis. That may stretch the intervals between iterations and increase management overheads.
Objective based planning allows for coarser grained stories and is more in line with agile spirit. Yet that may increase the length of iterations and affect their transparency.
All things considered, some feedback loop may be needed when deciding on the size of stories because shorter ones do not necessarily decrease the time to market as they may generate bigger backlogs and exponential overheads in case of complex dependencies.
Balancing Pushes and Pulls: Lean and Just-in-Time Workflows
When push comes to shove project planning turns into conflict management. That may happen with phased development models as well as with agile ones. With the former that will be due to applications forcibly pulled out whatever their value for users and reliability in order to meet unrealistic expectations. With the latter that will be due to requirements forcibly pushed into backlogs disregarding their size and exponential complexity.
The way out of this dilemma is a feedback mechanism between project teams pushing releases and business stakeholders pulling applications. And that is the rationale behind the Kanban development model:
Visualize workflow: up-to-date expectations, constraints, commitments and achievements must be clearly and selectively visible to all concerned. that can be done with backlogs and blackboards.
Limit work in progress: that can be obtained by regulating the selection of solution paths through limits on the size of backlogs and the fragmentation of users’ stories according to their architecture footprint.
Measure and manage flows: metrics and process design can be significantly improved if flows are differentiated depending on architecture layer (business, systems, platforms).
Make process policies explicit: that is already a cornerstone of agile backlog management; those principles should also apply to backboards.
Use models to recognize improvement opportunities: while initially ignored, the benefits of models in agile development is progressively acknowledged. Those benefits have be illustrated in the first part of this article by the use of activity diagrams for the exploration of problems spaces and solutions paths.
Applying those principles will bring about lean processes and just-in-time workflows, improving both users’ value and software quality.
Assuming, for the sake of the argument, that programs are models of implementations, one may also argue that the main challenge of software engineering is to translate requirements into models. But, contrary to programs, nothing can be assumed about requirements apart from being stories told by whoever will need system support for his business process.
Telling Stories with Models
Along that reasoning, one may consider the capture and analysis of requirements under the light of two archetypal motifs of storytelling, the Tower of Babel and the Rashomon effect:
While stakeholders and users may express their requirements using their own dialects, supporting applications will have to be developed under the same roof. Hence the need of some lingua franca to communicate with their builders.
A shared language doesn’t necessary mean common understandings; as requirements usually reflect local and time dependent business opportunities and goals, they may relate to different, if not conflicting, aspects of contexts and concerns that will have to be consolidated, eventually.
From such viewpoints, the alignment of system models to business stories clearly depends on languages and narratives discrepancies.
Business to System Analyst: Your language or mine ?
Stories must be told before being written into models, and that distinction coincides with the one between spoken and written languages or, on a broader perspective, between direct (aka performed) and documented communication.
Direct communication (by voice, signs, or mime) is set by time and location and must convey contexts and concerns instantly; that’s what happens when requirements are first expressed by business analysts with regard to actual and specific goals.
Direct communication requires instant understanding
Written languages and documented communication introduces a mediation, enabling stories to be detached from their native here and now; that’s what happens with requirements when managed independently of their original contexts and concerns.
Documented communication makes room for mediation
The mediation introduced by documented requirements can support two different objectives:
Elicitation: while direct communication calls for instant understanding through a common language, spoken or otherwise, written communication makes room for translation and clarification. As illustrated by Kanji characters, a single written language can support different spoken ones; that would open a communication channel between business and system analysts.
Analysis: since understanding doesn’t mean agreement, mediation is often necessary in order to conciliate, arbitrate or consolidate requirements; for that purpose symbolic representations have to be introduced.
Depending on (1) the languages used to tell the stories and (2) the gamut of concerns behind them, the path from stories to models may be covered in a single step or will have to mark the two steps.
Context and Characters
Direct communication is rooted in actual contexts and points to identified agents, objects or phenomena. Telling a story will therefore begin by introducing characters and objects supposed to retain their identity all along; characters will also be imparted with behavioral capabilities and the concerns supposed to guide them.
Stories start with characters and concerns
With regard to business, stories should therefore be introduced by a role, an activity, and a goal.
Every story is supposed be told from a specific point of view within the organization. That should be materialized by a leading role; and even if other participants are involved, the narrative should reflect this leading view.
If a story is to provide a one-lane bridge between past and future business practices, it must focus on a single activity whose contents can be initially overlooked.
Goals are meant to set specific stories within a broader enterprise perspective.
After being anchored to roles and goals, activities will have to be set within boundaries.
Casings and Splits
Once introduced between roles (Who) and goals (Why), activities must be circumscribed with regard to objects (What), actions (How), places (Where) and timing (When). For that purpose the best approach is to use Aristotle’s three unities for drama:
Unity of action: story units must have one main thread of action introduced at the beginning. Subplots, if any, must return to the main plot after completion.
Unity of place: story units must be located into a single physical space where all activities can be carried out without depending on the outcome of activities performed elsewhere.
Unity of time: story units must be governed by a single clock under which all happenings can be organized sequentially.
Stories, especially when expressed vocally, should remain short and, if they have to be divided, splits should not cross units boundaries:
Action: splits are made to coincide with variants set by agents’ decisions or business rules.
Place: splits are made to coincide with variants in physical contexts.
Time: splits are made to coincide with variants in execution constraints.
When stories refer to systems, those constraints should become more specific and coincide with interaction units triggered by a single event from a leading actor.
Filling the blanks
If business contexts, objectives, and roles can be identified with straightforward semantics set at corporate level, meanings become more complex when stories are to be fleshed out with details defined by the different business units. That difficulty can be managed through iterative development that will add specifics to stories within the casing invariants:
Each story is developed within a single iteration whose invariants are defined by its action, place, and time-scale.
Development proceed by increments whose semantics are defined within the scope set by invariants: operations relative to activities, features relative to objects, events relative to time-scales.
A story is fully documented (i.e an iteration is completed) when no more details can be added without breaking the three units rule or affecting its characters (role and goal) or the semantics of features (attributes and operations).
Iterations: a story is fully fleshed out when nothing can be changed without affecting characters’ features or their semantics.
From Documented Stories to Requirements
Stories must be written down before becoming requirements, further documented by text, model, or code:
Text-based documentation uses natural language, usually with hypertext extensions. When analysts are not familiar with modeling languages it is the default option for elicitation and the delivery of comprehensive, unambiguous and consistent requirements.
Models use dedicated languages targeting domains (specific) or systems (generic). They are a necessary option when requirements from different sources are to be consolidated before being developed into code.
Code (aka execution model) use dedicated languages targeting execution environments. It is the option of choice when requirements are self-contained (i.e not contingent to external dependencies) and expressed with formal languages supporting automated translation.
Whatever their form (user stories, use cases, hypertext, etc), documented requirements must come out as a list of detached items with clearly defined dependencies. Depending on dependencies, requirements can be directly translated into design (or implementation) models or will have to be first consolidated into analysis models.
Telling Models from Stories
Putting aside deployment, development models can be regrouped in two categories:
Analysis models describe problems under scrutiny, the objective being to extract relevant aspects.
Descriptions and specifications look from different perspectives
Seen from the perspective of requirements, the objective of models is therefore to organize the contents of business stories into relevant and useful information, in other words software engineering knowledge.
Following the principles set by Davis, Shrobe, and Szolovits for Knowledge Management (cf readings), such models should meet two groups of criteria, one with regard to communication, the other with regard to symbolic representation.
As already noted, models are introduced to support communication across organizational structures or intervals of time. That includes communication between business and systems analysts as well as development tools. Those aspects are supposed to be supported by development environments.
As for model contents, the ultimate objective is to describe the symbolic representations of the business objects and processes targeted by requirements:
Surrogates: models must describe the symbolic counterparts of actual objects, events and relationships.
Ontological commitments: models must provide sets of statements about the categories of things that may exist in the domain under consideration.
Fragmentary theory of intelligent reasoning: models must define what artifacts can do or can be done with.
The main challenge of analysis is therefore to map the space between requirements (concrete stories) and models (symbolic representations), and for that purpose traditional storytelling may offer some useful cues.
From Fictions to Functions
Just like storytellers use cliches and figures of speech to attach symbolic meanings to stories, analysts may use patterns to anchor business stories to systems models.
Cliches are mental constructs with meanings set in collective memory. With regard to requirements, the equivalent would be to anchor activities to primitives operations (e.g CRUD), and roles to functional stereotypes.
Archetypes can be used to anchor stories to shared understandings
While the role of cliches is to introduce basic items, figures of speech are used to extend and enrich their meanings through analogy or metonymy:
Analogy is used to identify features or behaviors shared by different stories. That will help to consolidate the description of business objects and activities and points to generalizations.
Metonymy is applied when meanings are set by context. That points to aggregate or composite objects or activities.
Primitives, stereotypes, generalization and composition can be employed to map requirements to functional patterns. Those will provide the building blocks of models and help to bridge the gap between business processes and system functionalities.
The value of a model is its fitness to purpose. Missing this simple truth will inevitably trigger a “flight for abstraction” and begets models devoid of any anchor to business relevancy.
Abstraction ladders must be propped up by actual contexts (A. Magnaldo)
Yet, that pitfall can be avoided if requirements and models are put in perspective:
Requirements are meant to describe systems in their business context, models describe system artifacts. They should not be confused because the former are supposed to be rooted in concrete descriptions, while the latter aim at their abstract representation.
Models are built from nodes and associations. Nodes refer to instances which are supposed to be uniformly and consistently identified in both business and system contexts; associations may refer to instances (relationships and flows) or classes (specialization and generalization), the former with consistent semantics for business and system realms, the latter with semantics specific to system artifacts.
Along that perspective, the mapping of requirements into models can be achieved by applying selectively the two faces of abstraction: first removing information from the description of actual contexts, then building symbolic representations according to business concerns.
Business Context and Concerns
Starting with requirements, the challenge is therefore to move up and down the abstraction ladder until one gets the focus right, providing a clear and sharp picture of business context and concerns.
With regard to systems engineering, models’ semantics are unambiguously determined by their target: business environments or systems artifacts:
Models of business environments describe the relevant features of selected objects and behaviors, including supporting systems. Such models are said extensional as they target subsets of actual contexts.
Models of systems artifacts specify the hardware and software components of supporting systems. Such models are said intensional as they define artifacts to be created.
Business analysts figure models from actual contexts and concerns, software architects specify blueprints for artifacts
Climbing up the abstraction ladder, the objective is to align descriptive models of objects and behaviors with prescriptive models of system artifacts. That can be achieved in three steps:
Awareness of contexts: mind the business pie, drop everything else.
Domains of concern: say what features mean.
Symbolic representations: consolidate the descriptions of surrogates.
It is worth to note that the first two levels deal respectively with instances and features of actual objects and activities, while the third deal with artifacts. As a corollary, abstraction at the first two levels should be understood in terms of partitions and subsets, with subtypes introduced only for symbolic representations.
Awareness of Context
As illustrated by sounds (filtering noises) or optics (image point), focusing is a basic perceptual task targeting actual instances of objects, events, or activities. With regard to models, it can be achieved with a pronged move:
Adjust the depth of field to encompass the relevant business context. Large depths of field (aka deep focus) will cover concerns across domains, small ones (aka shallow focus) will support specific business concerns.
Single out image points for identified objects or activities deemed to be pivotal.
Field 1 is too small, field 3 is too large. “Dimensions” has no identity of its own, “Broom” is pointless.
A too shallow focus will capture only some of relevant objects (Piano), activities (Move) or events (Concert). Conversely, extending the focus may go too deep, including irrelevant items (Trumpet, Violinist, Illness, or Repair). Moreover, some image points may depend of others for their identity (Dimensions), or be pointless altogether (Broom).
Domains of Concerns
While business contexts are the same for all, business concerns are by nature specific to domains. The challenge for requirements capture is therefore to anchor specific features to shared objects and activities whose identities are set by business context.
For that purpose concerns are to be organized into domains responsible for the identification of anchors (objects, agents, activities) and the semantics of features:
Shared domains deal with anchors whose continuity and consistency have to be managed across domains, independently of activities.
Specific domains deal with anchors whose continuity and consistency can be managed within a single domain.
At this stage the challenge is to distinguish between identified instances of business objects (piano, concert) and processes (cleaning, moving, playing) on one hand, and the description of roles (mover, cleaner, pianist) and business logic (clean, move, play) on the other hand .
Domains of Concerns and Business Processes
It must be reminded that such models are still at ground level as they describe sets of instances; yet, they can also be seen as the first step up the abstraction ladder, as their objective is to extract relevant features and overlook others.
Symbolic Representations
While business concerns are partial and biased, the symbolic representations managed at system level must be comprehensive and consistent; that’s the objective of requirements analysis.
To start with, symbolic representations are introduced for each set of objects, roles or activities:
Objects surrogates: used to manage the continuity and consistency of business objects independently of business processes (piano, concert).
Process surrogates: used to manage the continuity and consistency of business operations independently of business objects (move, play, clean).
Roles: used to manage the interactions between actual agents and system functionalities (mover, cleaner, pianist). When the continuity and consistency of operations performed by agents are managed (e.g pianist), roles must be associated to surrogates.
Activities: used to describe business logic (move, play, clean).
Business logic and surrogates (#) for objects, processes and pianist .
Those are “flat” descriptions representing ground level instances. In order to be effectively supported by systems, models may have to be expanded downward by specialization, or upward by generalization.
Levels of Abstraction
As already noted, specialization and generalization are not symmetric because, contrary to the former operation, the latter one does modify the semantics of existing artifacts.
The purpose of specialization is to introduce specific descriptions for subsets of instances or features. For instance, assuming requirements are about moving pianos, the representation must climb one step down the abstraction ladder, from concerts to concerts with pianos:
Solo piano concerts are a subset of concerts subject to the same identification mechanisms and integrity constraints (strong inheritance).
The description of moving operations is not used to manage instances and its specialization is only about features (weak inheritance).
Climbing down: specialization of features (Move Piano) and surrogates (Solo Concert)
The purpose of generalization is twofold as it may be limited to features (aka aspect generalization) or targets both identification mechanisms and features (aka object generalization).
Aspect generalization introduces a base artifact for the description for shared features. Such base artifact is said to be abstract because its inheritance is limited to features and it cannot support the instantiation of surrogates, specialized artifacts keeping their own identification mechanisms. As a corollary, the level of abstraction is not modified because the model remains anchored to the same sets of instances. For instance (a), some administrative procedures can be defined uniformly for all maintenance operations otherwise described and executed independently.
Climbing up: generalization of features only (a), and for both features and individuals (b).
That’s not the case with object generalization which redefines the initial sets of surrogates as subsets of newly created super-sets. For instance (b), a cleaning process becomes a maintaining processes without the repair extension. Since maintaining processes can be created as simple cleaning or repairing ones, the model is anchored to different levels of abstraction. And since descriptions should not cross levels, roles must be specialized similarly: maintainers are to be identified as such before being qualified as mechanics, even if their interventions are not managed as such (inheritance of transient identification mechanism).
Getting A Proper Grip
Models are neither true or false and can only be assessed for consistency and effectiveness.
Hence, assuming that (1) systems are meant to handle surrogates of business objects and processes and, (2) those surrogates are designed from models, it ensues that (3) a litmus test of model effectiveness would be the grip it provides on relevant objects and processes.
A grip on context and concerns
And that can only be achieved by pinning models to concrete and identified business objects and processes. That provides a template for modeling grips: concrete descriptions with primary identification in the middle, abstract ones above, aspects or concrete descriptions with inherited identification below.
Whereas UML has been brought to existence by very wise men under very propitious skies, the initial enthusiasm and first successes have never been transformed into wider acceptance and customary usage; subsequent updates and extensions didn’t help and may even have triggered some anticlimax. More than fifteen years after its launch, the utilization of UML remains limited, both in breadth (projects developed) and depth (features effectively used). Moreover, the UML house is deeply divided and there isn’t much consensus among the few that use it comprehensively and consistently, principally to support domain specific languages (DSL).
The Divided House of UML (Anne Desmet)
Certainly, there must have been a wrong turn somewhere, possibly at the UML2 crossing when the OMG committee lost sight of users modeling needs and took the road to meta-models. Considering UML’s shrinking stamp and dwindling relevancy, that road appears more and more like a dead-end; but it may be still possible to get back on track and retrieve the Us of the UML: unified semantics for all and sundry users.
Where to Look
Whether on driving or back seats, respectively for model driven or agile methods, models are widely accepted as a necessary constituent of development processes. Nonetheless, and despite being the only official standard, UML standing appears to falter, up to be already seen as a cold case. As suggested by Ivar Jacobson (“The road ahead for UML“), one of its main drawback would be its lack of modularity with regard of users needs. If that flaw is to be fixed, the question is where to look: directly at language level, or at supporting mechanisms.
Given the broad consensus that surrounded the initial project, one should at first look for a sound and stable subset to be used as a backbone and fleshed out according specific contexts, purposes or users. As a matter of fact that is what stereotypes and profiles are meant to do, except that without a well-defined backbone of unambiguous constructs, the only possible outcomes are domain specific languages. So, one should first consider how the separation of concerns could be better supported by language constructs.
Language Constructs, Model, and Separation of Concerns
Separation of Concerns
Despite its roots in the Object Oriented paradigm, UML has demonstrated its adaptability to all and every method or domain. Unfortunately, being a Jack of all trades often means a master to none, and the use of UML is clearly frustrated by its versatility; that translates either into shallow usage of ambiguous semantics, or into extensions targeting specific domains or technologies.
On the ground, three mechanisms can be used to make for the lack of focus: stereotypes, views, and customization.
UML stereotyping mechanism support predefined constructs for problem (business objects and processes) or solution (system architecture and object design) spaces. Stereotypes can be grouped into profiles, e.g for specific business domains or technical architectures.
Views (or perspectives) organize access to models according contents: logical, physical, conceptual, pragmatic, etc
Tool customization organizes access to models according users purposes and skills: analyst, architect, designer, developer, etc.
While those approaches have their benefits, they are set independently of languages constructs, either as UML extensions (stereotypes and profiles), or defined from outside by development methodologies (views) or projects organization (customization). As a consequence, they have little or no effect on the simplicity or efficiency of UML; they may even add to confusion and complexity when overlapping stereotypes are introduced to support multiple taxonomies, e.g technical architectures and business domains.
Language Constructs (a), Stereotyped Model (b), Combined Views or Profiles (c)
That may point to a clear direction: given the potency of the stereotyping mechanism and its pivotal role in UML utilization, significant benefits could be achieved through a better integration into core language constructs, even if that entails some constraints or limitations. Two straight modifications should be considered:
Model layers: language constructs should be re-organized along architectural concerns for enterprise (business processes), system (functionalities), and platforms (components).
Stereotypes visibility: language constructs should support the distinction between local taxonomies and “unified” ones, the former set with limited scope and visibility, the latter meant to be applies across layers.
While both modifications can be carried out on their own, their benefits would be boosted if they were set within the broader MDA framework and supported by specific language constructs.
Modular Language Constructs
Given the growing intricacy, ubiquity and diversity of systems, UML complexity and versatility should clearly be in demand, and the problem is to harness those capabilities according the needs and skills of the different kinds of users.
That’s arguably a critical flaw of UML, which lumps together essential with secondary constructs, as well as definite with ambivalent semantics. That brings weighty consequences, both for users and models:
Steep or even abrupt learning curve: confronted to a wall of mixed constructs users have to master the whole upfront, whatever their needs and skills.
Blurred concerns: describing various specific contents with the same ambivalent constructs will either distort language semantics, or blur concerns specificities.
Corrupted transformation: whatever the modeling tools, the bad apples of inputs will usually corrupt the whole of outputs. In other words any advance in model driven development requires a sound backbone of unambiguous language constructs.
As noted above, language constructs can be regrouped along two perspectives, one directly associated with users architectural concerns, the other reflecting the scope and visibility of targeted artifacts. While there is no particular reason to match complexity levels with architectural concerns, mapping them to granularity has a clear rationale. Such a “born again” UML would distinguish between two levels of language constructs:
Those pertaining to objects and activities identified by architectures, whatever their nature: enterprise, systems or platforms.
Those used to describe internals of objects and activities independently of their aspects and behaviors at architecture level.
Model Transformation: lumped (b) vs differentiated (a) language constructs.
That re-configuration would bring modularity to the language, enabling a smooth learning curve. More importantly, a clear-cut separation of concerns will enable some kind of Just-In-Time model transformation: instead of cumulative noises (b), one will get separate transformations for models architectural backbone on one hand, contingent specificities on the other hand (a). And that could be a real game-changer for lean and fit models.
While that could be achieved by different means, a simple solution would be to use the stereotyping mechanism to describe supporting structures of enterprise, functional, and technical architectures.
Transformation vs Portability
Model transformation is about changing contents within the same environment, portability is about moving the same contents across different environments; and despite apparent similarities, they deal with different concerns, set by users for the former, by tools vendors for the latter.
Transformation is normally performed under a single corporate roof according agreed semantics; as a corollary, it is meant to cover the full contents of models. That’s not the case for portability, whose primary objective is the exchange of consolidated contents between heterogeneous environments; while sources and targets may have to share the whole of their models, a sound policy should make room for selective portability of specific or confidential contents.
The Meta-Object Facility (MOF) is the solution of choice for portability. As a meta-language it is used to describe language constructs at source and target environments; mapping rules can then be defined and bridges built between environments. As it is, those bridges usually scale very poorly due to the exponential complexity of rules having to cover all and every model idiosyncrasies; and that’s unfortunate for portability which, instead of focused targets, has to deal with overweight models cluttered with useless contents (b).
Portability between modeling environments: Lumped (b) vs Differentiated (a) constructs.
That situation would be greatly improved (a) if the wheat of consolidated constituents could be separated from the chaff of ambiguous or irrelevant contents. On a broader perspective that will open the way leaner and fitter models.
One step back may put UML back on track
There is something of a consensus among the software engineering community regarding (1) the benefits of models and (2) the failures of UML. As should be expected, that consensus translates into fragmented modeling practices and, more generally, software engineering methodologies. Obviously there isn’t much of a future for UML along that path, but the case is still open and the trend can be reversed by putting users needs back on UML driving seat.
All too often when the agile project model is discussed, the debate turns into a religious war with waterfall as the villain. But asking which project model will bring salvation will only bring attrition, because the question is not which is the best but when it is the best. It’s like asking if a hammer is a better tool than a sickle !
B2C: Balancing Brain
Instead, one should first try to understand how they stand apart, and deduce from that what they are best for; the comparison between the left and right sides of the brain may provide a good starting point.
B2C: Balancing Brain Capacities
If it is (still) impossible to know what people think, it is possible to know where their thinking is rooted in brains, and the answer is unequivocal:
The left side of the brain is analytical; faced with a problem, it looks for parts and process them in sequence.
The right side of the brain is better at synthesis as it looks at the whole and processes all relevant information simultaneously.
Obviously casts will differ between individuals depending on inborn qualities and developed preferences; moreover, each individual will balance his brain sides according to the task at hand. The same should apply when projects must decide between iterative and procedural approaches.
What a Hand can Hold
When project management is first considered, the Whole vs Parts alternative should be the discriminating factor: since human brains cannot process an unlimited number of elements simultaneously, work units to be handled by teams must be clearly circumscribed, with a number of independent functional units not exceeding a dozen.
That could be a pitfall for agile developments if iterations and increments were to be associated with an exponential growth of complexity. Yet, partitioning a large project into sub-tasks doesn’t necessarily call for a waterfall schema if the sub-tasks can be performed independently.
What the Hand is Told
Sequential processing can be dumb because the intelligence can already be etched in the sequences. That’s not the case if relevant information is to be picked out and processed as a whole; that can only be done with a clear purpose guiding the hand.
Replacing an administrative process by a collaborative one entails some kind of shared ownership, with teams granted full responsibility for decisions, schedules, and quality. Otherwise the different concerns, purposes or authorities, possibly but not necessary at odds, should be set apart as sub-tasks, and milestones introduced for their consolidation.
What is Handed Over
Development projects may handle three kind of artifacts: texts, models, and code, the first and last being mandatory, the second being optional. Since texts and code are best processed sequentially they are handed over to brain left side; conversely, models are meant to combine different perspectives, e.g structures and behaviors, which put them on the right side of the brain.
Curiously, that seems to put agile in some kind of conundrum: despite models being the symbolic representation best suited to holistic processing, agile approaches are partial to code, even if models are not explicitly ruled out. As a matter of fact, agile tenets are more partial to products than to code, and what is handed over and tested against requirements is not meant to be a program but a running application.
Hand in Hand
Just like the two parts of the brain bring their best to shared concerns and purpose, agile and phased schemes should be enlisted according to their respective merits and shortcomings:
Agile is clearly a better option when shared ownership can be secured and milestones and models are not needed.
Phased solutions (“waterfall” is a red-herring) are necessary when organizational, functional or technical dependencies between projects mean that some consolidation cannot be avoided between development process.
Assuming agile methods are used whenever possible, models should provide the glue when external dependencies are to be taken into account:
Organizational dependencies are managed across model layers: business requirements govern system functionalities which govern platforms implementations.
Functional dependencies are managed across architecture tiers: transient non shared components (aka boundaries) are governed by transient shared components (aka controls) which are governed by persistent shared components (aka entities).
Development dependencies should not cross projects limits as they should be managed at domain level using inheritance or delegation.
