Agile & Models

Objective

Agile principles to software development are essentially about project and team management, with only sparse recommendations regarding products engineering or processes design. Given this seeming oversight some have even argued that there is no room for models and milestones under the agile roof. Such misconceptions could be unfortunate, especially for project managers having to deal with organizational or architectural dependencies. But omission is not opposition as authors like Robert C. Martin or Dean Leffingwell have demonstrated respectively for products and processes.

With the benefits of hindsight, the debate about agile vs traditional project management has turned into a debate about their respective benefits and drawbacks, and misunderstandings about milestones and models have been progressively ironed out.

Regarding traditional approaches, poorly defined and managed requirements are widely recognized as a major source of projects failures. And that is especially true for waterfall ones with their upfront and rigid approach to scope, changes, and quality:

• Fixed requirements assumption: since there is an inverse  relationship between the level of detail and the stability of requirements, defining all requirements upfront is arguably a very hazardous policy.
• Quality as an afterthought: as a detached concern, quality may easily turn into a collateral damage when set along mounting costs and scheduling constraints.

Agile methods, while providing organizational solutions to projects with well circumscribed requirements, do not go further into dealing with the factors that make requirements difficult to manage and acceptance tests difficult to define, namely projects external dependencies, organizational or technical.

For that purpose it is necessary to introduce intermediate outcomes (textual specifications, test cases, models, code, etc.),  in order to deal with discontinuities:

• Time discontinuity: when work units cannot be performed in continuity, output/input flows must be managed during intervals.
• Team discontinuity: when work units are to be performed by different teams (whatever the reason, technical or organizational), shared development flows must be managed on their own.
• Semantic discontinuity: when abstractions are to be introduced in order to align specific business concerns (users’ stories) with broader objectives.

Defining and managing those intermediate outcomes, whatever their support, is at the root of development process design and assessment.

Requirements and External Dependencies

Projects open on requirements and close on acceptance. Waterfall ones define full requirements upfront before proceeding with developments; Agile ones start with lean requirements to be refined, consolidated, and tested en-route. While shortened time-span and continuous updates should provide for greater accuracy and stability, they cannot by themselves prevent external factors from affecting requirements. Apart from “internal” dependencies between business processes, system functionalities, and platform implementations, projects may be subject to “external” dependencies across contexts:

• Enterprise architecture may affect requirements through organizational dependencies.
• New applications are often supported by existing functional architectures.
• Development dependencies are defined by constraints on artifacts structure and reuse.

All those concerns, problems, and solutions must be managed on their own, by models or otherwise.

Running without Models

From its origin, the agile approach has been defined in opposition to traditional phased (aka waterfall) methods:

• Agile project planning merge traditional tasks (e.g requirements, analysis, design, coding) into unified work units.
• Instead of sequenced execution of specialized tasks, jack-of-all-trade work units are performed iteratively.
• Whenever possible, textual or graphical descriptions (requirements, specifications, documentation, etc) are replaced by working software components.
• Covenants between customers and providers are replaced by shared ownership and collaboration.
• Features based planning (schedules are fixed for targeted features) is replaced by time-boxing (releases are made of whatever features may have been developed during a fixed time-span).

With users and developers working together to define, build, and test products, there is no need for models and actual developments can be organized around three kinds of artifacts:

1. Features: system functionalities meant to support business objectives. They may also be associated with non functional requirements (NFR).
2. Stories: realization of system features in terms of users’ interactions. Stories usually relate to business objectives. They are used to define, build, and test system features.
3. Releases: products implementing features (from stories) and components (from NFR) are released at fixed intervals by the Agile Release Train (ART).

Yet, this apparent simplicity is not without constraints, and a typical agile project has to fulfill three basic conditions (inspired by Aristotle’s three unities for drama):

1. Unity of action: a project should have one main objective that it follows, with secondary threads, if any, set within circumscribed requirements and returning to the main thread after completion. In other words, the project team must constantly know about development dependencies.
2. Unity of place: developments should be executed into a space within which all resources can be obtained directly and communication carried out without mediation.
3. Unity of time:  work units should be set within fixed and continuous time-frames. That is a necessity if innards and synchronization of tasks are to be managed independently of context contingencies.

Set in the broader perspective of enterprise architecture, agile projects can be characterized by:

• Portfolios, where business objectives are translated into consistent strategies (aka “epic” stories), to be eventually implemented by business processes. Business strategies may be associated with non functional requirements and schedules.
• Project teams, with users and developers taking responsibility for components or features, designed from stories or use cases, and implemented by running software (products).
• Releases, driven by fixed time-boxing and quality as controlled variables, scope becoming an adjustment variable managed according portfolio schedules and architecture constraints.

But on that basis, agile approaches may be challenged when:

• Physical constraints bear upon execution of engineering tasks.
• Non functional requirements introduce architectural dependencies between features.
• Business objectives entail cross dependencies different organizational units.
• Decisions are contingent on contexts uncertainties, resources availability, or tasks completion.

Those circumstances may call for intermediate models and milestones.

Dealing with Physical Constraints: Split up requirements

Discontinuities can be directly caused by geographical constraints or indirectly by engineering ones (e.g access to physical resources). The solution in that case should be to split the stories between teams and to consolidate features at program level:

1. Assuming a single authority at portfolio level, stories can be allocated to different teams subject to the same architectural constraints.
2. Each team defines, builds and tests its products as usual.
3. The program manager merges the teams’ outcomes into a single Agile Release Train (ART) according the targeted features and schedules.

In that case the different teams work in parallel on parts of the same development flows and there is no need of intermediate milestones or models.

The same organization can be applied for projects too large to be developed by a single team.

Dealing with Technical Dependencies: Specialized Teams

Because non functional requirements deal with system capabilities independently of specific business applications, they are not readily expressed as users’ stories. Moreover, their realization usually calls for technical skills and expertise focused on systems architecture. For both reasons a distinction is often introduced between (1) features supporting users’ stories, and (2) components, seen as technical assets shared across features.

A simplistic organization would set features and components teams working in parallel, but that would introduce a dependency as features could not be tested independently of non functional components.

As far as technical dependencies between components and features remain local, they can be dealt with through project planning and release management without the need for models. But models will have to be introduced if components are shared by features from different projects and reused in different contexts, in other words if dependencies are architectural.

Dealing with Architecture Dependencies: Models and Processes

Sharing architecture components across projects calls for organizational solutions. One option would be to adopt component-based processes that will split the development of new features between components teams organized along architectural layers: boundaries, control, persistency, and communication. In that case models will have to be introduced in order to manage development assets and support communications between teams.

While such an organization is well adapted to large enterprises with complex architectures, it clearly contradicts a core agile tenet, namely that the whole development processes should be driven by business value.

Alternatively, feature-based processes could allocate the development of components to features teams according their technical skills. That would keep projects solidly anchored to users needs, but would also require a broader specialization of developments teams.

With the collaborative and incremental approach still maintained, models of shared architecture components may not have to be managed; yet their absence will probably raise the risks of “silo” developments.

Hence the benefits of pragmatic solutions in line with the sharing of development assets:

1. Persistency and communication layers form the core of shared architecture components and are therefore less prone to change in business needs and short-term adjustments;
2. Control layers implement business logic and are meant to support business processes. Some belong to specific domains, others are shared across applications. The former may change according to contexts, concerns, or opportunities, the latter provide for continuity, but both are definitively driven by users needs.
3. Boundaries are specific both to business domains and technical platforms, as such they should directly adjust to users needs.

A mixed organization would combine component and feature driven processes, using models to manage shared development assets.

Theoretically, all projects could be run along core agile principles, some driven by users stories, some by system requirements (functional or not), with models used to describe shared architectural assets. Practically that would more difficult for architecture ones if their requirements are not defined at a single source. That would call for a “Disciplined Agile Architecture” as proposed by Scott Ambler

Dealing with Organizational Dependencies: Services

Whereas architecture dependencies are technical and are supposed to be managed by a single authority, organizational ones appear when objectives set at enterprise level call for the collaboration of different units, with their own responsibilities on interdependent components or features. That significantly extends the nature of models by adding a contractual dimension: models are not only needed to support collaborations, they are also required to materialize commitments.

From an agile perspective, such new dimension is critical because it puts a limit to shared ownership and replaces individual responsibilities with organizational (i.e collective) ones. Moreover, contrary to architectural dependencies which can be clearly expressed at components (i.e code) level, organizational ones usually appear well before, and cut across basic agile development cycles. Introducing models to relay such dependencies between projects will clearly contradict core agile principles.

Things would be different if organizational dependencies were matched to architectural ones, and that is precisely what services oriented architectures are meant to achieve:

• Enterprise architects are introduced as a new category of users.
• Objectives addressing business and architectures at enterprise level can be expressed as corporate stories (aka epics) and organized into portfolios.
• From the portfolio perspective business value is much more than the final users value and takes into account the full range of costs and returns.
• Services can be organized into backlogs and then developed and released according standard agile practices.

Along that perspective, there would be two types of models, a white-box one for development artifacts, a black-box one for architectures.

Agile and Model Driven Development

Shared ownership and the define/build/accept cycle are arguably at the core of agile development, which translates into actual continuity between users requirements and code. In other words, even if models are not explicitly excluded, it seems there isn’t much room for them, the reason being the nature of iterative development.

Agile iterations are not just about increments but also about test driven development and integration between requirements and design. In other words the agile cycle is based on a feedback loop between what the system is expected to do and how it may do it. One step further, and architectures may even emerge from iterations. With designs and even architectures being reconsidered all along, there is not much use for models until things are stabilized, and then it’s time for code.

Yet, shared ownership and emerging architectures are not always possible and models may be necessary when organizational units have to share architecture assets. That set agile and model driven approaches at different levels: the former for development teams, the latter for development processes, with models managed by epics and referenced by stories.

 

Further Reading

• Agile between Space & Time
• The Scope of Agile Principles
• Agile vs Waterfall: Right vs Left Brain ?
• Spaces, Paths, Paces (Part 1)
• Spaces, Paths, Paces (Part 2)
• Tests in Driving seats
• Projects as non-zero sum games

External Links

• Scaled Agile Framework