Objective
As previously noted, embarking for sizable and lengthy project on the assumption that detailed scope and schedules can be set upfront is a very hazardous policy. Alternatively, agile development models carry out specifications, development, and quality assurance into integrated iterations, making room for a progressive exploration of problem spaces and solution paths, and consequently for informed decision-making and better risk management.
Yet, whatever the method, at some point, scope and features will have to be committed; and with targeted features set dynamically, planned schedules are no more an option. Hence the need to reconsider the way time is taken into account.
Dependencies: Playing for Time
Paraphrasing Einstein, one may say that the only reason for processes is so that everything doesn’t happen at once. Why is that ?
First, processes are meant to support informed decisions. If problem spaces and solution paths cannot be settled upfront they must be done progressively. And that will clearly introduce informational dependencies supporting:
- Decisions about what is to be done: alternative paths and their priorities
- Decisions about how it should be done: supported features and their priorities
- Decisions about how it can be done: quality assurance and acceptance.
In that context the objective of development processes is to do as much of definition, building, and acceptance, and to delay decisions in order to gather the most of the relevant information until the “last responsible moment”, i.e without preempting any of the initial set of alternative options.
Assuming informed decisions are supported by architecture knowledge, processes must take into account engineering constraints regarding:
- Technical dependencies associated to the nature of development flows and environments.
- Functional dependencies associated to products functionalities and supporting systems.
- Organizational dependencies associated to business units and localization.
While those dependencies are defined within architecture layers, their impact on the organization of work units may have to be consolidated when projects are carried out across layers.
Schedules: Running for Time
Whether development cycles stay within or run across architecture layers, there will be no way to decide about deliveries and schedules at project inception. In other words dependencies will have to be sorted out and planning settled along the road.
Project planning means consolidating overlapping time-frames. That may not be a problem for projects set within single architecture layers (e.g migration), but that should definitively be taken into account when heterogeneous time-frames govern changes across architecture layers. With regard to changes managed at project level (endogenous events), the consolidation may be done within project time-frame; but that will not be possible for changes occurring in non managed environments (exogenous events).
Those events are set in time-frames governed by their own rationale (e.g business, organization, or technology) that cannot be subsumed into the engineering timetable:
- At enterprise level, time is set by business context. Both business objectives and business process solutions are meant to be decided with regard to actual (aka exogenous) business opportunities (a).
- At system level, time is set by project planning (endogenous events). Given functional requirements (e.g users’ stories or use cases), architects and designers have to decide about the scheduling of systems functionalities and services, and the release of corresponding applications. Since those decisions are not directly exposed to exogenous events, they can be made according to engineering constraints and resources availability (b).
- At platform level, time is set by business and operational objectives (endogenous events), and technical contexts (exogenous events). Assuming that risks are evenly set, the problem is to align endogenous (managed by project) with exogenous (anticipated from contexts) events: too early releases may preclude later but more useful ones; too late releases may hamper operations (c).
Whereas those time-scales are to be synchronized, there is no reason they would be congruent. Hence the need of some mechanism, static (e.g milestones) or dynamic (e.g backlogs) supporting the scheduling of work units.
From Milestones to Backlogs
As considered elsewhere, phased models of development may offer some benefits when dependencies originate from different environments or involve different authorities; yet they may impose still bigger penalties when requirements cannot be fully settled upfront. Conversely, agile approaches are the option of choice when complex problem spaces and solution paths are to be explored and refined progressively; but may prove ineffective in dealing with business, organizational, or technical dependencies set from outside projects. That difference of perspective is reflected by the mechanisms used to manage dependencies: milestones or backlogs.
Sorting out dependencies means consolidating informational and engineering constraints across architecture layers. When problem spaces and solution paths cannot be managed under the same authority (heterogeneous dependencies) phased development models use milestones to consolidate expectations and commitments across layers and time-frames.
Otherwise (homogeneous dependencies) agile development models use backlogs to explore problem spaces and solution paths, whatever the architecture layer and nature of dependencies. With the benefit of shared ownership, business events are propagated all along development paths, governing backlogs of stories (business requirements), features (engineering constraints), and releases (operational requirements).
Along that perspective backlogs and milestones can be seen as mirrored solutions of the same problem, namely how to deal with the functional and timing dimension of dependencies: backlogs deal only with the functional dimension as they consider the dependencies between tasks without taking into account their actual timing; milestones look from the opposite direction, anchoring aggregate tasks to timetables before considering dependencies between items. Depending on the point of view:
- Backlogs may appear as stacks of stones: the milestones are fragmented and the initial sequence translated into queues.
- Milestones may appear as flattened backlogs: dependencies are frozen and stories are ironed out before being merged into heaps.
Hence, with regard to the ranking of dependencies, the difference between backlogs and milestones is of granularity, finer for the former, coarser for the latter. But with regard to execution, the difference is of nature: contrary to milestones, backlogs don’t fix any timing.
That distinction between sequence (functional ranking) and schedule (time ranking) may be critical when iterative development has to be combined with heterogeneous dependencies.
Collaboration Levels
As noted above, the planning of work units must take into account three criteria:
- The dependencies between tasks, engineering or informational, determine the sequencing independently of actual timing. They are derived from technical or organizational constraints.
- The schedules anchor the start and completion of tasks to time-frames. Some time-frames are set within the enterprise (e.g resources availability), others are set from outside (e.g regulations or business opportunities).
- Pace determine the elapsed time taken to complete a task. It may be seen as a metronome used to adjust the throughput depending on resources availability on one hand, quality requirements on the other hand.
At first sight, comparison shows backlogs to outdo milestones on all three accounts: dependencies are managed at finer granularity, which enables dynamic scheduling and releases management; last but not least, due to iteration cycles and time-boxing, development paces can be aligned with resources and quality requirements. Yet, that edge can be misleading if dependencies translate into unmanageable or multiple backlogs.
Collaboration is at the core of iteration cycles, and it can only be achieved through shared and dynamic management of backlogs. Assuming a set of stories, each new cycle has to be preceded by decisions regarding priorities, refinements, and responsibilities; and if informational dependencies have to be taken into account, these decisions must be taken collectively and directly:
- Collectively: decisions about priorities and refinements must be negotiated between team members from both sides of the business/system (or users/developers) divide.
- Directly: all decisions must be taken by the team itself, without any organizational mediation or external interference.
But those conditions could be thwarted if different projects have to develop stories with shared features implemented on different platforms.
In that case consolidation mechanisms could bring back fixed scope/schedule configurations:
- Different teams, each with its own specific users and developers concerns, will have to commit to agreed features and timetables.
- By involving independent organizational units, those decisions will entail some procedural mediation carried out independently of iteration cycles.
Yet, that shouldn’t necessarily be the end of the stories, providing some mechanism could be found to synchronize iteration cycles. And that could be achieved with blackboards.
From Backlogs to Blackboards
Blackboards can be understood as shared backlogs stripped from their ranking mechanism so that items can be consulted and dealt with by different project teams. Alternatively, they can also be seen as timetables stripped from scheduling mechanism. Either way, those views illustrate how blackboards may support shared dependencies without forcing them into time-frames:
- Shared features are posted on a single blackboard where their status can be consulted and updated by the teams in charge of the stories concerned.
- Development teams post their releases on blackboards according to targeted platforms.
As it happens, that approach also answers the recurring question of stories’ nature and granularity: with the benefit of blackboards, fine-grained stories can be indexed as business cases, use cases, or new releases and cohabit in backlogs.
Combining backlogs with blackboards provides a collaboration mechanisms supporting external dependencies without impairing teams ownership of iteration cycles. Yet it is not a substitute for schedules as it doesn’t deal with the alignment of cycles with enterprise time-frames.
Time boxes as Pacemakers
Milestones and backlogs are synchronization mechanisms. The former, combined with timetables, coordinate teams along time-spans; the latter, combined with time boxes, coordinate tasks between cycles. Hence, while both aim at the same objective, namely that everything doesn’t happen at once, their understanding of time is very different: timetables are bound to an external measure of time, time boxes are just arbitrarily fixed intervals that can be tethered to any time-frame.
As a consequence, time boxes can be applied at different levels. At system level they are associated to project backlogs and used to set the tempo of iteration cycles. At enterprise level they are associated to business objectives and anchored to strategic plans. Assuming those plans are described by epics, it would be possible to use different tempos depending on level (enterprise or systems) or even applications.
Ideally, differentiated tempos should foster an emerging harmony between melody (users’ stories) and accompaniment (supporting systems) converging into the fulfillment of strategic objectives. Practically, the alignment of releases with epics cannot be taken for granted.
Squaring the Circles: Project Planning
Agile approaches are based upon the dynamic exploration of problem spaces and the iterative development of solution paths, the objective being to maximize the value of functional requirements under the constraints of technical ones. Yet, with spaces and paths defined dynamically, standard exploration procedures like breath or depth-first traversal are useless because the ranking of paths is part of the solution, which means that priorities must be revised at the end of each cycle.
Nonetheless, explorations cannot be everlasting and there must be some end to the story, when stakeholders get what they expect (or accept what they get) and users can begin to reap the benefits. Fixed scope and schedules being ruled out, the problem is to align iterations outcomes (b) with business objectives (a). Solutions can be positioned between two archetypal approaches, one driven by business objectives, the other by development tasks.
The first approach would see development teams take commitments with regard to broadly defined objectives: with problem spaces represented by trees progressively refined and explored, commitments can be made collectively on sets of solution paths within still undefined sub-trees (b1). The team will then take responsibility for the details of iteration cycles and will adjust its throughput as to align releases with business objectives.
Alternatively, and providing a finer granularity can be obtained and managed, stories could be broken down into tasks for which commitments will be made individually by team members (b2). Assuming that the tasks workloads can be assessed, iterations could then be planned and releases scheduled on the basis of time boxes parameters.
Not surprisingly, the choice of a planning policy is to be conditioned by the granularity of work units:
- Task based planning requires finer grained stories and comes with a phased flavor by reintroducing analysis. That may stretch the intervals between iterations and increase management overheads.
- Objective based planning allows for coarser grained stories and is more in line with agile spirit. Yet that may increase the length of iterations and affect their transparency.
All things considered, some feedback loop may be needed when deciding on the size of stories because shorter ones do not necessarily decrease the time to market as they may generate bigger backlogs and exponential overheads in case of complex dependencies.
Balancing Pushes and Pulls: Lean and Just-in-Time Workflows
When push comes to shove project planning turns into conflict management. That may happen with phased development models as well as with agile ones. With the former that will be due to applications forcibly pulled out whatever their value for users and reliability in order to meet unrealistic expectations. With the latter that will be due to requirements forcibly pushed into backlogs disregarding their size and exponential complexity.
The way out of this dilemma is a feedback mechanism between project teams pushing releases and business stakeholders pulling applications. And that is the rationale behind the Kanban development model:
- Visualize workflow: up-to-date expectations, constraints, commitments and achievements must be clearly and selectively visible to all concerned. that can be done with backlogs and blackboards.
- Limit work in progress: that can be obtained by regulating the selection of solution paths through limits on the size of backlogs and the fragmentation of users’ stories according to their architecture footprint.
- Measure and manage flows: metrics and process design can be significantly improved if flows are differentiated depending on architecture layer (business, systems, platforms).
- Make process policies explicit: that is already a cornerstone of agile backlog management; those principles should also apply to backboards.
- Use models to recognize improvement opportunities: while initially ignored, the benefits of models in agile development is progressively acknowledged. Those benefits have be illustrated in the first part of this article by the use of activity diagrams for the exploration of problems spaces and solutions paths.
Applying those principles will bring about lean processes and just-in-time workflows, improving both users’ value and software quality.
Further Reading
- Use Cases and associated patterns
- Variants, power-types, and rules patterns
- Iterative development
- Agile and Phased approaches
- Agile and Models
- Projects As Non-Zero Sum Games
- Work Units
- Products, Projects, Processes
- Project Management
Reblogged this on lava kafle kathmandu nepal.