Synopsis
Business driven development is arguably the cornerstone of the agile development model. On one side it means business value set by users and stakeholders, on the other side it entails continuous and just-in-time delivery; what happens in-between is set by backlogs (for development) and product increments (for delivery).
A sanguine understanding of continuous releases may assume that planning is no longer relevant, and that deployment can be carried out “on-the-fly”. But that would assume that stakeholders and product owners are ready to put aside roadmaps, overlook milestones, and more generally forget that time is money. That would go against a basic objective of agile, namely that developments must be driven by business needs, and products delivered just-in-time for best value. Given the well established track record of dynamic programming for manufacturing processes, could the technique be usefully applied to agile engineering processes ?
Delivery, Deployment, & Continuity
Continuity doesn’t mean synchronization: business, engineering, and operations are governed by different concerns set along different time-frames. Some buffering is needed, materialized by the distinction between releases (engineering concerns) and deployment (business and operational concerns).
Such distinctions introduce both overlapping (with business time-frame) and discontinuity (between development and deployment):
- Product roadmaps are set in business time-frames and determine development and deployment time-frames.
- Development time is set by product roadmaps and runs clockwise from project inception to software releases (a>b>c).
- Deployment time is also set relative to product roadmaps but it runs counterclockwise, from product deployment as planned by business back to released software components (c<e).
Development and deployments runs can be compared to crews tunneling through a mountain from both sides; where and when they meet leaves room to adjustments. Yet more is at stake in the meeting between development completion and deployment inception. Apart from time, adjustments may also bear on formats and contents; and given the specificity of development and deployment purposes, their adjustment may also be seen as the morphing of continuous software releases (project perspective) into discrete increments (product perspective).
Dynamic Programming
Dynamic programming (aka multistage programming) is a problem solving method that combines two principles:
- Divide & conquer is a general purpose strategy that deals with the intrinsic complexity of problems by breaking them down into collections of simpler sub-problems to be solved separately depending on sequencing constraints.
- Recursion deals with the lack of complete or accurate information upfront, solving the problem in stages rather than as one entity. Each stage is optimized separately on the basis of the current state reflecting decisions taken at previous stages, the optimality principle guaranteeing the optimality of the final outcome.
That incremental and iterative approach clearly befits the tenets of the agile development model.
Twofold Planning
As noted above, and whatever the technique, agile processes entwine two phases, one for development, the other for deployment, each with its own planning:
- The development planning, epitomized by backlog management, deals with the definition of work units and their sequencing.
- The deployment planning deals with the merging of software released into product and their incremental deployment.
That makes for multiple dynamics, first for updated backlogs, then for updated deployment targets, and finally for possible feedback through their nexus. That can only be achieved with dynamic programming.
Backlog: Multistage sequencing
Backlogs are used to manage work units targeting self-contained requirements items; they can be represented by graphs with nodes standing for work units and arcs weighted by constraints.
Basically, the problem is to optimize the development of a given set of items given users’ priorities, technical constraints, and resources availability. When all the information is available upfront, optimum solutions can be obtained with simple Shortest Path algorithms. Yet, given the iterative and exploratory nature of agile processes, backlogs are meant to be updated as the project advances, taking advantage of improved knowledge:
- Users may introduce new items, remove ones, or change their priorities due to a better understanding of requirements space.
- Engineers may also introduce new items (e.g for technical debt) or reconsider technical difficulties and dependencies due to a better understanding of solutions space.
Dynamic programming is introduced in order to support step-wise decisions optimizing the whole process:
- Backlog states (t1 & t2) are defined by the remaining work units, rankings, and feasibility constraints.
- Each stage redefines the optimum path to completion taking into account the current state and updated information. Recursive computation being based on the summary information etched in states, it ensues that all future decisions can be selected optimally without recourse to information regarding previously made decisions.
Given a set of feasible paths (as defined by technical dependencies and time), the aim at each stage is to select the optimum path for the remaining units based on current state. Optimization functions will typically consider users’ value, learning curve and associated risks, and resources availability and costs.
As illustrated below, nodes can represent grouped items, e.g when several projects have to share resources or releases are to be regrouped.
Deployment: Dynamic merging
Given a set of released software components, the aim of deployment planning is to decide which increment to add to deployed product. Assuming that technical concerns have already been dealt with by releases, the objective at each stage is select the items maximizing the ROI of the deployed product. It must be noted that, contrary to the development algorithm looking forward for the optimization, the deployment algorithm select the optimum path by looking backward at the ROI of deployed products.
But the backward impact of deployment optimization can go further and affect backlogs.
Feedback
Shared ownership and continuous delivery are two main pillars of the agile development model, the former giving the development full authority and responsibility, the latter ensuring the users have a firm hand on the helm. Yet, as already noted, development, deployment, and business are governed by different time-frames, which could induce some frictions, e.g if business units were forced to synchronize products deployments with software releases. While severe disruptions can be avoided if releases and deployments are managed separately, development teams cannot be completely sheltered from changes in business or operational priorities. That is where the dynamic reassessment of optimum paths is to help: assuming a change in deployment planning (nq instead of op), the new priorities are fed back into development (aka backlog) rankings, and the optimum path is updated.
It must be noted that such feedback only affects ranks and must leaves contents unchanged.
Conclusion: Business driven, Just-in-time delivery, & Lean Inventories
Dynamic programming appears as a primary factor with regard to three core tenets of the agile development model:
- Business driven development doesn’t mean that developments are pushed by requirements but that they are pulled by deployment.
- Just-in-time delivery can only be achieved with the help of a buffer between development and deployment. This buffer should not be confused with an inventory as it has nothing to do with product quantities.
- On the contrary, this buffer, combined with dynamic programming, plays a critical role in the cutback of intermediate documents and models (aka development inventories).
Further Readings
- Thread: Agile
- Agile between Space & Time
- The Scope of Agile Principles
- Agile vs Waterfall: Right vs Left Brain ?
- Agile & Models
- Thinking about Practices
- Spaces, Paths, Paces (Part 1)
- Spaces, Paths, Paces (Part 2)
- Tests in Driving seats
Thanks for the suggestion, but is that not supposed to be managed by the WordPress theme or the browser ?
JUst a suggestion – why not make your blog more readable by increasing the real-estate used for text? You are using 1/3rd of a HD screen