Objective
Contracts are as much about collaboration and trust as they are about protection. While projects are not necessarily governed by contracts, all entail some compact between participants. From in-house developments with shared ownership on one hand, to fixed price outsourcing on the other hand, success will depend on collaboration and a community of purpose.
Hence, a first objective should be to prevent prejudiced assumptions and defensive behaviors. Then, taking a cue from Game Theory, compacts should be designed to enhance collaboration between parties by establishing a non-zero sum playground where one’s benefits don’t have to come from others’ losses.
Players and Stakes
Basic participants of engineering projects can be put in two groups, possibly with subdivisions: business units expressing requirements, and engineering units meant to provide solutions.
The former (user, customers, or stakeholder) specify the needs, pay for the outcome, and expect to benefit from its use. Their success is measured by return on investment (ROI) and depends on cost, quality, and timely delivery.
The latter design the solution, develop the components, and integrate them into target environments. Narrowly defined, their success is measured by costs.
Hence, while stakes may clearly conflict on prices, there is room for collaboration on features, quality and timing, and that may bring benefits to both customers and providers.
Communication and Collaboration: Modeling Languages
Understanding is a prerequisite to collaboration, and except for in-house developments, there is no reason to expect immediate (aka mediation free) understanding between business and engineering units. Some common language is therefore required if heterogeneous concerns are to be matched.
While in-house developments under shared ownership can proceed with domain specific languages, collaboration between different organizational units along time can only be supported by modeling languages with non specific semantics.
Assessments and Strategies: Requirements Metrics
If participants are to act rationally based on symmetric information, they must agree on some measurements for size and complexity.
Given that collaboration is not to offset interests by nature different or even conflicting, participants’ strategies must be based on shared and reliable information regarding their respective stakes. Moreover, if timing is to matter, projects’ metrics will have to be reassessed periodically, and strategies redefined accordingly.
Despite clear shortcomings about targets and accuracy, requirements metrics are nonetheless a necessary component of any collaborative framework, with or without explicit contracts.
First of all, some measurements are required to set schedules and plan resources. Even faulty, those estimators will serve their purpose if agreed upon by all participants, with possible biases redressed by consent, and lack of accuracy reduced or circumscribed progressively.
Then, as engineering projects often come with intrinsic uncertainties, they may go off-trail. Providing timely and reliable indicators, participants should be able to anticipate hurdles, reassess strategies, and amend agreements.
Finally, metrics should make room for arbitrage and differentiated strategies; given trustworthy information and clear acceptance criteria, participants would be in a position to weight their respective stakes with risks, rearrange their priorities, and plan their endgames appropriately.
Prices and Acceptance: Contracts
While trustworthy information is a necessary condition for cooperation, it’s not a sufficient one, as players may be tempted to cheat or defect if they think they can get over with it. Leaving out ethical considerations and invasive regulations, the best policy should be to set rewards and sanctions such as to make a convincing case for cooperation benefits.
That can be achieved if participants can opt for differentiated and non conflicting strategies set according model layers.
Starting with business processes (aka computation independent models) and expected return on investment, customers consider system functionalities (a) and ask providers for feasibility (b), and schedules (c).
Given a functional architecture (aka platform independent models), participants may define strategies: customers associate features with date and expected benefits, providers plan resources and deliveries for products (aka platform specific models and code).
Commitments, by customers on prices and providers on dates, are made at two levels: functional architectures (d) and features (e).
As for any iterative process, changes must be explicitly circumscribed by invariants, e.g functional architecture of supporting systems. Yet, within those constrains, there may be room for adjustments insofar that providers’ costs and customers’ returns are contingent on schedules.
Processes: Time, Risks, Responsibilities and Milestones
As Einstein put it, “The only reason for time is so that everything doesn’t happen at once”. That seems a perfect fit to project planning, as illustrated by two archetypal project configuration:
- At one extreme, in-house projects with shared ownership may operate within their own time wraps (aka time-boxes) because decisions by customers and providers can be taken jointly and simultaneously independently of external factors.
- At the other extreme, outsourced projects are set along sequenced milestones and governed by different organizational units, each with its own time scale. With tasks performed independently and decisions made separately, time becomes a factor and processes are introduced to consolidate time-scales and manage risks and decisions alongside.
More generally, engineering processes are introduced when decisions are to be made along time by different organizational entities. Assuming that time is governed by changes and the decisions to be made thereof, the sequencing of tasks should be defined with regard to the nature of events and decision-making:
Technical: engineering processes come with their own intrinsic constraints regarding the sequencing of operations. When those operations can be performed independently of contexts the whole sequence can be seen as a single step wrapped into a time fold. Otherwise they must be associated with distinct process steps.
Contractual: while uncertainties about contexts may affect both expectations and commitment, decisions have to be made notwithstanding. But decisions carry risks and should therefore be factored out, with rationale and responsibilities stated explicitly. While some responsibilities may be shared, decisions about contexts should remain the sole responsibility of the participants directly in charge and appear as milestones set in contracts.
Managed: whatever the number of model layers and automated transformations, engineering processes set about and wind up with human activities. Given that resources and skills are by nature limited, participants have to make the most of their use. But once milestones are set and technical constraints identified, each participant should be able to optimize its own planning.
That makes for three basic project configurations:
- Requirements analysis is about business requirements and feasibility. Its main objective is to identify technical and contractual milestones.
- Use case developments are self-contained projects under shared ownership, bounded by contractual milestones. They come with defined prices and schedules whose mix can be managed by consent.
- System functionalities are set by cross-cutting objectives and their development is therefore governed by contractual milestones and schedules.
Managing engineering projects along these principles would greatly improve their integration with business needs on one hand, enterprise architecture on the other hand.
Further Reading
- Agile and Phased approaches
- Spaces, Paths, Paces (Part 2)
- Work Units
- Products, Projects, Processes
- Project Management
- Quality
- Contract Driven Modeling
- Risks
- Metrics
- Business Agility vs Systems Entropy
- EA: Entropy Antidote
Dan’s question is pivotal, but I wasn’t attacking DSL. My point is that functional architectures (system functionalities used across domains) should be expressed with some “unified” modeling language. Otherwise (e.g self-contained projects with shared ownership) DSL are clearly the best option.
Although Steven misinterpreted Remy’s statement about “… non specific semantics ..” as being in some way related to agile development, the statement is an unsupported attack on DSLs. I’m interested in what others have to say in support of Remy’s claim. I have recently come to see DSLs as a natural result of RE and relentless domain separation. I see DSLs as a way to maximize communication with the domain experts and still have a pathway to the Turing machine implementation. Why is it necessary for to have non-specific semantics? Remy, let’s see if others will help you clarify your assertion.
Steven,
You say “If shared ownership cannot be established, either the project is not important enough to invest resources in at all or we must slice the project thin enough so that shared ownership can be established for each slice as we progress.”
Believe me, things are seldom so simple, are projects are not always sliced as easily as Salami.
Of course, we use language when we develop software, and not just computer languages.
However, the customer never understands the technical language necessary to describe designs to the depth that the development team does. Domain-driven development shows how to talk the customer’s language instead of our own in order to discuss the basic problem.
This helps but does not go far enough. We may understand the problem together, but the customer still does not really understand our solution until they actually experience it.
User stories and acceptance test-driven development allow us to continue to collaborate with the customer in the customer’s language. But we need to deliver working software frequently to make sure that the software really fulfills the customer’s needs not just fulfills the “contract” established by the acceptance tests.
The longer we go without the concrete feedback of the customer actually experiencing a slice of the working software that represents how we would solve their problem, the more layers of complexity that get added to the assumption that they understand what we will be delivering. This not only increases the risk of delivering a solution that will not fly, it decreases our ability to apply what we have learned at time T in the project to what we deliver after time T because the bulk of the design was created at the beginning.
The Agile experience has been that we can deliver partial ROI quickly and learn what the real problems and expectations are so we can resolve them before having invested months in design. This is much more satisfying than what I experienced in the 90s where we built a big “house of cards” before ever really seeing if we got anything wrong.
I am certainly not against spending a week or two up front establishing a shared vision with all the stakeholders, but then it only makes sense to go into detail on what we can deliver before those details change, especially in the chaotic environment where shared ownership supposedly cannot possibly work.
If shared ownership cannot be established, either the project is not important enough to invest resources in at all or we must slice the project thin enough so that shared ownership can be established for each slice as we progress.
Steven, you argue that the whole idea of using language to describe something rather than simply pointing at it is a waste of time. You contend that it will never be possible to describe working software; we will only ever be able to point at it. You might be right. Remy has another blog post about how humanity has overcome such limitations in the past. You might also read Steven Pinker’s “The Language Instinct” if you are interested seeing the possibilities. However, if you are interested in developing the means to describe software before it exists, you probably have some useful contributions to the topic.
This post from Matthew Caine may bring some arguments to this discussion.
http://www.mcpa.biz/2011/10/money-is-for-nothing-change-is-for-free/
Steven:
AA It is seen / known that the stakeholders do not have a reliable full description of a new system they need at the beginning. So, I agree that developing a detailed SRS is not feasible or useful.
BB To my understanding Use Case Modeling (Use Case Diagram + Description of each Use Case) is relatively better because it starts with and builds on known business needs.
CC You stated that “Use cases prematurely commit to details during the time in the project when there is the least knowledge about the problem and its solution”. It is true that the “solution” is NOT known BUT the “business problems and needs” ARE known.
DD To my understanding “each Use Case with its Goal” is the thinnest slice one can start with. Once the big picture (Use Case Diagram) is arrived at, Use Cases can be sequentially / iteratively evolved WITHOUT getting into design (means of meeting them). Here I pursue development of ONLY the Use Case Dialog BUT NOT the Sequence Diagrams.
EE To me it appears that in the Agile approach the slicing can be arbitrarily thin and building of a desired system slices can be done without knowing other requirement slices. Furthermore, it appears that the system slices built in any order will synthesize into a whole system (that is too much to hope for). I know there are a lot of professionals committed to this way of working but I want to know if there is any principle supporting this approach to economically develop systems reliably.
Please provide pointers to reliable publications / examples on EE. Let me know (separately to kenablersys@yahoo.com) if there is any logical / practical gap or flaw in DD.
@ Steven,
Have you ever try to organize commitees along time across large organizations ? Have you any idea of how contexts, people and perspectives can change during intervals ?
Why do we really need do all that bureaucracy?
If we just break the project into vertical slices than can be done in a week or two and just execute involving whoever cares enough to participate then we can actually get something done and see how it flies.
Then we can repeat until done. If something changes between iteration N and iteration N+1, we deal with it then instead of holding meetings to guess what will happen in the future and then execute a big long project based on those guesses.
If nobody will participate in doing something now, then the project must not be very important, so shelve the project until the stakeholders will participate.
Herding cats for months at a time does not work, so slice the project thin enough that you only need to herd cats 2 weeks at a time.
Reblogged this on lava kafle kathmandu nepal.
In my first comment, I identified the first point where your words turned from a generality that encompasses both the old and new worlds to an out of the blue presumption that severely compromises any hope of agility.
“While in-house developments under shared ownership can proceed with domain specific languages, collaboration between different organizational units along time can only be supported by modeling languages with non specific semantics.”
Why? Because you say so?
Why does collaboration between different organization units over time prohibit real-time face-to-face collaboration on a thin functional slice of the single highest priority feature in order to establish a working agreement, discover the real requirements, and settle on an architectural skeleton?
What about the organization gives us no choice but to schedule everything before we start, which then requires us to fully elicit the requirements and model them all up front?
Wouldn’t it be more productive in the long run to fix the problems in the organization that paint us into this BDUF corner?
Conway’s law – the structure of the software will inevitably reflect the structure of the organization which develops it – makes it clear that the software will be poorly structured if we do not fix the impediments to doing the software the best way we know how.
You just paste your prejudices on my words.
I’m aware that some projects succceed without models, and others fail despite models.
Besides, legacy systems should not be confused with functional architectures.
I used legacy to refer to archaic organizational structures rather than archaic code bases. But, both have to be refactored to address blockages in the value flow.
Your writings show a distinct preference to accept archaic organizational structures as a given instead of refactoring them where needed (and working around them while slowly refactoring them in the background). Continuing to accept those impediments and use contracts, overspecialized resources, and multiple conflicting responsibilities with a lack of clear priorities as the reason to advocate continuing to do BDUF does not help us become more agile.
It may seem to be the best way in the short run, but the long run never comes if we continue to just accept those impediments and let them keep us from achieving agility.
Things are relatively non-controversial until a leap is made in the paragraph that reads :
“While in-house developments under shared ownership can proceed with domain specific languages, collaboration between different organizational units along time can only be supported by modeling languages with non specific semantics.”
I do not see how this statement can be made without significant justification.
In practice, I have found that most stakeholders think they understand modeling languages once they gain the most superficial understanding. Subtleties and nuances that have big implications for the system design are very often accepted without true understanding, leading to poor systems or massive rework. For example, cardinalities on relationships are rarely understood correctly by business people, or even so called domain experts.
Since discovering Agile, I have found feedback on working software that implements very thin slivers of end-to-end functionality (for even a single scenario) establishes much more reliable communication than the detailed models required to establish the same level of agreement.
Use cases prematurely commit to details during the time in the project when there is the least knowledge about the problem and its solution. Using the legacy of unwieldy enterprises and rigid contracts as an excuse to continue doing BDUF.
Fix the legacy instead. If you really want to get projects done quickly and correctly, deploy “skunk works teams” with the right mixture of skills to get projects done incrementally driven by real-time feedback from the people who have the problem instead of being driven by preconceived requirements and timelines. If protecting the legacy structure of the organization is more important than efficiently delivering the value the organization needs to compete in today’s world, then I want no part of it.