Conversations across software engineering forums sometimes reveal unexpected views, as it’s the case for the benefits of accountability.
One would assume that competition would impel enterprises to scrutiny with regard to resources employed and product outcomes, pushing for the assessment of internal activities based on some agreed metrics. And yet, now and again, software development is viewed as a boutique occupation, if not an art pursuit, carried out by creative craftsmen for enlightened if demanding patrons; a vocation too distinctive to be gauged by common yardsticks.
Difficulties of Oversight
Setting apart creative delusions, the assessment of software development is effectively confronted with rational as well as practical obstacles.
To begin with rationality, and unlike traditional products, there is no market pricing mechanism that could match software development costs with customers’ value. As a consequence business stakeholders and systems engineers prefer to play safe and keep their respective assessments on the opposed banks of the customer/provider divide.
As for the practicality of assessments, the choice is between idiosyncratic approaches (e.g users’ points) and reasoned ones (essentially function points). The former ones being by nature specific and subject to changes in business opportunities, whereas the latter ones are being plagued by implementation plights that make them both costly and unreliable.
Yet, the diluting of IT systems in business environments is making that conundrum irrelevant: the fusing of business processes and supporting software is blanketing the discontinuities between business value and development costs.
Perils of Oversight
Given the digital integration between systems and business environments and the part played by software in production, marketing and operations, enterprises can no longer ignore the economics of software development.
As far as enterprises are concerned, economics use prices for two key purposes, external and internal.
With regard to their business environment, enterprises need metrics to price the resources they could buy and the products they could sell; their competitive edge fully depends on the thoroughness and accuracy of both.
With regard to their internal governance, enterprises need metrics to gauge the efficiency of their factors and the maturity of their processes, and allocate resources accordingly. That internal assessment is the basis of their versatility and plasticity:
Confronted to continuous, frequent, and often abrupt changes in business environments, enterprise must be able to adapt their activities without having to change its architectures. That cannot be achieved without timely and accurate assessments of the way their resources are put to use.
Conversely, enterprises may have to change their architectures without affecting their performances; that cannot be achieved without a comprehensive and accurate assessments of alternative options, organizational as well as technical.
To summarize, the spread and intricacy of software footprint over both sides of the crumbling fences between enterprise systems and business environments makes software economics a necessary component of enterprises governance, so a tally of software beans should not be an option.
Contrary to some unfortunate misconceptions, measurements are as much about collaboration and trust as they are about rewards or sanctions. By shedding light on objectives and pitfalls they prevent prejudiced assumptions and defensive behaviors; by setting explicit quality and acceptance criteria they help to align respective expectations and commitments.
Since projects begin with requirements, decisions about targeted functionalities and resources commitments are necessarily based upon estimations made at inception. Yet at such an early stage very little is known about the size and complexity of the components to be developed. Nonetheless, quality planning all along the engineered process is to be seriously undermined if not grounded in requirements as expressed by stakeholders and users.
Business vs Functional Complexity
Contracts without transparency are worthless, and the first objective is therefore to track down complexity across enterprise architecture and business processes. For that purpose one should distinguish between business and application domains, business processes, and use cases, which describe how system functionalities support business processes.
Based upon intrinsic metrics computed at domain level, functional metrics are introduced to measure system functionalities supporting business processes and compare alternatives solutions. Finally requirements metrics should also provide a yardstick to assess development models.
Business Requirements Metrics
The first step is to assess the intrinsic size and complexity of business domains and processes independently of system functionalities, and that can be done according symbolic representations:
The footprint includes artifacts for symbolic objects and activities, partitions (objects classifications or activities variants). Symbolic objects and activities are qualified as primary or secondary depending on their identification. The reliability of the footprint is weighted by the explicit qualification of artifacts as primary or secondary.
Symbolic representations for objects and activities are associated with features (attributes or operations) defined within semantic domains.
A part from absolute measurements a number of basic ratios can be computed for anchors (primary objects and activities) and associated partitions.
A robust appraisal of the completeness and maturity of requirements can be derived from:
Percentage of artifacts with undefined identification mechanism.
Percentage of partitions with undefined characteristics (exclusivity, changeability).
The organization and structure of requirements can be estimated by:
Average number of artifacts and partitions by domain (a).
Total number of secondary objects and activities relative to primary ones.
Average and maximum depth of secondary identification.
Total number of primary activities relative to primary objects
Total number of features (attributes and operations) relative to number of artifacts.
Ratio of local features (defined at artifact level) relative to shared (defined at domain level) ones.
Finally intrinsic complexity can be objectively assessed using partitions:
Total number of activity variants relative to object classifications.
Total number of exclusive partitions relative to primary artifacts, respectively for objects and activities.
Percentage of activity variants combined with object classifications.
Average and maximum depth of cross partitions.
It must be noted that whereas those ratios do not depend of any modeling method, they can nonetheless be used to assess requirements or refactor them according specific methods, patterns, or practices.
Functional Requirements Metrics
Functional metrics target the support systems are meant to bring to business processes:
The footprint of supporting functionalities is marked out by roles to be supported, active physical objects to be interfaced, events to be managed, and processes to be executed. As for symbolic representations, corresponding artifacts are to be qualified as primary or secondary depending on their identification, with accuracy and reliability of metrics weighted by the completeness of qualifications.
Functional artifacts (objects, processes, events, and roles) are associated with anchors and features (attributes or operations) defined by business requirements.
Given that use cases are meant to focus on interactions between systems and contexts, they should provide the best basis for functional metrics:
Interactions with users, identified by primary roles and weighted by activities and flows (a).
Access to business (aka persistent) objects, weighted by complexity and features (b).
Control of execution, weighted by variants and couplings (c).
Processing of objects, weighted by variants and features (d).
Processing of actual (analog) events, weighted by features (e).
Processing of physical objects, weighted by features (f).
Additional adjustments are to be considered for distributed locations and synchronous execution.
Use Cases metrics can also provide a basis for quality checks:
Average and maximum number of root use cases relative to business and application domains.
Average and maximum number of root use cases relative to primary activities.
Average and maximum number of secondary use cases relative to root ones.
Average number of primary (for identified) and secondary roles relative to root use cases.
Average number primary (aka external) and secondary (issued by use cases) events relative to root use cases.
Average and maximum number of primary objects relative to use cases.
The number of variants supported by a use case relative to the total associated with its footprint, respectively for persistent and transient objects.
Requirements Metrics, Contracts, and Acceptance Criteria
As noted above, requirements metrics are a means to a dual end, namely to set a price on developments and define corresponding acceptance criteria. Moreover, as far are business is concerned, change is the rule of the game. Hence, if many projects can be set on detailed and stable requirements, projects with overlapping concerns and outlying horizons will usually entail changing contexts and priorities. While the former can be contracted out for fixed prices, the latter should clearly allow some room for manoeuvre, and requirements metrics can help to manage that room.
Assuming that projects are initiated from clearly identified business domains or processes, requirements capture should be set against a preliminary wire-frame referencing new or existing pivotal elements of business (1) and system (2) contexts. Those wire-frames (aka anchors, aka backbones) will be used both for circumscribing envelops and managing changes.
Envelops should be defined along architecture layers: enterprise for business objectives (1), functional for supporting systems (2), and technical for deployment platforms (3). That will set budgets outer limits for given architectural contexts.
Moves within rooms (4) are governed by functional requirements and anchored to pivotal business objects or processed (wire-frame’s nodes) . Their estimations should take into account analogies with previous developments.
Acceptance criteria could then be defined both for invariants (changes are not to overstep architectural constraints) and increments (added functionalities are properly implemented).
Requirements Metrics and Agile
While agile development models are meant to be driven by business value, project assessment essentially relies on informed guesses about users stories. That let two questions unanswered:
Given that the business value of a story is often defined at process level, how to align it with its local assessment by project team.
If problem spaces and solution paths are to be explored iteratively, how to reassess dynamically the stories in backlog.
Answers to both questions are to be helped if requirements are qualified with regard to their nature and footprint.
Informed decision making about what to consider and when clearly depend on the nature of stakes. Hence the importance of a reasoned requirements taxonomy setting apart business requirements, system functionalities, quality of service, and technical constraints on platform implementations.
Dynamic assessment and ranking of backlog elements require some hierarchy and modularity:
Architectural options, functional or technical, must be weighted by supported business features and quality of services.
Dynamic ranking of users’ stories implies that their value can be mapped to features metrics at the corresponding level of granularity.
Finally, requirements metrics may also be used to design quality checks to downstream models. With metrics for intrinsic characteristics of business domains on one hand, system functionalities on the other hand, subsequent models can be assessed for consistency, and alternatives solutions compared.