Requirements Metrics Matter

Objectives

Contrary to some unfortunate misconceptions, measurements are as much about collaboration and trust as they are about rewards or sanctions. By shedding light on objectives and pitfalls they prevent prejudiced assumptions and defensive behaviors;  by setting explicit quality and acceptance criteria they help to align respective expectations and commitments.

Since projects begin with requirements, decisions about targeted functionalities and resources commitments are necessarily based upon estimations made at inception. Yet at such an early stage very little is known about the size and complexity of the components to be developed. Nonetheless, quality planning all along the engineered process is to be seriously undermined if not grounded in requirements as expressed by stakeholders and users.

Business vs Functional Complexity

Contracts without transparency are worthless, and the first objective is therefore to track down complexity across enterprise architecture and business processes. For that purpose one should distinguish between business and application domains, business processes, and use cases, which describe how system functionalities support business processes.

Based upon intrinsic metrics computed at domain level, functional metrics are introduced to measure system functionalities supporting business processes and compare alternatives solutions. Finally requirements metrics should also provide a yardstick to assess development models.

Business Requirements Metrics

The first step is to assess the intrinsic size and complexity of business domains and processes independently of system functionalities, and that can be done according symbolic representations:

• The footprint includes artifacts for symbolic objects and activities, partitions (objects classifications or activities variants). Symbolic objects and activities are qualified as primary or secondary depending on their identification. The reliability of the footprint is weighted by the explicit qualification of artifacts as primary or secondary.
• Symbolic representations for objects and activities are associated with features (attributes or operations) defined within semantic domains.

A part from absolute measurements a number of basic ratios can be computed for anchors (primary objects and activities) and associated partitions.

A robust appraisal of the completeness and maturity of requirements can be derived from:

• Percentage of artifacts with undefined identification mechanism.
• Percentage of partitions with undefined characteristics (exclusivity, changeability).

The organization and structure of requirements can be estimated by:

• Average number of artifacts and partitions by domain (a).
• Total number of secondary objects and activities relative to primary ones.
• Average and maximum depth of secondary identification.
• Total number of primary activities relative to primary objects
• Total number of features (attributes and operations) relative to number of artifacts.
• Ratio of local features (defined at artifact level) relative to shared (defined at domain level) ones.

Finally intrinsic complexity can be objectively assessed using partitions:

• Total number of activity variants relative to object classifications.
• Total number of exclusive partitions relative to primary artifacts, respectively for objects and activities.
• Percentage of activity variants combined with object classifications.
• Average and maximum depth of cross partitions.

It must be noted that whereas those ratios do not depend of any modeling method, they can nonetheless be used to assess requirements or refactor them according specific methods, patterns, or practices.

Functional Requirements Metrics

Functional metrics target the support systems are meant to bring to business processes:

• The footprint of supporting functionalities is marked out by roles to be supported, active physical objects to be interfaced, events to be managed, and processes to be executed. As for symbolic representations, corresponding artifacts are to be qualified as primary or secondary depending on their identification, with accuracy and reliability of metrics weighted by the completeness of qualifications.
• Functional artifacts (objects, processes, events, and roles) are associated with anchors and features (attributes or operations) defined by business requirements.

Given that use cases are meant to focus on interactions between systems and contexts, they should provide the best basis for functional metrics:

• Interactions with users, identified by primary roles and weighted by activities and flows (a).
• Access to business (aka persistent) objects, weighted by complexity and features (b).
• Control of execution, weighted by variants and couplings (c).
• Processing of objects, weighted by variants and features (d).
• Processing of actual (analog) events, weighted by features (e).
• Processing of physical objects, weighted by features (f).

Additional adjustments are to be considered for distributed locations and synchronous execution.

Use Cases metrics can also provide a basis for quality checks:

• Average and maximum number of root use cases relative to business and application domains.
• Average and maximum number of root use cases relative to primary activities.
• Average and maximum number of secondary use cases relative to root ones.
• Average number of primary (for identified) and secondary roles relative to root use cases.
• Average number primary (aka external) and secondary (issued by use cases) events relative to root use cases.
• Average and maximum number of primary objects relative to  use cases.
• The number of variants supported by a use case relative to the total associated with its footprint, respectively for persistent and transient objects.

Requirements Metrics, Contracts, and Acceptance Criteria

As noted above, requirements metrics are a means to a dual end, namely to set a price on developments and define corresponding acceptance criteria.  Moreover, as far are business is concerned, change is the rule of the game. Hence, if many projects can be set on detailed and stable requirements, projects with overlapping concerns and outlying horizons will usually entail changing contexts and priorities. While the former can be contracted out for fixed prices, the latter should clearly allow some room for manoeuvre, and requirements metrics can help to manage that room.

1. Assuming that projects are initiated from clearly identified business domains or processes, requirements capture should be set against a preliminary wire-frame referencing new or existing pivotal elements of business (1) and system (2) contexts. Those wire-frames (aka anchors, aka backbones) will be used both for circumscribing envelops and managing changes.
2. Envelops should be defined along architecture layers:  enterprise for business objectives (1), functional for supporting systems (2), and technical for deployment platforms (3). That will set budgets outer limits for given architectural contexts.
3. Moves within rooms (4) are governed by functional requirements and anchored to pivotal business objects or processed (wire-frame’s nodes) . Their estimations should take into account analogies with previous developments.
4. Acceptance criteria could then be defined both for invariants (changes are not to overstep architectural constraints) and increments (added functionalities are properly implemented).

Requirements Metrics and Agile

While agile development models are meant to be driven by business value, project assessment essentially relies on informed guesses about users stories. That let two questions unanswered:

• Given that the business value of a story is often defined at process level, how to align it with its local assessment by project team.
• If problem spaces and solution paths are to be explored iteratively, how to reassess dynamically the stories in backlog.

Answers to both questions are to be helped if requirements are qualified with regard to their nature and footprint.

Informed decision making about what to consider and when clearly depend on the nature of stakes. Hence the importance of a reasoned requirements taxonomy setting apart business requirements, system functionalities, quality of service, and technical constraints on platform implementations.

Dynamic assessment and ranking of backlog elements require some hierarchy and modularity:

• Architectural options, functional or technical, must be weighted by supported business features and quality of services.
• Dynamic ranking of users’ stories implies that their value can be mapped to features metrics at the corresponding level of granularity.

That can be achieved with functional features assessed along architecture layers.

Requirements Metrics and Quality Planning

Finally, requirements metrics may also be used to design quality checks to downstream models. With metrics for intrinsic characteristics of business domains on one hand, system functionalities on the other hand, subsequent models can be assessed for consistency, and alternatives solutions compared.

Providing unified OO modeling concepts, metrics can be computed on analysis and design models and set against their counterpart for original requirements. Examples of standard metrics for UML models include:

• The number of root packages models is to be compared to symbolic containers for business and application domains, and the sub packages to primary (#) objects or activities.
• The number of classes in models and packages is to be
• The number of actors is to be set against the number of roles, with primary (#) roles standing for identified agents.
• The number of root use cases is supposed to coincide with primary (#) activities and roles.
• The number of structural inheritance hierarchies should be compatible with the number of frozen and exclusive partitions respectively for objects and activities.

Further Reading

• About Measurements
• Functional Metrics
• From Requirements to Function points
• CMMI

External Links

• Use Case Points
• COSMIC