Generally speaking, quality may refer to intrinsic properties, functional characteristics, or some external yardstick. With regard to software engineering it would mean code, users experience, and operations, each with its own specific stakeholders and criteria.
On one side, traditional phased approaches to QA are meant to deal with those different aspects, yet they fall short when those facets are weaved together across enterprise architectures and business environments. On the other side agile quality solutions may also fail to cope with transverse business functions shared across architectures. Hence the need of a bird’s-eye view putting quality into a broader enterprise perspective.
Who Cares for Quality
Whatever the attributes considered, quality should clearly encompass actual products as well as their uses. For that purpose quality has to be assessed with regard to the requirements as expressed by business stakeholders, users, or systems engineers and administrators. Given the constraints and specificity of changing environments, objective yardsticks are of limited use and quality is often to be assessed for the lack thereof:
- Business requirements: the product doesn’t meet expectations with regard to business contents (objects and logic).
- Functional requirements: while the product meets business requirements, the part played by supporting systems doesn’t meet users’ expectations.
- Quality of service: while the product meets business and functional requirements, users’ experience doesn’t meet expectations.
- Technical requirements: while the product meets users’ expectations (business, functional, and ease of use), there are problems with deployment, maintenance, or operations.
Crossing those concerns, quality assessment has to deal with two primary challenges:
- Since assessment at each level can be conditioned by lower levels, outcomes must be described and traced accordingly. That is to be the role of quality management.
- Since assessment has to cover both products and their use during their shelf life, uncertainty will have to be taken into account. That is to be the role of quality assurance.
A third aspect can be added for externalities, i.e factors whose impact cannot be clearly or uniquely attributed: external risks are not under control, ergonomy cannot be accurately measured, and the assessment of ROI for processes improvement remains a matter of insight.
Quality Management & Documentation
The primary objective of quality management is to identify, define, and track the targeted outcomes and the factors deemed to affect their characteristics: contracts, products traceability, models reuse, tests, etc.
Depending on target and development model, management footprint can be defined at three levels of detail:
- With regard to the use of products in their operational context, the focus is to be on deployed systems compared to textual specifications (a).
- With regard to the intrinsic properties of deliverables, the focus is to be extended to software components (b).
- When products are to be deployed in different environments, or to be maintained or modified along time, additional documentation will be necessary to trace changes to functional (c) and enterprise (d) architectures.
In any case (i.e with or without intermediate documentation,) traceability is to be a corner-stone of quality management:
- Business processes with regard to business objectives, e.g how to assess insurance premiums or compute missile trajectory.
- Code with regard to textual requirements.
- System functionalities with regard to business processes. Use cases are widely used to describe how systems are to support business processes, and system functionalities are combined to realize use cases.
- System components as technical implementations of functionalities targeted to different users, locations, and configurations.
And another dimension of traceability is required when quality assurance has to deal with uncertainty, risks, and decision-making.
From Management to Assurance
The objective of quality assurance is to define, carry on, and monitor operations in order to improve the characteristics concerned and reduce the probability that something will go amiss during the planned shelf life of products.
For that purpose assurance footprint and granularity must be aligned with the layers defined by quality management:
- Integration and acceptance tests are carried out in reference to requirements on the assumption that software components have been validated.
- Code checking and unit tests are carried out in reference to business and functional requirements on the assumption that their consistency has been checked.
- External consistency is checked with regard to business requirements independently of functional or technical ones.
- Internal consistency is checked with regard to functional requirements on the assumption that the business requirements (external) consistency has been checked.
Those operations, meant to deal with the quality of each layers, have to be combined with schemes of secure transformations between layers, e.g reuse, patterns or code generation. That would put quality on a sound basis were it not for externalities.
Quality Assurance & Risk Management
As already noted, QA has to take into account uncertainties and risks both external (business or technical environments) and internal (development processes). Assuming quality assurance has to include risk assessment, policies should be driven by risk acceptance levels:
- No risk: quality assurance can be designed as to eliminate some uncertainties (e.g reuse and code generation).
- No risk taken: whereas business and technology options are not sure bets some must be carried out regardless of what happens in the environment (e.g unexpected regulatory change or delay in critical technology). In that case QA must provide fallback solutions.
- Managed risks: some defaults or delays can be priced and weighted by likelihood. In that case QA should monitor the risks and balance their cost (e.g resources consumption, late delivery) against the cost of preventive (e.g more systematic checks on consistency, additional staff) or corrective (e.g tests or maintenance) measures.
That will put quality management at the nexus between regulatory compliance, risks management and quality assurance.