Knowledge Driven Test Cases

Preamble

The way tests are designed and executed is being doubly affected by development methods and AI technologies. On one hand well-founded approaches (e.g test-driven development) are often confined to faith-based niches; on the other hand automated schemes and agile methods push many testers out of their comfort zone.

Reality check (Kader Attia)

Resetting the issue within a knowledge-based enterprise architecture would pave the way for sound methods and open new doors for their users.

outline

Tests can be understood in terms of preventive and predictive purposes, the former with regard to actual products, the latter with regard to their forthcoming employ. On that account policies are to distinguish between:

  • Test plans, to be derived from requirements.
  • Test cases, to be collected from environments.
  • Test execution, to be run and monitored in simulated environments.

The objective is to cross these pursuits with knowledge architecture layers.

Test plans are derived from business requirements, either on their own at process level (e.g as users stories or activities), or combined with functional requirements at application level (e.g as use cases). Both plans describe sequences of actions meant to be performed by organizational entities identified at enterprise level. Circumstances are then specified with regard to quality of service and technical requirements.

Mapping tests to knowledge architecture layers.

Test cases’ backbones are built from business scenarii fleshed out with instances mimicking identified entities from business environment, and hypothetical decisions taken by entitled users. The generation of actual instances and decisions could be automated depending on the thoroughness and consistency of business requirements.

To be actually tested, business scenarii have to be embedded into functional ones, yet the distinction must be maintained between what pertains to business logic and what pertains to the part played by supporting systems.

By contrast, despite being built from functional scenarii, integration and acceptance ones are meant to be blind to business or functional contents, and cases can therefore be generated independently.

Unit and components tests are the building blocks of all test cases, the former rooted in business requirements, the latter in functional ones. As such they can be used to a built-in integration of tests and development.

TDD in the loop

Whatever its merits for phased projects, the development V-model suffers from a structural bias because flaws rooted in requirements, arguably the most damaging, tend to be diagnosed after designs are encoded. Test driven development (TDD) takes a somewhat opposite approach as code specifications are governed by testability. But reversing priorities may also introduce reverse issues: where the V-model’s verification and validation come too late and too wide, TDD’s may come hasty and blinkered, with local issues masking global ones. Applying the OODA (Observation, Orientation, Decision, Action) loop to test cases offers a way out of the dilemma:

  • Observation (West): test and assessment for component, integration, and acceptance test cases.
  • Orientation (North): assessment in the broader context of requirements space (business, functional, Quality of Service), or in the local context of application (East).
  • Decision (East): confirm or adjust the development paths with regard to functional scenarii, development backlog, or integration constraints.
  • Action (South): develop code at unit, component, or process levels.
From V to O: How to iterate across layers

As each station is meant to deal with business, functional, and operational test cases, the challenge is to ensure a seamless integration and reuse across iterations and layers.

managing Tests cases

Whatever the method, tests plans are meant to mirror requirements scope and structure. For architecture oriented projects, tests should be directly aligned with the targeted capabilities of architecture layers:

For architecture oriented projects test plans should be set with regard to capabilities.

For business driven projects, test plans should be set along business
scenarii, with development units and associated test cases defined with regard to activities. When use cases, which cover the subset of activities supported by systems, are introduced upfront for both business and functional requirements, test plans should keep the distinction between business and functional requirements.

Stories (bottom left), activities (right), use cases (top left).

All things considered, test cases are to be comprehensively and consistently run against requirements distinct in goals (business vs architecture), layers (business, functions, platforms), or formalism (text, stories, use cases, …).

In contrast, test cases are by nature homogeneous as made of instances of objects, events, and behaviors; ontologies can therefore be used to define and manage these instances directly from models. The example below make use of instances for types (propulsion, body), car model (Renault Clio), and car (58642).

Ontologies can then be used to manage test cases as instances of activities (development tests) or processes (integration and acceptance tests):

Test cases as instances of processes (integration) and activities (development)

The primary and direct benefit of representing test cases as instances in ontologies is to ensures a seamless integration and reuse of development, integration, and acceptance test cases independently of requirements context.

But the ontological approach have broader and deeper consequences: by defining test cases as instances in line with environment data, it opens the door to their enrichment through deep-learning.

knowledgeable test cases

Names may vary but tests are meant to serve a two-facet objective: on one hand to verify the intrinsic qualities of artefacts as they are, independently of context and usage; on the other hand to validate their features with regard to extrinsic circumstances, present or in a foreseeable future.

That duality has logical implications for test cases:

  • The verification of intrinsic properties can be circumscribed and therefore by carried out based on available information, e.g: design, programing language syntax and semantics, systems configurations, etc.
  • The validation of functional features and behaviors is by nature open-ended with regard to scope and time-frame; it ensues that test cases have to rely on incomplete or uncertain information.

Without practical applications that distinction has been of little consequence, until now: while the digital transformation removes the barriers between test cases and environment data, the spreading of machine learning technologies multiplies the possibilities of exchanges.

Along the traditional approach, test cases relies on three basic sources of information:

  • Syntax and semantics of programing languages are used to check software components (a)
  • Logical and functional models (including patterns) are used to check applications designs (b).
  • Requirements are used to check applications compliance (c).
Traditional (a,b,c) and knowledge driven (d,e,f) test cases

With barriers removed, test cases as instances can be directly aligned with environment data, opening doors to their enrichment, e.g:

  • Random data samples can be mined from environments and used to deal with human instinctive or erratic behaviors. By nature knee jerks or latent behavior cannot be checked with reasoned test cases, yet they neither occur in a void but within known operational of functional or circumstances; data analytics can be used to identify these quirks (d).
  • Systems being designed artifacts, components are meant to tally with models for structures as well as behaviors. Crossing operational data with design models will help to refine and hone integration and acceptance test cases (e).
  • Whereas integration tests put the focus on models and code, acceptance tests also involve the mapping of models to business and organizational concepts. As a corollary, test cases are to rely on a broader range of knowledge: external regulations, mined from environments, or embedded in organization through individual and collective skills (f).

Given the immersion of enterprises in digital environments, and assuming representing test cases as ontological instances, these are already practical opportunities. But the real benefits of knowledge based test cases are to come from leveraging machine learning technologies across enterprise and knowledge architectures.

FURTHER READING

Squared Outline: Languages

As a capability of live organisms, languages are best understood in terms of communication.

That understanding is of particular interest for enterprises immersed in digital environments inhabited by hybrids with deep learning capabilities.

  1. Languages begin with the need of direct (here) and immediate (now) communication. While there is no time for explanations, messages must convey some meaning, if only to distinguish friends from foes. Hence the use of signs pointing to categories of objects or phenomena. That’s the language lexical layer linking instantly observations (data) to information (bottom right).
  2. Rules governing the combination of signs follow soon because more has to be communicated about circumstances and what is to be done with. That’s the language syntactic layer linking observations (data) to current information (top right).
  3. The breakthrough comes with symbolic representation: once
    disentangled from immediate circumstances, communications can encompass whatever is deemed relevant in contexts and concerns;
    That’s the language semantic layer that weave together information and knowledge (top left).
  4. The cognitive ability to “manipulate” symbolic representations (aka models) independently of circumstances opens the door to any kind of constructions. That’s the language pragmatic layer meant to put knowledge to actual use (bottom left).

That functional taxonomy can be usefully applied to the digital transformation of enterprise architectures, the first layer aligned with data, the second and third with information, and the fourth with knowledge.

FURTHER READING

Digital Strategy

Preamble

Enterprise governance has to face combined changes in the way business times and spaces are to be taken into account. On one hand social networks put well-thought-out market segments and well planned campaigns at the mercy of consumers’ weekly whims. On the other hand traditional fences between environments and IT systems are crumbling under combined markets and technological waves.

The Strategic Objective is to Weave Systems and Knowledge Architectures (Wayne Thiebaud)

To overcome these challenges enterprises strategies should focus on four pillars:

  • Governance: the immersion of enterprises in digital environments and the crumbling of traditional fences require in-depth changes in information modeling and knowledge management schemes.
  • Data and Information: massive and continuous inflows of data calls for a
    seamless integration of data analytics (perception), information models (reasoning), and knowledge (decision-making).
  • Security & Confidentiality: new regulatory environments and costs of privacy breaches call for a clear distinction between data tied to identified individuals and information associated to designed categories.
  • Innovation: digital environments induces a new order of magnitude for the pace of technological change. Making opportunities from changes can only be achieved through collaboration mechanisms harnessing enterprise knowledge management to environments intakes.

FURTHER READING

Focus: Data vs Information

Preamble

Distinctions must serve a purpose and be assessed accordingly. On that account, what would be the point of setting apart data and information, and on what basis could that be done.


From Data Stripes to Information Structure (Victor Vasarely)

Until recently the two terms seem to have been used indifferently; until, that is, the digital revolution. But the generalization of digital surroundings and the tumbling down of traditional barriers surrounding enterprises have upturned the playground as well as the rules of the game.

Previously, with data analytics, information modeling, and knowledge management mostly carried out as separate threads, there wasn’t much concerns about semantic overlaps; no more. Lest they fall behind, enterprises have to combine observation (data), reasoning (information), and judgment (knowledge) as a continuous process. But such integration implies in return more transparency and traceability with regard to resources (e.g external or internal) and objectives (e.g operational or strategic); that’s when a distinction between data and information becomes necessary.

Economics: Resources vs Assets

Understood as a whole or separately, there is little doubt that data and information have become a key success factor, calling for more selective and effective management schemes.

Being immersed in digital environments, enterprises first depend on accurate, reliable, and timely observations of their business surroundings. But in the new digital world the flows of data are so massive and so transient that mining meaningful and reliable pieces is by itself a decisive success factor. Next, assuming data flows duly processed, part of the outcome has to be consolidated into models, to be managed on a persistent basis (e.g customer records or banking transactions), the rest being put on temporary shelves for customary uses, or immediately thrown away (e.g personal data subject to privacy regulations). Such a transition constitutes a pivotal inflexion point for systems architectures and governance as it sorts out data resources with limited lifespan from information assets with strategic relevance. Not to mention the sensibility of regulatory compliance to data management.

Processes: Operations vs Intelligence

Making sense of data is pointless without putting the resulting information to use, which in digital environments implies a tight integration of data and information processing. Yet, as already noted, tighter integration of processes calls for greater traceability and transparency, in particular with regard to the origin and scope: external (enterprise business and organization) or internal (systems). The purposes of data and information processing can be squared accordingly:

  • The top left corner is where business models and strategies are meant to be defined.
  • The top right corner corresponds to traditional data or information models derived from business objectives, organization, and requirement analysis.
  • The bottom line correspond to analytic models for business (left) and operations (right).

Squaring the purposes of Data & Information Processing

That view illustrates the shift of paradigm induced by the digital transformation. Prior, most mappings would be set along straight lines:

  • Horizontally (same nature), e.g requirement analysis (a) or configuration management (b). With source and destination at the same level, the terms employed (data or information) have no practical consequence.
  • Vertically (same scope), e.g systems logical to physical models (c) or business intelligence (d). With source and destination set in the same semantic context the distinction (data or information) can be ignored.

The digital transformation makes room for diagonal transitions set across heterogeneous targets, e.g mapping data analytics with conceptual or logical models (e).

That double mix of levels and scopes constitutes the nexus of decision-making processes; their transparency is contingent on a conceptual distinction between data and information.

At operational level the benefits of the distinction are best expressed through what is commonly known as the OODA (Observation, Orientation, Decision, Action) loop:

  • Data is used to align operations (systems) with observations (territories).
  • Information is used to align categories (maps) with objectives.

Roles of Data (red) & Information (blue) in integrated decision-making processes

Then, the conceptual distinction between data and information is instrumental for the integration of operational and strategic decision-making processes:

  • Data analytics feeding business intelligence
  • Information modeling supporting operational assessment.

Not by chance, these distinctions can be aligned with architecture layers.

Architectures: Instances vs Categories

Blending data with information overlooks a difference of nature, the former being associated with actual instances (external observation or systems operations), the latter with symbolic descriptions (categories or types). That intrinsic difference can be aligned with architecture layers (resources are consumed, assets are managed), and decision-making processes (operations deal with instances, strategies with categories).

With regard to architectures, the relationship between instances (data) and categories (information) can be neatly aligned with capability layers, as represented by the Pagoda blueprint:

  • The platform layer deals with data reflecting observations (external facts) and actions (system operations).
  • The functional layer deals with information, i.e the symbolic representation of business and organization categories.
  • The business and organization layer defines the business and organization categories.

It must also be noted that setting apart what pertains to individual data independently of the informations managed by systems clearly props up
compliance with privacy regulations.


Architectures & Decision-making

With regard to decision-making processes, business intelligence uses the distinction to integrate levels, from operations to strategic planning, the former dealing with observations and operations (data), the latter with concepts and categories (information and knowledge).

Representations: Knowledge Architecture

As noted above, the distinction between data and information is a necessary counterpart of the integration of operational and intelligence processes; that implies in return to bring data, information, and knowledge under a common conceptual roof, respectively as resources, assets, and service:

  1. Resources: data is captured through continuous and heterogeneous flows from a wide range of sources.
  2. Assets: information is built by adding identity, structure, and semantics to data.
  3. Services: knowledge is information put to use through decision-making.

Ontologies, which are meant to encompass all and every kind of knowledge, are ideally suited for the management of whatever pertains to enterprise architecture, thesaurus, models, heuristics, etc.

CaKe_DataInfoKnow

That approach has been tested with the Caminao ontological kernel using OWL2; a beta version is available for comments on the Stanford/Protégé portal with the link: Caminao Ontological Kernel (CaKe_).

Conclusion: From Metadata to Machine Learning

The significance of the distinction between data and information shows up at the two ends of the spectrum:

On one hand, it straightens the meaning of metadata, to be understood as attributes of observations independently of semantics, a dimension that plays a critical role in machine learning.

On the other hand, enshrining the distinction between what can be known of individuals facts or phenomena and what can be abstracted into categories is to enable an open and dynamic knowledge management, also a critical requisite for machine learning.

FURTHER READING

External Links

Squared Outline: Artificial Brains

Artificial intelligence is generally defined in relation with the human ability to figure out situations and solve problems. The term may also put the focus on an hypothetical disembodied artificial brain.

A Human Perspective on Artificial Brains

Taking the brain perspective makes for easier outlining:

  1. Artificial brains can build and process symbolic and non symbolic representations, respectively with semantic and neural networks.
  2. Artificial brains can solve problems applying logic and abstraction or 
    data analytics, respectively to symbolic and non symbolic representations.
  3. Like human ones, artificial brains combine sensory-motor capabilities with purely cognitive ones.
  4. Like human ones, and apart from sensory-motor capabilities, artificial brains support a degree of cognitive plasticity and versatility between symbolic and non symbolic representation and processing.

These generic capabilities are at the root of the wide-ranging and dramatic advances of machine learning.

FURTHER READING

Squared Outline: Ontologies

The upsurge in the scope and performances of artificial brains sometimes brings a new light on human cognition. Semantic layers and knowledge graphs offer a good example of a return to classics, in that case with Greek philosophers’ ontologies.

According to their philosophical origins, ontologies are systematic accounts of existence for whatever can make sense in an universe of discourse. From that starting point four basic observations can be made:

  1. Ontologies are structured set of names denoting symbolic (aka cognitive) representations.
  2. These representations can stand at different epistemic levels: terms or labels associated to representations (nothing is represented), ideas or concepts (sets of terms), instances of identified objects or phenomena, categories (sets of instances), documents.
  3. Ontologies are solely dedicated to the validity and internal consistency of the representations. Not being concerned with external validity, As they are not meant to support empirical purposes.
  4. Yet, assuming a distinction between epistemic levels, ontologies can be used to support both internal and external consistency of models. 

That makes models a refinement of ontologies as they are meant to be externally consistent and serve a purpose.

FURTHER READING

EXTERNAL LINKS

Stories In Logosphere

As championed by a brave writer, should we see the Web as a crib for born again narratives, or as a crypt for redundant texts.

melikOhanian_letters
Crib or Crypt (Melik Ohanian)

Once Upon A Time

Borrowing from Einstein, “the only reason for time is so that everything doesn’t happen at once.” That befits narratives: whatever the tale or the way it is conveyed, stories take time. Even if nothing happens, a story must be spelt in tempo and can only be listened to or read one step at a time.

In So Many Words

Stories have been told before being written, which is why their fabric is made of words, and their motifs weaved by natural languages. So, even if illustrations may adorn printed narratives, the magic of stories comes from the music of their words.

A Will To Believe

To enjoy a story, listeners or readers are to detach their mind from what they believe about reality, replacing dependable and well-worn representations with new and untested ones, however shaky or preposterous they may be; and that has to be done through an act of will.

Stories are make-beliefs: as with art in general, their magic depends on the suspension of disbelief. But suspension is not abolition; while deeply submerged in stories, listeners and readers maintain some inward track to the beliefs they left before diving; wandering a cognitive fold between surface truths and submarine untruths, they seem to rely on a secure if invisible tether to the reality they know. On that account, the possibility of an alternative reality is to transform a comforting fold into a menacing abyss, dissolving their lifeline to beliefs. That could happen to stories told through the web.

Stories & Medium

Assuming time rendering, stories were not supposed to be affected by medium; that is, until McLuhan’s suggestion of medium taking over messages. Half a century later internet and the Web are bringing that foreboding in earnest by melting texts into multimedia documents.

Tweets and Short Message Services (SMS) offer a perfect illustration of the fading of text-driven communication, evolving from concise (160 characters) text-messaging  to video-sharing.

That didn’t happen by chance but reflects the intrinsic visual nature of web contents, with dire consequence for texts: once lording it over entourages of media, they are being overthrown and reduced to simple attachments, just a peg above fac-simile. But then, demoting texts to strings of characters makes natural languages redundant, to be replaced by a web Esperanto.

Web Semantic Silos

With medium taking over messages, and texts downgraded to attachments, natural languages may lose their primacy for stories conveyed through the web, soon to be replaced by the so-called “semantic web”, supposedly a lingua franca encompassing the whole of internet contents.

As epitomized by the Web Ontology Language (OWL), the semantic web is based on a representation scheme made of two kinds of nodes respectively for concepts (squares) and conceptual relations (circles).

Scratch_WarPeace00
Semantic graphs (aka conceptual networks) combine knowledge representation (blue, left) and domain specific semantics (green, center & right)

Concept nodes are meant to represent categories specific to domains (green, right); that tallies with the lexical level of natural languages.

Connection nodes are used to define two types of associations:

  • Semantically neutral constructs to be applied uniformly across domains; that tallies with the syntactic level of natural languages (blue, left).
  • Domain specific relationships between concepts; that tallies with the semantic level of natural languages (green, center).

The mingle of generic (syntactic) and specific (semantic) connectors induces a redundant complexity which grows exponentially when different domains are to be combined, due to overlapping semantics. Natural languages typically use pragmatics to deal with the issue, but since pragmatics scale poorly with exponential complexity, they are of limited use for semantic web; that confines its effectiveness to silos of domain specific knowledge.

Scratch_silos.jpg
Natural Language Pragmatics As Bridges Across Domain Specific Silos

But semantic silos are probably not the best nurturing ground for stories.

Stories In Cobwebs

Taking for granted that text, time, and suspension of disbelief are the pillars of stories, their future on the web looks gloomy:

  • Texts have no status of their own on the web, but only appear as part of documents, a media among others.
  • Stories can bypass web practice by being retrieved before being read as texts or viewed as movies; but whenever they are “browsed” their intrinsic time-frame and tempo are shattered, and so is their music.
  • If lying can be seen as an inborn human cognitive ability, it cannot be separated from its role in direct social communication; such interactive background should also account for the transient beliefs in fictional stories. But lies detached from a live context and planted on the web are different beasts, standing on their own and bereft of any truth currency that could separate actual lies from fictional ones.

That depressing perspective is borne out by the tools supposed to give a new edge to text processing:

Hyper-links are part and parcel of internet original text processing. But as far and long as stories go, introducing links (hardwired or generated) is to hand narrative threads over to readers, and by so transforming them into “entertextment” consumers.

Machine learning can do wonders mining explicit and implicit meanings from the whole of past and present written and even spoken discourses. But digging stories out is more about anthropology or literary criticism than about creative writing.

As for the semantic web, it may work as a cobweb: by getting rid of pragmatics, deliberately or otherwise, it disables narratives by disengaging them from their contexts, cutting them out in one stroke from their original meaning, tempo, and social currency.

The Deconstruction of Stories

Curiously, what the web may do to stories seems to reenact a philosophical project which gained some favor in Europe during the second half of the last century. To begin with, the  deconstruction  philosophy was rooted in literary criticism, and its objective was to break the apparent homogeneity of narratives in order to examine the political, social, or ideological factors at play behind. Soon enough, a core of upholders took aim at broader philosophical ambitions, using deconstruction to deny even the possibility of a truth currency.

With the hindsight on initial and ultimate purposes of the deconstruction project, the web and its semantic cobweb may be seen as the stories nemesis.

Further Reading

External Links