Declarative Frameworks & Emerging Architectures

Preamble

Ingrained habits die hard, especially mental ones as they are not weighted down by a mortal envelope. Fear is arguably a primary factor of persistence, if only because being able to repeat something proves that nothing bad has happened before.

Live, Die, Repeat (Philippe de Champaigne)

Procedures epitomize that human leaning as ordered sequences of predefined activities give confidence in proportion to generality. Compounding the deterministic delusion, procedures seem to suspend time, arguably a primary factor of human anxiety.

Procedures are Dead-ends

From hourglasses to T.S. Elliot’s handful, sand materializes human double bind with time, between will of measurement and fear of ephemerality.

Procedures seem to provide a way out of the dilemma by replacing time with prefabricated frames designed to ensure that things can only happen when required. But with extensive and ubiquitous digital technologies dissolving traditional boundaries, enterprises become directly exposed to competitive environments in continuous mutation; that makes deterministic schemes out of kilter:

  • There is no reason to assume the permanence of initial time-frames for the duration of planned procedures.
  • The blending of organizations with supporting systems means that architectural changes cannot be carried out top-down lest the whole be paralyzed by the management overheads induced by cross expectations and commitments.
  • Unfettered digital exchanges between enterprises and their environment, combined with ubiquitous smart bots in business processes, are to require a fine grained management of changes across artefacts.

These shifts call for a complete upturn of paradigm: event driven instead of scheduled, bottom-up instead of top-down, model based instead of activity driven.

Declarative frameworks: Non Deterministic, Model Based, Agile

The procedural/declarative distinction has its origin in the imperative/declarative programming one, the principle being to specify necessary and sufficient conditions instead of defining the sequence of operations, letting programs pick the best options depending on circumstances.

Applying the principle to enterprise architecture can help to get out of a basic conundrum, namely how to manage changes across supporting systems without putting a halt to enterprise activities.

Obviously, the preferred option is to circumscribe changes to well identified business needs, and carry on with the agile development model. But that’s not always possible as cross dependencies (business, organizational, or technical) may induce phasing constraints between engineering tasks.

As notoriously illustrated by Waterfall, procedural (if not bureaucratic) schemes have for long be seen as the only way to deal with phasing constraints; that’s not a necessity: with constraints and conditions defined on artifacts, developments can be governed by their status instead of having to be hard-wired into procedures. That’s precisely what model based development is meant to do.

And since iterative development models are by nature declarative, agile and model-based development schemes may be natural bedfellows.

Epigenetics & Emerging architectures

Given their their immersion in digital environments and the primacy of business intelligence, enterprises can be seen as living organisms using information to keep an edge in competitive environments. On that account homeostasis become a critical factor, to be supported by osmosis, architecture versatility and plasticity, and traditional strategic planning.

Set on a broader perspective, the merging of systems and knowledge architectures on one hand, the pervasive surge of machine learning technologies on the other hand, introduce a new dimension in the exchange of information between enterprises and their environment, making room for emerging architectures.

Using epigenetics as a metaphor of the mechanisms at hand, enterprises would be seen as organisms, systems as organs and cells, and models (including source) as genome coded with the DNA.

According to classical genetics, phenotypes (actual forms and capabilities of organisms) inherit through the copy of genotypes and changes between generations can only be carried out through changes in genotypes. Applied to systems, it would entail that changes would only happen intentionally, after being designed and programmed into the systems supporting enterprise organization and processes.

Enterprises epigenetics and emerging architectures

The Extended Evolutionary Synthesis considers the impact of non coded (aka epigenetic) factors  on the transmission of the genotype between generations. Applying the same principles to systems would introduce new mechanisms:

  • Enterprise organization and their use of supporting systems could be adjusted to changes in environments prior to changes in coded applications.
  • Enterprise architects could use data mining and deep-learning technologies to understand those changes and assess their impact on strategies.
  • Abstractions would be used to consolidate emerging designs with existing architectures.
  • Models would be transformed accordingly.

While applying the epigenetics metaphor to enterprise mutations has obvious limitations, it nonetheless puts a compelling light on two necessary conditions for emerging structures:

  • Non-deterministic mechanisms governing the way changes are activated.
  • A decrypting mechanism between implicit or latent contents (data from digital environments) to explicit ones (information systems).

The first condition is to be met with agile and model based engineering, the second one with deep-learning.

FURTHER READING

EA Capacity & Maturity

Preamble

Appraising enterprises capability and maturity means navigating between the fuzzy depths of business models and the marked features of supporting systems.

Keeping in touch with environment (Urs Fischer)

Business capabilities are by nature a fleeting lot to assess, considering the innate diversity and volatility of circumstances and the fact that successes belong to exception more than rule. By contrast, the assessment of systems capabilities is much easier as they can be defined in architectural terms.

Digital transformation may help to solve the dilemma by dissolving it: given the merging of systems and organization, the key success factor for enterprises is their capacity to assess changes and opportunities and adjust their processes and architectures accordingly. On that account, performances are to depend on the dynamic alignment of enterprises representations (maps) with their business and technological environments (territories).

enterprise architecture & environments

Based on a broadly accepted architecture paradigm epitomised by the Zachman framework, and figured by the Pagoda blueprint, enterprise environments and architectures are to be defined along three levels (aka tiers or layers):

Enterprise must align their maps and territories
  • Data level, for digital environment, operations, and platforms; described by physical and analytical models.
  • Information level, for systems and engineering, described by logical and functional models.
  • Knowledge level, for business objectives and organization; described by conceptual and process models.

That framework can be used to assess enterprises capacity to change independently of business specificites.

DEALING WITH CHANGES

Whereas absolute measurements are tied to valuation contexts, relative ones can be ecumenical, hence the benefit of targeting the capacity to change instead of trying to measure capacity by itself.

As far as enterprise architectures are concerned, changes can originate from business or technological environments, the former at process level, the latter at application level.

To begin with, as much change as possible should be dealt with at application level through organizational, functional, or operational adjustments, without affecting architectural assets. That is to be achieved through architectures versatility and plasticity (aka agility) .

When architectural changes are needed, their footprint can be layered in terms of Model Driven Architecture (MDA) , i.e computation independent (CIM), platform independent (PIM), and platform specific (PSM) models.

Change in Enterprise Architectures

Ideally, changes should spread top-down from computation independent models to platform specific ones, with or without affecting platform independent ones. On that account, the footprint of changes rooted in technical environment should be circumscribed to platforms adaptations and charted by platform specific models.

By contrast, changes rooted in business environment could induce changes in any or all architecture layers:

  • Platform specific (PSM), e.g when business logic is implemented by rules engines.
  • Platform independent (PIM), e.g new business functions.
  • Computation independent (CIM), e.g new business processes.

That model-based approach is to be used to define enterprises changes in terms of entropy.

Entropy & capacity to change

Change is a matter of time, especially for business, and a delicate balance is to be achieved between assessments (which improve when given time until they become redundant), and commitments (which risk missing opportunities if kept waiting for too long).

The issues can be expressed by applying the OODA (Observation, Orientation, Decision, Action) loop to system engineering:

  • Changes in business and technology environments are observed at digital (e.g data mining) or conceptual level (e.g business intelligence) (a).
  • Assessment deals with the reliability of observations as well as their meaning with regard to enterprise objectives, organization, and systems (b).
  • Policies are updated and decisions made regarding the adjustment of objectives, resources, organization, or assets (c).
  • Decisions are implemented as technical and business commitments (d).

On that basis, the capacity to change is to depend on:

  • Osmosis: quality (accuracy, reliability,…), delay, and automaticity of observations with regard to changes in environments (data mining).
  • Operational traceability across decisions, actions and observations (process mining, verification).
  • Alignment of business and digital environments (validation).
  • Consistency of architecture models (CIMs, PIMs, PSMs).
  • Value chains and lean engineering: business logic (CIMs) directly embedded in software designs (PSMs).
Capacity to Change

With the benefits of digital transformation, these dimensions should be defined in terms of information processing.

osmosis & blind spots

The generalization of digital exchanges between enterprises and their environment brings back the concept of entropy, defined by cybernetics as the quantum of energy within a system that cannot be put to use. Applied to enterprises, entropy can be understood as a blind spot on environment data, arguably a critical hindrance to their capacity to move and adjust. The primary objective should therefore to minimize that blind spot, i.e to maximize the outcome of data processing .

As digital environments bring about level data playground, to get a competitive edge enterprises have to make a better sense of it; hence the importance of a distinction between data and information, the former obtained from the environment, the latter obtained through the processing of the former. On that basis, the capacity to change, defined as the opposite of entropy, is to be determined by the way data is processed into information.

Digital osmosis, i.e the exchange of digital data between enterprises operations and environments is clearly a primary factor: inbound streams can be mined as to provide comprehensive and timely snapshots, outbound streams can be weaved into business processes, enhancing the capacity to translate decisions into action.

Digital osmosis could also bolster lean engineering with the direct integration of (digital) business logic into software (e.g using rules engines), reinforcing the shortcuts between observations and decisions whenever orientation can be avoided. That would also enhance traceability between platform specific (PSMs) and operational models, as well as the monitoring of actions and the analysis of feedbacks (process mining).

Osmosis could be easily (if not accurately) estimated with the ratio between digitized flows and the whole of data flows, with measurements weighted by delays between observations and data processing.

A taxonomy of changes

Digital and business environments are not to be confused, and there is no reason to assume that their respective changes tally. As it happens, digital transformation may reinstall organizations at the nexus of changes by providing a powerful leverage on systems; and if entropy is considered, that is to be achieved through symbolic representations, aka models.

While names may vary, a distinction is generally made between changes according to their horizon, shorter for operational or tactical ones, longer for strategic ones. As so often with quantitative classifications, that understanding has been of limited use given the diversity of time-frames across industries. The digital transformation open the door to a qualitative approach, defining changes according to the nature of of their footprint:

  • From the business perspective, it should restate the primacy of organization for the harnessing of IT benefits.
  • From the architecture perspective, it would rank assets according to “digital modality”: symbolic (information, knowledge), tangible (e.g platforms), or a combination of both (functional architecture).
Changes can be defined according to the digital modality of their footprint

Taking the strategic perspective, changes in technical or economic factors are to affect assets and processes for a large and often undetermined number of production cycles; they must consequently be set in time-frames extending beyond managed horizons (actual chronologies depend on industries specificities). Despite regular updates, supporting models and hypothesis may lose their relevance during the intervals, with the resulting entropy hampering the assessment of opportunities and policies. Such discrepancies can be circumscribed if planned organization and computation independent models (CIMs) are systematically checked against observations.

Compared to strategic ones, functional changes can be aligned with a limited number of production cycles, which irons out most of the discrepancies between maps and territories associated with strategic planning. Instead, entropy could arise from a lack of transparency and traceability between the models used to map organization and processes (CIMs) to functional (PIMs) and technical (PSMs) architectures.

Finally, operational changes can be carried out at process level, without affecting architectures. In that case entropy could be the result of a misalignment of commitments and observations granularity.

On that basis, planning and managing changes in digital and business environments should be driven by models.

ENTROPY & maturity

As defined by cybernetics, entropy can be explained by the discrepancies between environment states (aka micro-states) and their representation (macro-states). Applied to enterprises environments and architectures, it would mean territories and operations for the former, maps and policies for the latter.

On that account the maturity of an organization should be assessed with regard to its ability to manage changes within and across environments:

  • Consistency of changes within environments: between territories and operations (digital environments), between maps and policies (business environments) .
  • Validity of changes across environments: between maps and territories, and between policies and operations.
Consistency is checked within environments, validity is checked across environments

Maturity levels could then be set according to the scope of managed entropy:

  1. Digital osmosis: all exchanges with environments come with digital counterpart sorted with regard to operational data, data attached to managed information, environment data detached from managed information.
  2. Digital value chains: digital integration of business and engineering processes ensuring transparency and traceability along value chains.
  3. Organizational pivot: value chains can be redefined as to make the best of information assets.
  4. Knowledge based architectures: enterprise layers (platforms, systems, organization) are aligned with data, information, and knowledge layers, leveraging the benefits of machine learning across enterprise organization and systems.

As it’s safe to assume that scaling maturity levels across enterprise architectures can only be carried out progressively, progresses have to be backed up by a comprehensive and ecumenical repository of physical and symbolic resources and assets, and be supported by workflows combining iterative (continuous and business driven) and model-based (phased, architecture driven) engineering processes .

FURTHER READING

Focus: Entropy & Homeostasis

Preamble

As defined by thermodynamics entropy is a measure of the energy within a system that cannot be harnessed to useful use; cybernetics has took over, making entropy a pillar of information theory.

Figuring Digital Matter (Marcelo Cidade)

Notwithstanding the focus put on viable systems and organizations (as epitomized by the pioneering work of Stafford Beer), cybernetics’ actual imprint on corporate governance has been frustrated by the correspondence assumed between information and energy. But the immersion of enterprises into digital environments brings entropy back in front, along with a paradigmatic shift out of thermodynamics.

domain: Physics vs Economics

The second law of thermodynamics states that entropy within a system is constant, and so is information as defined by cybernetics. But economics laws, if there is such a thing, are to differ: as far as business is concerned information is not to be found in commons but comes from the processing of raw data.

As a matter of fact, thermodynamic stability is meaningless in economics because business information is not a given and uniform quantity but a fluctuating and polymorph one. It ensues that for enterprises in competitive environments measures of entropy set in isolation are pointless: taking a leaf from Lewis Carroll’s Red Queen, the entropy of any given enterprise can only be assessed in relation with its competitors.

Taking a general perspective, entropy is meant to be observed either between the two ends of communication channels, or at the boundaries of complex systems (animals, machines, or organizations). While the former can be seen as a formal coding issue, the latter is of particular relevance for enterprises immersed in digital environments.

Departing from thermodynamics view of entropy measuring a system’s thermal energy unavailable for conversion into mechanical work, economics will consider unexplained information, i.e which cannot be acted upon.

But leaving physics also means forsaking the comfort of its metrics. That’s not a problem for communication entropy, which can be dealt with as a coding issue; but the difficulty appears when counts of digits have to be replaced by measures of knowledge.

complexity: microstates vs macrostates

When there are no clear and reliable metrics to put light on it, entropy becomes a black hole defined by disorder of randomness, the former in reference to the order borne out by classifications, the latter in reference to the predictability borne out by probabilities. Since both approaches deal with the relationship between configurations and information, the issue can be summarized with:

  • A quantum of hypothetical information (dashed line).
  • The complexity of identified items (aka microstates).
  • The complexity of representations (aka macrostates).
  • The quantum of information accounted for, ordered or predicted.
How to assess what is missed ?

As stated by the basic law of thermodynamics, to be of any energetic use the internal heat of a system has to be lower than the external one. Generalized in terms of complexity, it means that the quantum of information accounted for will first increase with the number of configurations considered, reach a maximum, and then decrease until the complexity of configurations (aka internal heat) equates the complexity of the target (aka external heat).

Translated to economics, that scheme must be restated in terms of digital environments and business information.

Entropy: From data to information

The generalization of digital exchanges between enterprises and their environment points to the necessary distinction between data (exchanged flows) and information (managed assets). What happens in-between can be defined along two perspectives:

  • Representation: whether the exchanges are carried out at digital level (data observations of microstates) or in association with symbolic representations (models of macrostates) of managed information.
  • Coupling: whether the processing of data is tied to operations or carried out independently.
Entropy Layers

The issue of entropy can be restated accordingly:

  1. The direct consequence of the digital transformation is to increase osmosis between enterprises and their environment, with data flows bypassing traditional boundaries and feeding directly operational processes (bottom right).
  2. As a result, data analytics can be integrated with business processes, improving operational decision-making. In terms of entropy the outcome would be a more effective use of information models (bottom left).
  3. Next, information models (aka symbolic representations) can themselves be changed as to improve their effectiveness, i.e reduce entropy at process as well as architecture level (top left).
  4. Last but not least, deep learning could be applied to digitalized contexts (aka microstates) in order to derive new symbolic representations (aka macrostates) (top right).

On that basis entropy policies should be set at operational and strategic levels, the former dealing with processes, the latter with architectures.

Homeostasis: from Data to knowledge

As far as enterprises are concerned, homeostasis means the adaptation of organizations and systems to changes in competitive environments. To that end, policies set along the entropy paradigm could draw on improving the perception of microstates (data level) as well as the representation of macrostates (information level).

At data level, the perception of changes depends on the osmosis between systems and environments. To that end data mining is to be used to hone the sampling of populations and refine microstates.

At symbolic level (representations), the economics perspective differs from physics on two critical aspects:

  • Contrary to heat, economic information is not a constant, which means that macrostates are to be continuously updated.
  • Moreover, the economic playground is not homogenous but combines physical (e.g demographics or climate), socio-economic (e.g income or education) and symbolic (e.g politics or culture) microstates, which means that macrostates are to be differentiated.
Improving entropy

With or without differentiated macrostates, policies are to rely on two categories of models:

  • Extensional ones target environments, dealing with the analysis of business context, operations, and changes (descriptive models), as well as what should be expected (predictive models).
  • Intensional ones deal with enterprise architectures, current or planned, at processes as well as architecture level (prescriptive models).

As figured above, representations and policies are to be combined, hence the need to anchor them to enterprise architectures, as can be illustrated with the Pagoda blueprint:

Architectures & Homeostasis (crosses and numbers refer to the table above)
  • Populations of instances (microstates) are obtained from digital environments as well as from processes execution (1). Data analytics are used to improve descriptive and predictive models (operational macrostates), and consequently the osmosis between processes and environment (2).
  • Architecture improvements can be achieved through changes in prescriptive models (structural macrostates). Compared to strategic planning carried out top-down from business models and requirements, homeostasis also relies on bottom-up forces which can combine with business models and coalesce into emerging architectures (3).
  • Business and organization models are arguably a primary factor in sorting out data, respectively from environments and systems; and being by nature essentially symbolic, they are more pliable than systems models (0). Homeostasis could therefore bypass architectural macrostates, with models of business and organization emerging directly from digital environments through data mining and deep learning (4).

Such alignment of enterprise architectures and symbolic representations is to greatly enhance the transparency and traceability of digital transformations by organising changes and policies with regard to their nature (physical, functional, organizational, or business) and decision-making horizon (operational or strategic).

FURTHER READING

Squared Outline: Languages

As a capability of live organisms, languages are best understood in terms of communication.

That understanding is of particular interest for enterprises immersed in digital environments inhabited by hybrids with deep learning capabilities.

  1. Languages begin with the need of direct (here) and immediate (now) communication. While there is no time for explanations, messages must convey some meaning, if only to distinguish friends from foes. Hence the use of signs pointing to categories of objects or phenomena. That’s the language lexical layer linking instantly observations (data) to information (bottom right).
  2. Rules governing the combination of signs follow soon because more has to be communicated about circumstances and what is to be done with. That’s the language syntactic layer linking observations (data) to current information (top right).
  3. The breakthrough comes with symbolic representation: once
    disentangled from immediate circumstances, communications can encompass whatever is deemed relevant in contexts and concerns;
    That’s the language semantic layer that weave together information and knowledge (top left).
  4. The cognitive ability to “manipulate” symbolic representations (aka models) independently of circumstances opens the door to any kind of constructions. That’s the language pragmatic layer meant to put knowledge to actual use (bottom left).

That functional taxonomy can be usefully applied to the digital transformation of enterprise architectures, the first layer aligned with data, the second and third with information, and the fourth with knowledge.

FURTHER READING

Digital Strategy

Preamble

The digital transformation induces fundamental changes for the exchanges between enterprises and their environment.

To begin with, their immersion into digital environments means that the traditional fences surrounding their IT systems are losing their relevance, being bypassed by massive data flows to be processed without delay.

Then, the induced osmosis upturns the competition playground and compels drastic changes in governance: less they fall behind, enterprises have to redefine their organization, systems, and processes.

Strategies are meant to draw passageways between current circumstances and conjectured horizons (Kader Attia)

New playground

Strategic thinking is first and foremost making differences with regard to markets, resources and assets, and time-frames. But what makes the digital revolution so disruptive is that it resets the ways differences are made:

  • Markets: the traditional distinctions between products and services are all but forgotten.
  • Resources and assets: with software, smart or otherwise, now tightly mixed in products fabric, and business processes now driven by knowledge, intangible assets are taking the lead on conventional ones.
  • Time-frames: strategies have for long been defined as a combination of anticipations, objectives and policies whose scope extends beyond managed horizons. But digital osmosis and the ironing out of markets and assets traditional boundaries are dissolving the milestones used to draw horizons perspectives.

New perspectives

To overcome these challenges enterprises strategies should focus on four pillars:

  • Governance: the immersion of enterprises in digital environments and the crumbling of traditional fences require in-depth changes in the assessment of enterprises capability and maturity, putting the focus on the ability to change.
  • Data and Information: massive and continuous inflows of data calls for a
    seamless integration of data analytics (perception), information models (reasoning), and knowledge (decision-making).
  • Security & Confidentiality: new regulatory environments and costs of privacy breaches call for a clear distinction between data tied to identified individuals and information associated to designed categories.
  • Innovation: digital environments induce a new order of magnitude for the pace of technological change. Making opportunities from changes can only be achieved through collaboration mechanisms harnessing enterprise knowledge management to environments intakes.

FURTHER READING

Focus: Data vs Information

Preamble

Distinctions must serve a purpose and be assessed accordingly. On that account, what would be the point of setting apart data and information, and on what basis could that be done.


From Data Stripes to Information Structure (Victor Vasarely)

Until recently the two terms seem to have been used indifferently; until, that is, the digital revolution. But the generalization of digital surroundings and the tumbling down of traditional barriers surrounding enterprises have upturned the playground as well as the rules of the game.

Previously, with data analytics, information modeling, and knowledge management mostly carried out as separate threads, there wasn’t much concerns about semantic overlaps; no more. Lest they fall behind, enterprises have to combine observation (data), reasoning (information), and judgment (knowledge) as a continuous process. But such integration implies in return more transparency and traceability with regard to resources (e.g external or internal) and objectives (e.g operational or strategic); that’s when a distinction between data and information becomes necessary.

Economics: Resources vs Assets

Understood as a whole or separately, there is little doubt that data and information have become a key success factor, calling for more selective and effective management schemes.

Being immersed in digital environments, enterprises first depend on accurate, reliable, and timely observations of their business surroundings. But in the new digital world the flows of data are so massive and so transient that mining meaningful and reliable pieces is by itself a decisive success factor. Next, assuming data flows duly processed, part of the outcome has to be consolidated into models, to be managed on a persistent basis (e.g customer records or banking transactions), the rest being put on temporary shelves for customary uses, or immediately thrown away (e.g personal data subject to privacy regulations). Such a transition constitutes a pivotal inflexion point for systems architectures and governance as it sorts out data resources with limited lifespan from information assets with strategic relevance. Not to mention the sensibility of regulatory compliance to data management.

Processes: Operations vs Intelligence

Making sense of data is pointless without putting the resulting information to use, which in digital environments implies a tight integration of data and information processing. Yet, as already noted, tighter integration of processes calls for greater traceability and transparency, in particular with regard to the origin and scope: external (enterprise business and organization) or internal (systems). The purposes of data and information processing can be squared accordingly:

  • The top left corner is where business models and strategies are meant to be defined.
  • The top right corner corresponds to traditional data or information models derived from business objectives, organization, and requirement analysis.
  • The bottom line correspond to analytic models for business (left) and operations (right).

Squaring the purposes of Data & Information Processing

That view illustrates the shift of paradigm induced by the digital transformation. Prior, most mappings would be set along straight lines:

  • Horizontally (same nature), e.g requirement analysis (a) or configuration management (b). With source and destination at the same level, the terms employed (data or information) have no practical consequence.
  • Vertically (same scope), e.g systems logical to physical models (c) or business intelligence (d). With source and destination set in the same semantic context the distinction (data or information) can be ignored.

The digital transformation makes room for diagonal transitions set across heterogeneous targets, e.g mapping data analytics with conceptual or logical models (e).

That double mix of levels and scopes constitutes the nexus of decision-making processes; their transparency is contingent on a conceptual distinction between data and information.

At operational level the benefits of the distinction are best expressed through what is commonly known as the OODA (Observation, Orientation, Decision, Action) loop:

  • Data is used to align operations (systems) with observations (territories).
  • Information is used to align categories (maps) with objectives.

Roles of Data (red) & Information (blue) in integrated decision-making processes

Then, the conceptual distinction between data and information is instrumental for the integration of operational and strategic decision-making processes:

  • Data analytics feeding business intelligence
  • Information modeling supporting operational assessment.

Not by chance, these distinctions can be aligned with architecture layers.

Architectures: Instances vs Categories

Blending data with information overlooks a difference of nature, the former being associated with actual instances (external observation or systems operations), the latter with symbolic descriptions (categories or types). That intrinsic difference can be aligned with architecture layers (resources are consumed, assets are managed), and decision-making processes (operations deal with instances, strategies with categories).

With regard to architectures, the relationship between instances (data) and categories (information) can be neatly aligned with capability layers, as represented by the Pagoda blueprint:

  • The platform layer deals with data reflecting observations (external facts) and actions (system operations).
  • The functional layer deals with information, i.e the symbolic representation of business and organization categories.
  • The business and organization layer defines the business and organization categories.

It must also be noted that setting apart what pertains to individual data independently of the informations managed by systems clearly props up
compliance with privacy regulations.


Architectures & Decision-making

With regard to decision-making processes, business intelligence uses the distinction to integrate levels, from operations to strategic planning, the former dealing with observations and operations (data), the latter with concepts and categories (information and knowledge).

Representations: Knowledge Architecture

As noted above, the distinction between data and information is a necessary counterpart of the integration of operational and intelligence processes; that implies in return to bring data, information, and knowledge under a common conceptual roof, respectively as resources, assets, and service:

  1. Resources: data is captured through continuous and heterogeneous flows from a wide range of sources.
  2. Assets: information is built by adding identity, structure, and semantics to data.
  3. Services: knowledge is information put to use through decision-making.

Ontologies, which are meant to encompass all and every kind of knowledge, are ideally suited for the management of whatever pertains to enterprise architecture, thesaurus, models, heuristics, etc.

CaKe_DataInfoKnow

That approach has been tested with the Caminao ontological kernel using OWL2; a beta version is available for comments on the Stanford/Protégé portal with the link: Caminao Ontological Kernel (CaKe_).

Conclusion: From Metadata to Machine Learning

The significance of the distinction between data and information shows up at the two ends of the spectrum:

On one hand, it straightens the meaning of metadata, to be understood as attributes of observations independently of semantics, a dimension that plays a critical role in machine learning.

On the other hand, enshrining the distinction between what can be known of individuals facts or phenomena and what can be abstracted into categories is to enable an open and dynamic knowledge management, also a critical requisite for machine learning.

FURTHER READING

External Links

Redeeming Conceptual Debts

Preamble

To take advantage of their immersion into digital environments enterprises have to differentiate between data (environment’s facts), information (systems’ representations), and knowledge (enterprise behavior).

Outside / Insight (Anna Hulacova)

That cannot be achieved without ironing out the semantic discrepancies between corresponding representations.

Symbolic Representations

Along with the Symbolic System modeling paradigm, the aim of computer systems is to manage the symbolic representations of business objects and processes pertaining to enterprises contexts and concerns. That view can be summarized in terms of maps and territories:

Maps and territories of systems and their environment

Behind the various labels and modus operandi, maps can be defined on three basic layers:

  • Conceptual models, targeting enterprises organization and business independently of supporting systems.
  • Logical models, targeting the symbolic objects managed by supporting systems as surrogates of business objects and activities.
  • Physical models, targeting the actual implementation of symbolic surrogates as binary objects.

The Pagoda Architecture Blueprint is derived from the Zachman’s frameworks

These maps can be aligned with commonly agreed enterprise architecture layers, respectively for organizations and processes, systems functionalities, and platforms, with a fourth added for analytical models of business environments.

Conceptual Debt

Ideally, that alignment should pave the way to the integration of systems and knowledge architectures, as represented by the Pagoda blueprint:

Insofar as systems engineering is concerned, that would require two kinds of transformations: from conceptual to logical models (aka analysis), and from logical to physical models (aka design).

While the latter is just a matter of expertise (thank to the GoF), that’s not the case for the former which has to deal with a semantic gap between descriptions of specific and changing business domains and organizations on one side, generic and stable systems architectures on the other side.

As a result, what can be termed a conceptual debt has accumulated with the the number of logical models supporting physical ones without the backing of relevant ones for business or organization. The objective is therefore to bring all models into a broader knowledge architecture.

Models & Ontologies

As introduced by Greek philosophers, ontologies are systematic accounts of whatever is known about a domain of concern. From that point, three basic observations can be made:

  1. Ontologies are made of categories of things, beings, or phenomena; as such they may range from lexicon or simple catalogs to philosophical doctrines.
  2. Ontologies are driven by cognitive (i.e non empirical) purposes, namely the validity and consistency of symbolic representations.
  3. Ontologies are meant to be directed at specific domains of concerns, whatever their epistemic nature: engineering, business, politics, religions, mythologies, astrology, etc.

With regard to models, only the second observation puts ontologies apart: compared to models, ontologies are about understanding and are not necessarily driven by empirical purposes.

On that account ontologies appear as an option of choice for the integration of symbolic representations:

  • Data: instances identified at territory level, associated with terms or labels; they are mapped to business intelligence (environments) and operational (systems) models.
  • Information: categories associated with sets of instances; categories can be used for requirements analysis or software design.
  • Knowledge: ideas or concepts connect changing and overlapping sets of terms and categories; documents can be associated to any kind of item.

With models consistently mapped to ontologies, the conceptual debt could be restructured in the broader context of enterprise knowledge architecture.

Ontologies & Knowledge

As expounded by Davis, Shrobe, and Szolovits in their pivotal article, knowledge is made of five constituents:

  1. Surrogates, used as symbolic counterparts of actual objects and phenomena.
  2. Ontological commitments defining the categories of things that may exist in the domain under consideration.
  3. Fragmentary theory of intelligent reasoning defining what things can do or can be done with.
  4. Medium making knowledge understandable by computers.
  5. Medium making knowledge understandable by humans.

Points 1 and 5 are not concerned by the conceptual gap, the former being dealt with through the anchoring of identified individuals to surrogates (see below), and the latter being with human interfaces. That leaves points 2-4 as the conceptual hub where information models have to be integrated into knowledge architecture.

Assuming RDF (Resource Description Framework) graphs are used for knowledge representation (point 4), and taking a restaurant for example, the contents of information models (point 2) will be denoted by:

  • Primary nodes (rectangles), for elements specific to cooking and customers relationship management, to be decorated with features (bottom right).
  • Connection nodes (circles and arrows), for semantically neutral (aka syntactic) associations to be uniformly implemented across domains, e.g with predicate calculus (bottom left).
  • Semantic connectors supporting both syntactic and semantic associations (bottom, middle). 

Inserting information into knowledge architecture

Using ontologies to integrate models into knowledge architecture is to enable the restructuring of the conceptual debt.

Minding Semantic Gaps

Keeping with the financial metaphor, conceptual debts can be expressed in terms of spreads between models, and as such could be restructured through models transformation.

To begin with, all representations have to be anchored to environments through identified (#) instances.


Anchoring systems to their environment

Then, instances are to be associated to categories according to features
(properties or relationships) :

  • Customers, reservations, tables, and waiters are identified individuals managed through symbolic surrogates.
  • Names of dishes and ingredients do not refer to symbolic surrogates representing business objects, but are just labels pointing to recipes (documents).
  • Idem for the names of wines, except for exceptional vintages with identified bottles to be managed through symbolic surrogates.

As defined above, these models can be equivalently expressed as ontologies:

  • Properties are single-valued attributes.
  • Relationships define links between categories.
  • Aspects are structured sets of features meant to be valued through category instances.
  • Documents are contents to be accessed directly or through networks, (e.g preparations or wine reviews).
Fleshing out model backbone with features, relationships, and documents (black, italic)

It must be noted that the distinction between neutral and specific contents is not meant to be universal but be justified by pragmatic concerns, for instance:

  • Addresses are not defined as aspects but as category instances so that surrogates of actual addresses can be used to optimize deliveries.
  • Links to customers and addresses, being self-explanatory, can be defined as non specific.
  • The relationship from dishes to ingredients is structured and specific.

Sorting out truth-preserving constructs from domain specific ones is a key success factor for models transformation, and consequently debt restructuring.

Restructuring The Debts

Restructuring financial debts means redefining assets and incomes; with regard to systems it would mean reassessing architectures with regard to value chains.

To begin with, the Pagoda blueprint central pillar is to support the integration of systems and knowledge architectures and consequently the dynamic alignment of systems capabilities, meant to be stable and shared, with business opportunities, by nature changing and specific.

Then, the pairing of systems and knowledge architectures, like a DNA double helix, is to be used to restructure both technical and conceptual debts.


Pairing assets and incomes across architectures

With regard to technical debts, restructuring isn’t to present significant difficulties:

  • Pairing income flows (applications) to tangible assets (platforms) can be done at data level.
  • Model transformations between data (code) and information (models) levels can be achieved using homogeneous domain specific and programming languages.

Things are more complex with conceptual debts, for pairing as well as transformations:

  • There is no direct pairing because value chains (processes) are set across assets (organization).
  • Model transformations are to bridge the semantic gap between the
    symbolic representations of environments (knowledge) and systems (information) .

Nonetheless, these difficulties can be overcame combining integrated architectures and ontologies.

Regarding the structure of the conceptual debt, the income part is to be defined through business objectives (customers, products, channels, supply chain, etc.), and assets to be defined by corresponding enterprise architectures capabilities.

How to mind the gap between external and systems representations.

Regarding models transformations, ontologies will be used to mind the semantic gap between environments (knowledge) and systems (information) representations:

  • Power-types: describe instances of categories (age, income, education, …).
  • Specialization and generalization: defined with regard of intent, subsets for individuals (wine, gender), sub-types for aspects (temperature, serve in menu).
  • Knowledge based relationships (dashed line): used to describe objects and phenomena, actual, planned, or expected (face recognition of customers, influence of weather on dishes, association of wines and dishes, …
  • Concepts: introduced to relate information and knowledge: gourmet.
Ontological descriptions

With the backbones of symbolic representations soundly anchored to environment, it would be possible to complement functional and logical models with their conceptual counterpart and by doing so to eliminate conceptual debts. A symmetric policy could be applied to refactoring in order to redeem the technical debt associated to legacy code.

Managing Conceptual Debt

Like financial ones, conceptual debts are facts of life that have to be managed on a continuous basis. That can be achieved using Open Concepts, which may also leverage the restructuring of technical debts (TD) and consequently further in-depth digital transformation.

Using Open Concepts to consolidate technical and conceptual debts

Ontologies would then be used to ensure:

  • A separate management of models directly tied to systems, and ontologies with broader justification.
  • A distinction between a kernel (aka knowledge engine), environment profiles, and business domains.
EA & Knowledge Management

Further Reading