Leveraging a knowledge graph’s inference capabilities, organizations can extrapolate new data connections and explain any new connection they create.
Digital transformation is all the rage, and in most cases, the goal of digital transformation is to treat data like an asset. In some instances, that means monetizing data, and in others, the goal is to leverage data more efficiently to derive insight to make better decisions. However, in reality, both are hard to achieve. Digital transformation demands rapid insights from increasingly hybrid, varied, and changing data, but traditional data integration platforms were not designed for today’s environment. As a result, organizations can no longer keep up with data’s growing complexity, nor can they identify hidden relationships and connections between the data to uncover new opportunities. Increasingly, what’s needed is an inference capability that allows companies to bring different datasets together and analyze them to derive insights.
Driving growth and innovation in today’s complex world of never-ending data capture is only possible when IT teams can break free of rigid data structures and outdated integration styles.
Agility is key to business success, and enterprises are desperate to make data usable when it counts, not for answers tomorrow or next week, but for right now.
But deriving untapped value requires the ability to connect data based on its business meaning, irrespective of the format, source, or underlying tech. The sheer amount of data derived from machine learning and other sources require the ability to associate related information stored in disparate sources and then apply a rich web of relationships to discover new associations. This is key to realizing the promise of digital transformation. But how does one go about achieving this?
Moving from premises to logical consequences: How data fabrics provide needed inference
Organizations today are adopting modern integration approaches like data fabrics to power collaborative, cross-functional projects and products and escape reactive workflows. Weaving together data from internal silos and external sources, they create a network of information to power the business’ applications, Al, and analytics. Quite simply, they support the full breadth of today’s complex enterprise by creating connections between information stored in disparate sources.
Knowledge graphs are an integral part of an effective data fabric as they create a reusable network of information, represent data of various structures, and support multiple schemas. Creating the semantic understanding of enterprise and third-party data, knowledge graphs serve as the core of the data fabric– enriching and accelerating existing investments and providing critical access to business insight. More importantly, knowledge graphs turn data into machine-understandable, real-world knowledge that supports situational changes, so meaning alters depending on circumstances. Once established, the knowledge graph also uses this rich web of relationships to discover new associations within the data. These inferred relationships create a richer, more accurate view of an enterprise’s data.
By providing layered associations between concepts, knowledge graphs provide nuanced understanding so knowledge-driven organizations can identify new discoveries. They also provide the context that is often missing from data because the knowledge graph is purpose-built to support the fluctuating nature of knowledge. The result is a more flexible foundation for digital operations as the technology easily accepts new data, definitions, and requirements.
The knowledge graph’s data model, often called an ontology or vocabulary, lays out common relationships between entities and allows companies to describe complex domains. Consider medicine as an example. To develop a new therapy, pharmaceutical companies must have access to multiple facts, modeling constructs, and business rules, all of which must interact with each other to imply new connections. This inference capability is what makes it possible for manufacturers to link people to infrastructure via the applications they use. It also helps them to apply controls based on the similarity of new incidents to past incidents and find inferred links between investigators and therapeutic areas based on the conditions being investigated in studies. And the list goes on.
Applying multiple data models to a data fabric at the same time enables organizations to support multiple applications that require different interpretations of the same data. Traditional data integration approaches such as data lakes or data warehouses are limited in this capacity, as they make it difficult to support more than one schema. This is one reason enterprises have to continually create new data silos for each new application, project, or analysis. Such an approach reduces the ability to perform inference analysis.
Enabling the connected enterprise-additional components of a successful data fabric
Leveraging a knowledge graph’s inference capabilities, organizations not only extrapolate new data connections but also explain any new connection it creates. In contrast to black box recommendation systems, which cannot provide any explanation or rationale for their results, the knowledge graph can explain all inferences and results in terms of data, schema, and business rules. This explanatory transparency enables users to review how the knowledge graph arrived at an answer and the business logic referenced to do so. This is not only critical for providing trusted results and accountability within an organization but is also necessary for certain legal and regulatory requirements.
While a knowledge graph is the key ingredient of the data fabric, it is not the only thing an organization needs to be successful. An effective data fabric requires leveraging and connecting existing source systems. It also requires the ability to connect to existing data catalogs, data lakes, databases, and other data management platforms. For data fabric deployments, leveraging work completed in data catalogs is key to accelerating data discovery and semantic enrichment. Using the data catalog as an input, the knowledge graph builds a data map of an enterprise’s data assets which further accelerates data fabric creation through partially automated learning and auto-mapping of existing sources.
Creating an enterprise-wide data model is another common question regarding deploying a data fabric. Many think this is a potentially expensive and time-consuming prerequisite to the initiative, but, in reality, they only need to define as many concepts as needed for their initial use case. Start by identifying a critical business problem to spearhead the broader data fabric initiative. Approach the data fabric with an MVP mindset and focus only on the minimal amount of work needed to accomplish the first business objective.
Organizations of all sizes are placing an even greater focus and investment on digital transformation. Despite this renewed attention, fundamental data challenges remain a primary obstacle. Digital transformation requires data mastery and thanks to the legacy of IT is not a simple thing to achieve. There are just too many things to manage: data formats, standards, data types, velocities, schemas, systems, databases, silos, methodologies, models, etc. The sheer diversity of the modern enterprise IT landscape is daunting.
Leveraging the modern approach of knowledge graphs, organizations can not only connect their internal data silos in a meaningful new way, but they can also discover hidden facts and relationships through inferences that would otherwise be unable to catch on a large scale. By capturing the nuanced meaning that different business units may have for the same entity, organizations can create a reusable digital foundation that keeps pace with continued shifts in the market and be prepared for whatever comes next.
