Connecting data consumers with data sources

Industries and researchers often face the challenge of finding relevant data. Even when they obtain the data, understanding and utilizing it can be difficult. So, we have contributed to developing a framework which will help solve this problem.

The framework addresses these issues by implementing data documentation and semantic interoperability, which collectively enhances data management by making data easily accessible and interoperable across various platforms and applications. Developed as part of the OntoTrans project, the flexible and interoperable framework known as the Open Translation Environment (OTE) enables data consumers to represent their required data specification in a standard ontological form and connect them to relevant data sources, properly documented by the provider. OTE was successfully developed and tested for the materials modelling industry and its relevance was showcased to the Norwegian industry using the NOTE project for maritime applications, battery production, manufacturing and chemistry.

Data documentation:

The core component of OTE is the Ontology-based Translation Environment API (OTEAPI), a system for data documentation. OTEAPI provides a coherent, flexible, and reusable approach to data documentation at multiple levels: catalogue, structural, and semantic.

OTEAPI is implemented as a set of RESTful APIs, that connects data sinks (where the new data or output is stored) to data sources (where data originates). These connections are established through pipelines composed of modular components, or filters. These filters can perform tasks such as downloading, parsing, mapping, filtering, converting, and processing data. Importantly, ontologies are used to establish a single source of truth vocabulary. These data documentation pipelines can be reused and mixed and matched into a working data pipeline. Also, data movement happens only when the pipeline is executed. A standout feature of OTEAPI is its ability to document data sinks in the same manner as data sources, ensuring data consumers receive data in precisely the form needed.

Semantic interoperability:

The description of the source and sink is not sufficient for the data flow. A data conversion step is required. Semantic interoperability is achieved when the OTEAPI semantic data documentation system is exploited by an interoperability framework. The interoperability framework will find the path from the source to the sink through the knowledge graph. OTEAPI is agnostic to the specific interoperability frameworks, ensuring adaptability to various standards. In the OntoTrans project, OTEAPI utilized DLite, a lightweight interoperability framework based on simplistic data models, ensuring efficient data documentation and management across diverse use cases.

While these tools were developed for the materials modelling domain, their applications can extend to various other industries. By providing a standardized approach to data documentation and access, these tools will enable industries across domains to utilize their data efficiently.