RDF imports and transforming RDF using SPARQL CONSTRUCT

An RDF import is a combination of multiple resources:

File: The RDF file to be imported
Transformation query (optional): A user-defined CONSTRUCT query that transforms the input RDF into another RDF
Graph (optional): The target graph (document) to which converted items will be POSTed

Either the transformation query or the target graph needs to be specified, not both.

If the graph is specified, the resulting RDF data is appended to a single document.

If the transformation query is specified, the resulting data should contain document instances in order to attach to the document hierarchy. The documents have to be URI resources. The document graphs have to be explicitly specified using a GRAPH block in the CONSTRUCT template (which is a\n Jena-specific extension of SPARQL 1.1), otherwise the import result will end up in the default graph of the application's RDF dataset, which is not accessible via LinkedDataHub.

The import process runs asynchronously in the background, i.e. the import item is created before the process completes.\n Currently the only way to determine when it completes is to refresh the import item and check the import\n status (completed/failed). Upon successful report, metadata such as the number of imported RDF triples is attached\n to the import.

The resulting RDF is split into documents (named graphs), which are then created one by one and validated against constraints in the process.\n Constraint violations, if any, are attached to the import item.

Lets assume we want to import SKOS concept data:

@prefix : <http://vocabularies.unesco.org/thesaurus/> .\n@prefix skos: <http://www.w3.org/2004/02/skos/core#> .\n@prefix dcterms: <http://purl.org/dc/terms/> .\n@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n\n:concept10 a skos:Concept ;\n    dcterms:modified \"2006-05-23T00:00:00\"^^xsd:dateTime ;\n    skos:inScheme <http://vocabularies.unesco.org/thesaurus> ;\n    skos:narrower :concept4938 , :concept7597 ;\n    skos:prefLabel \"Right to education\"@en , \"Droit à l'éducation\"@fr , \"Derecho a la educación\"@es , \"Право на образование\"@ru ;\n    skos:related :concept9 , :concept556 , :concept557 , :concept1519 , :concept5052 ;\n    skos:topConceptOf <http://vocabularies.unesco.org/thesaurus> .\n\n:concept1000 a skos:Concept ;\n    dcterms:modified \"2006-05-23T00:00:00\"^^xsd:dateTime ;\n    skos:broader :concept389 ;\n    skos:inScheme <http://vocabularies.unesco.org/thesaurus> ;\n    skos:prefLabel \"Talent\"@en , \"Talent\"@fr , \"Talento\"@es , \"Талант\"@ru ;\n    skos:related :concept993 , :concept996 , :concept3086 .\n\n:concept10003 a skos:Concept ;\n    dcterms:modified \"2006-05-23T00:00:00\"^^xsd:dateTime ;\n    skos:altLabel \"Entrevue\"@fr ;\n    skos:broader :concept4725 ;\n    skos:inScheme <http://vocabularies.unesco.org/thesaurus> ;\n    skos:prefLabel \"Interviews\"@en , \"Entretien\"@fr , \"Entrevista\"@es , \"Интервью\"@ru .

Transformation

This step is used to transform the RDF data that is being imported, if necessary (to a different vocabulary, for example). It also connects instances in the imported data to the documents in LinkedDataHub's dataset.

The mapping is a user-defined SPARQL CONSTRUCT. These are the rules that hold for mapping queries:

BASE value is automatically set to the imported file's URI. Do not add an explicit BASE to the query.
$base binding is set to the value of the application's baseURI
useOPTIONAL for optional values
use BIND() to introduce new values and/or cast literals to the appropriate result datatype or URI
when building document URIs, use natural IDs from the input data (or UUIDs if there are no IDs) and remember to URI-encode them using encode_for_uri
use a GRAPH block in the constructor template to construct triples for a specific document
construct dh:Container instances to create new container documents or dh:Item instances to create new item documents. dct:title values are mandatory for documents.
if you're constructing non-information resource (e.g. thing, concept) descriptions, assign them URIs with fragment identified (e.g. #this) and pair them with item documents using the foaf:primaryTopic property

We plan to provide a UI-based mapping tool in the future.

Example

In this example we pair each SKOS concept from the imported dataset with a new document:

PREFIX dh:    <https://www.w3.org/ns/ldt/document-hierarchy#>\nPREFIX sioc:  <http://rdfs.org/sioc/ns#>\nPREFIX foaf:  <http://xmlns.com/foaf/0.1/>\nPREFIX skos:  <http://www.w3.org/2004/02/skos/core#>\nPREFIX dct:   <http://purl.org/dc/terms/>\n\nCONSTRUCT\n{\n    GRAPH ?item\n    {\n        ?concept ?p ?o .\n\n        ?item a dh:Item ;\n            foaf:primaryTopic ?concept ;\n            sioc:has_container ?container ;\n            dh:slug ?id ;\n            dct:title ?prefLabel .\n    }\n}\nWHERE\n{\n    SELECT *\n    {\n        ?concept a skos:Concept .\n        BIND (STRAFTER(STR(?concept), \"http://vocabularies.unesco.org/thesaurus/\") AS ?id)\n        BIND (uri(concat(str($base), \"concepts/\")) AS ?container)\n        BIND (uri(concat(str(?container), encode_for_uri(?id), \"/\")) AS ?item)\n\n        ?concept ?p ?o\n        OPTIONAL\n        {\n            ?concept skos:prefLabel ?prefLabel\n            FILTER (langMatches(lang(?prefLabel), \"en\"))\n        }\n    }\n}

When the import is complete, you should be able to see the imported documents as children of the ${base}concepts/ container.

The result of our mapping:

@prefix : <http://vocabularies.unesco.org/thesaurus/> .\n@prefix skos: <http://www.w3.org/2004/02/skos/core#> .\n@prefix dcterms: <http://purl.org/dc/terms/> .\n@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\n@prefix sioc: <http://rdfs.org/sioc/ns#> .\n@prefix dh: <https://www.w3.org/ns/ldt/document-hierarchy#> .\n\n:concept10 a skos:Concept ;\n    dcterms:modified \"2006-05-23T00:00:00\"^^xsd:dateTime ;\n    skos:inScheme <http://vocabularies.unesco.org/thesaurus> ;\n    skos:narrower :concept4938 , :concept7597 ;\n    skos:prefLabel \"Right to education\"@en , \"Droit à l'éducation\"@fr , \"Derecho a la educación\"@es , \"Право на образование\"@ru ;\n    skos:related :concept9 , :concept556 , :concept557 , :concept1519 , :concept5052 ;\n    skos:topConceptOf <http://vocabularies.unesco.org/thesaurus> .\n\n<concepts/c499e66a-8036-4637-929d-0d809177883e/> a dh:Item ;\n    sioc:has_container <concepts/> ;\n    dh:slug \"c499e66a-8036-4637-929d-0d809177883e\" ;\n    dcterms:title \"Right to education\"@en ;\n    foaf:primaryTopic :concept10 .\n\n:concept1000 a skos:Concept ;\n    dcterms:modified \"2006-05-23T00:00:00\"^^xsd:dateTime ;\n    skos:broader :concept389 ;\n    skos:inScheme <http://vocabularies.unesco.org/thesaurus> ;\n    skos:prefLabel \"Talent\"@en , \"Talent\"@fr , \"Talento\"@es , \"Талант\"@ru ;\n    skos:related :concept993 , :concept996 , :concept3086 .\n\n<concepts/f41910fa-9077-4656-8f73-752fd923a79b/> a dh:Item ;\n    sioc:has_container <concepts/> ;\n    dh:slug \"f41910fa-9077-4656-8f73-752fd923a79b\" ;\n    dcterms:title \"Talent\"@en ;\n    foaf:primaryTopic :concept1000 .\n\n:concept10003 a skos:Concept ;\n    dcterms:modified \"2006-05-23T00:00:00\"^^xsd:dateTime ;\n    skos:altLabel \"Entrevue\"@fr ;\n    skos:broader :concept4725 ;\n    skos:inScheme <http://vocabularies.unesco.org/thesaurus> ;\n    skos:prefLabel \"Interviews\"@en , \"Entretien\"@fr , \"Entrevista\"@es , \"Интервью\"@ru .\n\n<concepts/2afb2e06-5081-4db1-9255-660fcd1b3ec8/> a dh:Item ;\n    sioc:has_container <concepts/> ;\n    dh:slug \"2afb2e06-5081-4db1-9255-660fcd1b3ec8\" ;\n    dcterms:title \"Interviews\"@en ;\n    foaf:primaryTopic :concept10003 .

If you are ready to import some RDF, see our step-by-step tutorial on creating an RDF import.