The ldh CLI drives the same HTTP API as the UI and covers most UI actions

The LinkedDataHub CLI wraps the HTTP API into a single executable with convenient parameters. It can be used for testing, automation, scheduled execution and similar tasks. Performing actions with the CLI is usually quicker than using the user interface, and easier to reproduce.

Some commands correspond to a single request to LinkedDataHub, others combine several into tasks with multiple interdependent requests, such as the CSV import.

You will need your WebID certificate — either the .p12 keystore or a PEM file holding it with its private key — as well as its password, among other arguments.

Install

Every LinkedDataHub release attaches an ldh-<version>.tar.gz archive holding the launcher and the jar. It needs a Java 21 runtime and nothing else:

tar -xzf ldh-<version>.tar.gz
export PATH="$PWD/ldh-<version>:$PATH"

ldh --help

To build from source instead, run make cli from the root of the LinkedDataHub repository, which prints the export line to run afterwards. It requires Java 21 and Maven and is the equivalent of:

cd cli
mvn package
export PATH="$PWD/bin:$PATH"
cd ..

ldh --help

Either way, the self-contained ldh.jar is what the ldh launcher runs, and ldh --version reports the version it was built at. The CLI carries the same version as the platform.

If you use the CLI regularly, add the export line to your shell profile. Shell completion for bash and zsh is generated by the CLI itself:

source <(ldh generate-completion)

The CLI talks to the HTTP API directly and embeds Apache Jena. Unlike the shell scripts it replaces, it needs no curl, python or Jena command line tools on $PATH.

Deprecated shell scripts

The HTTP API shell scripts in the bin folder that ldh replaces are deprecated. The certificate and WebID tooling (webid-keygen.sh, webid-keygen-pem.sh, webid-uri.sh, webid-modulus.sh, server-cert-gen.sh) talks to no API and is not deprecated.

Migrating from a script to its command means dropping the .sh suffix, prefixing the invocation with ldh (the hyphenated name becomes a verb-object pair, e.g. admin/acl/create-group.sh becomes ldh admin create group), and passing the .p12 keystore to -f instead of the .pem certificate.

Authentication

Commands authenticate with a WebID client certificate, read either from a PKCS12 keystore — the format produced by bin/webid-keygen.sh and shipped as ssl/owner/keystore.p12 for the owner agent — or from a PEM file holding the certificate together with its private key, such as ssl/owner/cert.pem. The format is recognised from the file's content, not from its extension:

ldh get \
  -c ./ssl/owner/keystore.p12 \
  -p "$owner_cert_password" \
  --accept text/turtle \
  "https://localhost:4443/"

which, on the Northwind Traders dataspace, prints the root document's description:

@prefix def:  <https://w3id.org/atomgraph/linkeddatahub/default#> .
@prefix dct:  <http://purl.org/dc/terms/> .
@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

<https://localhost:4443/> a def:Root ;
    dct:title "Northwind Traders" ;
    dct:description "Knowledge Graph representation of the Northwind Traders sample database" ;
    rdf:_1 <https://localhost:4443/#overview-intro> ;
    # …
    .

The CLI does not validate TLS server certificates, so self-signed development instances work out of the box. Do not point it at untrusted hosts.

Parameters

Parameters shared by most commands:

Parameters every command accepts
Parameter Value
-c, --cert PKCS12 keystore, or PEM file with the certificate and its private key, holding the WebID certificate of the agent
-p, --cert-password Password of the keystore, or passphrase of an encrypted PEM key. Not needed when the PEM key is unencrypted
-b, --base Base URI of the dataspace
--proxy The host the request is proxied through (optional). Use it with port 5443, on which client certificate authentication is always enabled — for example --proxy https://localhost:5443/.

Other parameters are command-specific. Most commands take the URI of the document they act on as a default (positional) parameter.

Environment variables

The repeated parameters can be set once in the environment instead of being passed to every command:

Environment variables and the parameters they supply
Variable Parameter
LDH_CERT_FILE -c, --cert
LDH_CERT_PASSWORD -p, --cert-password
LDH_BASE -b, --base
LDH_PROXY --proxy
export LDH_CERT_FILE=./ssl/owner/keystore.p12
export LDH_CERT_PASSWORD="$owner_cert_password"
export LDH_BASE=https://localhost:4443/

ldh create container --parent "$LDH_BASE" --title "Categories" --slug categories

Output

Commands that create or append to a document print its URL as the only line on standard output, so they compose in shell pipelines. All diagnostics go to standard error.

item=$(ldh create item --container "$LDH_BASE" --title "Example" --slug example)

ldh push writes many documents in one run and prints one line per written document URL or upload URI, in write order; progress and skipped entries go to standard error, and the first failed request stops the run.

Exit codes are 0 on success, 1 on an HTTP error status or runtime failure (the message goes to standard error; add --verbose for a stack trace), and 2 on a usage error.

Usage

A usage message with the parameters of a command is printed when the command is run without required arguments, or with --help. Commands take named parameters and default (positional) parameters; either kind can be optional. For example:

$ ldh add select
Missing required options and parameters: '--title=TITLE', '--query-file=ABS_PATH', 'TARGET_URI'
Usage: ldh add select [-h] [--verbose] [-b=BASE_URI] [-c=CERT_FILE]
                      [--description=DESCRIPTION] [-p=PASSWORD]
                      [--proxy=PROXY_URL] --query-file=ABS_PATH
                      [--service=SERVICE_URI] --title=TITLE [--uri=URI]
                      TARGET_URI
Adds a SELECT query to a document.
      TARGET_URI          URI of the document
  -b, --base=BASE_URI     Base URI of the application (env: LDH_BASE)
  -c, --cert=CERT_FILE    PKCS12 keystore or PEM with the WebID certificate and
                            private key of the agent (env: LDH_CERT_FILE)
      --description=DESCRIPTION
                          Description of the query (optional)
  -h, --help              Show this help message and exit.
  -p, --cert-password=PASSWORD
                          Password of the keystore, or passphrase of an
                            encrypted PEM key (env: LDH_CERT_PASSWORD)
      --proxy=PROXY_URL   The host this request will be proxied through
                            (optional) (env: LDH_PROXY)
      --query-file=ABS_PATH
                          Path to the file with the query string
      --service=SERVICE_URI
                          URI of the SPARQL service (optional)
      --title=TITLE       Title of the query
      --uri=URI           URI of the query (optional, blank node if not set)
      --verbose           Print stack traces of errors

The optional parameters are marked with (optional). In this case the default parameter is the URI of the document the query is appended to.

An add select invocation looks like this (with the environment variables supplying the connection parameters):

ldh add select \
  --title "Products by sales" \
  --query-file products-by-sales.rq \
  "${LDH_BASE}products/"

Commands

Commands group by verb — create, add and remove make the single request they name, while import holds the composite workflows — plus the packages family and the admin scope. The following commands are currently supported:

Commands grouped by purpose
Purpose Command
Low-level (Graph Store Protocol)
GET request ldh get
POST request ldh post
PUT request ldh put
PATCH request ldh patch
DELETE request ldh delete
Documents
Create container document ldh create container
Create item document ldh create item
Push a directory of RDF files and uploads into the document tree it maps to ldh push
Content
Append object block (instance of ldh:Object) to document ldh add object-block
Append XHTML block (instance of ldh:XHTML) to document ldh add xhtml-block
Remove block from document ldh remove block
Instances of system classes
Append uploaded file (instance of nfo:FileDataObject) to document ldh add file
Append CONSTRUCT query (instance of sp:Construct) to document ldh add construct
Append service (instance of sd:Service) to document ldh add generic-service
Append result set chart (instance of ldh:ResultSetChart) to document ldh add result-set-chart
Append SELECT query (instance of sp:Select) to document ldh add select
Append SPARQL view (instance of ldh:View) to document ldh add view
Imports
Create CSV import ldh add csv-import
Import CSV data ldh import csv
Create RDF import ldh add rdf-import
Import RDF data ldh import rdf
Administration
Add owl:imports to ontology ldh admin add ontology-import
Clear the ontology cache, reloading one ontology if named ldh admin clear ontology
Access control
Add agent to group ldh admin add agent
Create authorization ldh admin create authorization
Create group ldh admin create group
Make the dataspace publicly readable by any agent ldh admin make-public
Ontologies
Add class ldh admin add class
Add CONSTRUCT query ldh admin add constructor
Create ontology ldh admin create ontology
Add property constraint ldh admin add property-constraint
Add restriction ldh admin add restriction
Add SELECT query ldh admin add select
Import ontology ldh admin import ontology
Packages
List available packages ldh packages list
Import a package ldh packages add
Drop a package import ldh packages remove

Usage example. The Northwind categories CSV is uploaded into a document, so create the document first:

doc=$(ldh create item \
  --container "${LDH_BASE}categories/" \
  --title "Categories data" \
  --slug categories-data)

ldh add file \
  --title "categories.csv" \
  --file categories/categories.csv \
  --content-type text/csv \
  "$doc"

add file prints the content-addressed URI of the upload ({base}uploads/{sha1}) rather than the document URI — the Data stage of the tutorial feeds exactly that URI into a CSV import.

See also the data import user guides.

Request bodies

ldh put and ldh post read RDF from an optional FILE argument, recognizing the syntax from its extension (ldh put "$LDH_BASE" root.ttl), or from standard input, where -t/--content-type is required since a stream carries no name. Either way relative URIs resolve against the target URI. ldh patch reads a SPARQL 1.1 update from standard input, validates it and sends it verbatim.

What a document is about

ldh create item and ldh create container take --primary-topic, the resource the document is about, resolved against the created document's URI: '#this' names a fragment of the document itself, an absolute URI a resource described somewhere else. It writes the foaf:primaryTopic link only — the topic's own type and properties are the constructor's business, and a vocabulary may refuse a topic without them.

ldh create item \
  --container "${LDH_BASE}products/" \
  --title "Masala Chai" \
  --slug 78 \
  --primary-topic '#this'

Pushing a directory

ldh push replays a directory into the document tree it maps to — the app as a repository convention: one RDF file per document, a folder beside it for the files that document holds, and subfolders for its children.

ldh push "$LDH_BASE"                              # the current directory
ldh push --dir demo/northwind-traders "$LDH_BASE"
ldh push --dry-run "$LDH_BASE"                    # print the plan, send nothing

With D the URL of the directory being walked (the target URI for the pushed directory itself):

  • root.ttl at the top of the tree is PUT to the target URI. It describes the target document itself, so there is no separate step for it.
  • Any other RDF file name.ext is PUT to D/name/, with its relative URIs resolved against that URL.
  • Every other file is uploaded into D, as add file would: the file name as the title, the media type detected, the upload URI {base}uploads/{sha1}.
  • A subdirectory name maps to D/name/: its document is the RDF file name.ext beside it, its files are uploaded into that document, and its subdirectories recurse.

A file is a document when Jena tells its RDF syntax from the file name and can parse it — .ttl, .nt, .rdf, .jsonld, .trig and the other registered extensions; .csv, .rq, .md, images and the like are uploads. Within a directory the order is the root document, then documents, then uploads, then subdirectories, each sorted by name, so a subdirectory's document exists before anything is uploaded into it. PUT replaces a document, so a repeated push converges: the document is rewritten and its uploads re-appended, not duplicated.

Entries whose name starts with . are never pushed. A .ldhignore file in any directory excludes entries from that directory's whole subtree, gitignore-style: a pattern without a / matches an entry's name at any depth beneath (*.sh, Makefile), a pattern with a / matches the path relative to the ignore file's directory (categories/unesco-mappings.ttl), and a trailing / restricts the pattern to directories (admin/). Nothing is excluded by default beyond hidden entries, so scripts and build output are listed there. Skipped entries are reported on standard error. --dry-run walks the tree, parses every document and prints the plan without sending a request or loading a certificate.

Versioned documents

ldh get also exposes a versioned document's TimeMap, TimeGate and historical versions through three mutually exclusive options — see the versioning reference for the options and examples.

Packages

Packages are managed with the packages group: a dataspace imports a package with a single ldh:import triple, which ldh packages add declares and ldh packages remove retracts on the dataspace's settings document, while ldh packages list reads the registry catalog and marks the imported ones. See the package management guide.

Find the CLI on GitHub, or see it in action in the tutorial and the demo apps.