Backlinks
Environment variables, secrets and RDF configuration files
This page covers the RDF configuration files, Compose overrides, and the environment variables and secrets of each service (defined in the environment sections of docker-compose.yml).
System base URI¶
The system base URI is the URI at which the LinkedDataHub service is accessible.
A common case is changing the system base URI from the default https://localhost:4443/ to your own. Take https://ec2-54-235-229-141.compute-1.amazonaws.com/ as an example. Split the URI into components and set them in the .env file:
PROTOCOL=https HTTP_PORT=80 HTTPS_PORT=443 HOST=ec2-54-235-229-141.compute-1.amazonaws.com
A dataspace serves its documents from the root of its origin, so the base URI is always the origin followed by / — there is no sub-path component to configure. Serving several dataspaces from one instance is a matter of giving each its own subdomain, as described under Dataspaces.
The administration dataspace of each dataspace is served on the admin. subdomain
of its end-user host (e.g. admin.ec2-54-235-229-141.compute-1.amazonaws.com). The subdomain must resolve in DNS and be covered by the
server's TLS certificate, otherwise the admin dataspace will not be reachable.
Dataspace URIs need to be relative to the system base URI in order to be reachable — after the change, the origins in config/dataspaces.trig have to follow. For the Northwind Traders dataspace, deploying from local development to the demo host means:
# before
<urn:linkeddatahub:apps/northwind-traders/end-user> a lds:Dataspace ;
lds:origin <https://northwind-traders.demo.localhost:4443> ;
lds:ontology <https://northwind-traders.demo.localhost:4443/ns#> .
# after
<urn:linkeddatahub:apps/northwind-traders/end-user> a lds:Dataspace ;
lds:origin <https://northwind-traders.demo.linkeddatahub.com> ;
lds:ontology <https://northwind-traders.demo.linkeddatahub.com/ns#> .Restart the services (make down, then make up) for the changes to take effect. Note that changing the base URI does not rewrite URIs already stored in the dataset — documents minted under the old base URI keep it.
Configuration files¶
LinkedDataHub uses two main RDF configuration files that define dataspaces and services:
- config/dataspaces.trig
- Contains public metadata for each dataspace, including:
-
- Base URIs and origins
- Dataspace titles and descriptions
- Associated ontologies
- Custom stylesheets
- This file contains public-facing metadata and can be safely shared.
- config/system.trig
- Contains internal deployment wiring, including:
-
- Dataspace-to-service bindings (admin and end-user roles)
- SPARQL endpoint URLs
- Graph Store Protocol endpoints
- This file contains internal configuration and is not intended for public sharing, but does not contain credentials.
- secrets/credentials.trig
- Optional file containing service authentication credentials, including:
-
- Bearer tokens (
a:authToken) - HTTP Basic auth credentials (
a:authUser,a:authPwd)
- Bearer tokens (
- This file is gitignored and must not be committed to version control. See the credentials secret entry below for configuration details.
All files are in TriG format and are mounted into the LinkedDataHub container at startup. The separation allows you to version control dataspace metadata and service wiring while keeping credentials out of version control entirely. The dataspace reference shows the Northwind Traders dataspace as a worked example across both config/ files.
Compose overrides¶
Environment-specific configuration goes into docker-compose.override.yml, which Docker Compose merges over docker-compose.yml automatically. Mounting a stylesheet under development, for example:
services:
linkeddatahub:
volumes:
- ./files/northwind.xsl:/usr/local/tomcat/webapps/ROOT/static/com/atomgraph/linkeddatahub/northwind/xsl/layout.xsl:roCompose-level variables¶
The variables in the .env file are interpolated by Docker Compose into docker-compose.yml. They define the system base URI and the host port mappings:
- PROTOCOL
- URI scheme of the system base URI (http or https)
- HOST
- Hostname of the system base URI
- HTTP_PORT
- Host port mapped to the container's port 8080, which redirects HTTP to HTTPS
- HTTPS_PORT
- Host HTTPS port, mapped to the container's port 8443
Service configuration¶
SPARQL service endpoints are configured in config/system.trig. See service configuration in the triplestores reference for the RDF properties and examples, and the dataspace reference for the conceptual overview.
linkeddatahub service¶
Secrets¶
- owner_cert_password
- Password of the owner's WebID certificate
- secretary_cert_password
- Password of the secretary's WebID certificate
- client_truststore_password
- Password of the client truststore
- google_client_id
- OAuth client ID
- Login with Google authentication is enabled when this value is provided
- google_client_secret
- OAuth client secret
- orcid_client_id
- ORCID OpenID Connect client ID
- Login with ORCID authentication is enabled when this value is provided
- The client is expected to be registered on production ORCID (https://orcid.org). For a client registered on ORCID's sandbox, set the ORCID_ISSUER environment variable to https://sandbox.orcid.org: the authorization, token and key endpoints derive from it, and only ID tokens from that issuer are accepted
- orcid_client_secret
- ORCID OpenID Connect client secret
- credentials
- RDF dataset file (./secrets/credentials.trig) containing service authentication credentials (optional)
- Supports HTTP Basic authentication (
a:authUser,a:authPwd) and Bearer token authentication (a:authToken) - See authentication in the triplestores reference for RDF examples
WebID authentication¶
- ENABLE_WEBID_SIGNUP
- false to disable. Enabled by default.
-
Currently this will only hide the signup button in the UI, without disabling the endpoint.
- WEBID_CACHE_EXPIRATION
- Expiration time (in seconds) of cached WebID profiles, bounding how long a revoked WebID stays authenticated. Defaults to 86400.
- JWKS_CACHE_EXPIRATION
- Expiration time (in seconds) of cached JWKS keys used for OpenID Connect token verification. Defaults to 86400.
Email server¶
- MAIL_SMTP_HOST
- Hostname of the email server
- MAIL_SMTP_PORT
- Port number of the email server
- MAIL_USER
- Username
- MAIL_PASSWORD
- Password (if required)
Linked Data¶
ENABLE_LINKED_DATA_PROXY takes false to disable the Linked Data proxy, which is enabled by default.
Assistant¶
COMPOSE_PROFILES, the Compose variable, is passed to the platform too. When it includes assistant, which also starts the web-algebra service, the platform offers the assistant: it publishes lds:assistant true for every dataspace in its /context, and documents show the Assistant button. The assistant is disabled by default.
Sitemap¶
ENABLE_SITEMAP takes true to serve the sitemap.xml of every end-user dataspace, which is disabled by default: its queries walk every public document of the dataspace. Without it, every robots.txt disallows crawling. The sample .env turns it on, since a new install is small; a large deployment can leave it unset.
HTTP(S)¶
- SELF_SIGNED_CERT
- Set to false if you are not using the self-signed server certificate (e.g. using a LetsEncrypt certificate instead). Not to be confused with the WebID client certificate. Defaults to true.
- MAX_CONTENT_LENGTH
- Maximum allowed request body size, in bytes; a larger body is refused with 413 Payload Too Large. Defaults to 2097152.
- MAX_UPLOAD_LENGTH
- Maximum size, in bytes, of a file upload (a
multipart/form-datarequest body); a larger one is refused with 413 Payload Too Large. Defaults to MAX_CONTENT_LENGTH. nginx limits request bodies to this value when it is set, and to MAX_CONTENT_LENGTH otherwise. - MAX_QUERY_RESPONSE_LENGTH
- The longest query result, in bytes, that the /sparql endpoint answers a query with; a longer one is refused with 502 Bad Gateway. Counted on the triplestore's response to the platform, not on what the caller receives. Defaults to 5242880 (5 MiB).
- HTTP_MAX_THREADS
- Maximum request threads on Tomcat's plain HTTP connector, the one the proxy forwards to. Defaults to Tomcat's own 200. Beyond capacity it bounds how many server-side renders can be in flight against each other: a render calls back into this same connector through the proxy for the page's labels, so it holds one thread while it waits for another to answer — see the HTTP client pool.
Debug¶
- JPDA_ADDRESS
- The address through which the Java debugger can connect, for example *:8000. Note that the port has to be mapped to the host in order for the debugger to work, e.g. 8000:8000.
- CATALINA_OPTS
- Tomcat's Java options
Proxy¶
- LDHC_FRONTEND_PROXY
- Frontend proxy URL for HTTP requests (optional)
- Configures a proxy server for the HTTP client infrastructure layer when making frontend requests
- LDHC_BACKEND_PROXY
- Backend proxy URL for SPARQL service access (optional)
- Configures a proxy server for accessing SPARQL services and backend endpoints
HTTP client timeouts¶
Timeouts and connection lifetimes for LinkedDataHub's pooled HTTP clients, used for the Linked Data proxy and for accessing SPARQL services. All values are in milliseconds and are passed as CATALINA_OPTS system properties.
- CLIENT_SOCKET_TIMEOUT
- Socket (read) timeout — how long to wait for data on an established connection. Defaults to 120000.
- CLIENT_CONNECT_TIMEOUT
- Connection timeout — how long to wait to establish a connection. Defaults to 10000.
- CONNECTION_REQUEST_TIMEOUT
- How long a request waits for a free connection from the pool before it gives up. Defaults to 30000; left unset the underlying client waits indefinitely, and a thread parked on the pool never comes back on its own.
- CLIENT_SELF_REQUEST_TIMEOUT
- Connect and read timeout for requests the platform makes to its own URLs, such as a server-side render fetching its labels from the dataspace's /sparql and /ns. Those are answered by the platform's own request threads, so a wait that is too long lets a burst of renders hold every thread waiting on each other; after this bound the render gives up on the label. Defaults to 5000.
- CLIENT_CONNECTION_TIME_TO_LIVE
- Maximum lifetime of a pooled connection before it is closed. Defaults to 300000.
- CLIENT_VALIDATE_AFTER_INACTIVITY
- Idle time after which a pooled connection is validated before reuse. Defaults to 10000.
HTTP client pool¶
The size of the connection pool those clients share, and how often they retry. The entrypoint passes them to the platform as CATALINA_OPTS system properties, like the timeouts above.
- MAX_CONN_PER_ROUTE
- Maximum pooled connections per host. Defaults to 200.
- MAX_TOTAL_CONN
- Maximum pooled connections in total. Defaults to 400.
- MAX_REQUEST_RETRIES
- Maximum number of times a failed request is retried. Defaults to the HTTP client's own retry policy.
Keep MAX_CONN_PER_ROUTE at least as large as HTTP_MAX_THREADS. A request the platform makes to its own URL holds a pooled connection for as long as it waits for one of the same Tomcat's threads to answer it, so a pool smaller than the connector queues those calls behind each other and a burst of renders drains one bounded wait at a time. The image's defaults are sized to each other; changing one means changing the other.
Varnish services¶
The backend ports of the varnish-frontend, varnish-admin and varnish-end-user caching services can be customized when running LinkedDataHub behind additional proxies or in non-standard Docker networking configurations:
- VARNISH_FRONTEND_BACKEND_PORT
- Port for frontend Varnish backend. Defaults to 7070.
- VARNISH_ADMIN_BACKEND_PORT
- Port for admin Varnish backend. Defaults to 3030.
- VARNISH_END_USER_BACKEND_PORT
- Port for end-user Varnish backend. Defaults to 3030.
The varnish-admin and varnish-end-user services keep SPARQL results in a store of their own, apart from the main one:
- VARNISH_SPARQL_STORAGE
- Size of the store SPARQL results are kept in, in Varnish's size syntax. A result too large for it ends the fetch, and the triplestore abandons the query, so it also bounds what one runaway query costs. Defaults to 64M.
varnish-frontend also caches anonymous SPARQL queries on their body, so the same query from different visitors reaches the platform once.
fuseki service¶
A single Apache Jena Fuseki server holds a TDB2 dataset for every dataspace role — an end-user and an admin dataset per dataspace — declared in config/fuseki/config.ttl and persisted under the fuseki/ folder, one sub-folder per dataset. JAVA_OPTIONS carries the Java options of the Fuseki process, and is what sizes its heap.
Datasets are named after the dataspace origin with the deployment host dropped and the role appended — northwind-traders.demo.end-user, northwind-traders.demo.admin — so the names hold across development and production; the root dataspace's are plain end-user and admin. See the dataspace reference for the service declarations that bind a dataspace to its datasets.
egress service¶
A forward proxy (Squid) that the outbound SPARQL SERVICE and LOAD requests of both Fuseki and the platform pass through. It permits public destinations
and refuses loopback, private and link-local ones, resolving each where it connects
so redirect hops and DNS answers are checked too. Federation with public endpoints
keeps working, while a SERVICE clause cannot reach another dataset, the cache or the platform.
Fuseki is pointed at it through its JAVA_TOOL_OPTIONS; the platform's own in-process queries (imports and PATCH updates) are routed through
it when EGRESS_PROXY is set to its host:port. Without a proxy and without ALLOW_INTERNAL_URLS, the platform disables SERVICE in those queries rather than leaving it open.
sef-compiler service¶
Compiles each dataspace's client-side stylesheet together with the stylesheets of
its imported packages into a SEF that the browser runs, so a package's rendering reaches the client as
well as the server. The platform composes the wrapper and submits it; the compiled
SEF is written to the sef bind mount (/var/www/linkeddatahub/sef, served under /static/xsl/sef/) and its URL travels to the page as a Link header. The service is built from the same Dockerfile as the platform, so its copy
of the stylesheets is the deployed one.
SEF_ROOT and SEF_COMPILER reach the platform through CATALINA_OPTS and are optional: without them packages compose server-side only, and the client runs the stock stylesheet. A deployment with its own compose file adds the service, the platform's depends_on on it and the bind mount.
CLIENT_STYLESHEET names the client stylesheet the SEF is composed from and defaults to the platform's
client.xsl. A deployment whose page bootstraps its own client stylesheet sets it to that stylesheet's
webapp path - it must import the platform's client.xsl directly, by an href that resolves from that path - and its layout prefers ldh:client-stylesheet() over its own SEF; otherwise the site's rules vanish from a package dataspace on the
first client-side navigation, while a reload brings them back.
web-algebra service¶
Writes and runs the assistant's Web-Algebra plans.
nginx forwards /webalgebra to it with the caller's client certificate, and the service is not exposed otherwise.
It runs a plan as
the caller by presenting the secretary's certificate with an On-Behalf-Of header, and reaches the instance through nginx's client
certificate port. Its outbound requests to public destinations, such as the model's
API or a SPARQL endpoint a plan names, go through
egress.
The service is opt-in: it belongs to the assistant Compose profile and starts only with COMPOSE_PROFILES=assistant in .env, which the platform reads as well. Its image, ghcr.io/atomgraph/webalgebra-server-ldh, is proprietary: AtomGraph publishes the image but not its source, and LinkedDataHub's license does not cover it.
Secrets¶
- secretary_cert_password
- Password of the secretary's WebID certificate, which the service presents to the instance
- openai_api_key
- OpenAI API key the service writes plans with (optional). Declared in the override together with OPENAI_API_KEY_FILE, as shown in the assistant setup. Without it the service runs plans but does not write them
Environment¶
- WEBALGEBRA_VERSION
- Compose-level variable: the tag of the ghcr.io/atomgraph/webalgebra-server-ldh image. Defaults to latest
- OPENAI_API_KEY_FILE
- Path of the openai_api_key secret inside the container, /run/secrets/openai_api_key
- OPENAI_MODEL
- The model plans are written with. Defaults to gpt-4o
- WEBALGEBRA_TRUSTED_ORIGIN
- The origin the service passes the caller's identity to: this instance, with every dataspace subdomain under it. Set from PROTOCOL, HOST and HTTPS_PORT
- WEBALGEBRA_PROXY_HOST, WEBALGEBRA_PROXY_PORT
- Where requests for the trusted origin are sent inside the Docker network: nginx's client certificate listener, nginx:9443
- WEBALGEBRA_KEYSTORE, WEBALGEBRA_KEYSTORE_PASSWORD_FILE
- The secretary's PKCS12 keystore and its password file
- WEBALGEBRA_TRUSTSTORE
- The server certificate, as PEM, which the service trusts when it connects to nginx
nginx service¶
- SERVER_CERT_FILE
- Location of the server's SSL certificate. Defaults to /etc/nginx/ssl/server.crt.
- SERVER_KEY_FILE
- Location of the server's SSL certificate's key. Defaults to /etc/nginx/ssl/server.key.
- SSL_VERIFY_CLIENT
- optional_no_ca to enable TLS client certificate authentication on the $HTTPS_PORT port; off to disable it, which also disables LinkedDataHub's WebID-TLS authentication method.
- Disabling can be used to avoid the certificate prompt in the browser in end-user facing dataspaces. The client certificate authentication is still available on port 5443.
- MAX_BODY_SIZE
- Maximum allowed request body size (linkeddatahub has a separate setting for this). Defaults to 2097152.
By default nginx is configured to guard against DoS by limiting the rate of requests per second, which can be necessary on a public instance. The limiting can be disabled in the nginx configuration inlined in docker-compose.yml by commenting out all lines starting with limit_req and limit_conn using #.
SPARQL endpoint limits¶
An external client's request to a dataspace's /sparql is capped by concurrency as well as by rate, since a few heavy queries in flight at once can exhaust the triplestore's memory however slowly they arrive. A request beyond any of the limits is answered with 429 Too Many Requests. Other paths, and callers from private (RFC 1918) addresses such as the platform itself, are not counted.
- NGINX_SPARQL_CONN_PER_CLIENT
- Maximum /sparql requests in flight per client. Defaults to 2.
- NGINX_SPARQL_CONN_TOTAL
- Maximum /sparql requests in flight across all clients. Defaults to 8.
- NGINX_SPARQL_RATE
- Request rate per client, with a burst of 10. Defaults to 2r/s.
Server certificates¶
The certificates generated by the server-cert-gen.sh script are self-signed and therefore are shown as not secure in web browsers. On a local machine this should not be a problem; on public/production servers use LetsEncrypt certificates instead. They can be mounted into nginx as follows:
services:
nginx:
environment:
- SERVER_CERT_FILE=/etc/letsencrypt/live/kgdev.net/fullchain.pem
- SERVER_KEY_FILE=/etc/letsencrypt/live/kgdev.net/privkey.pem
volumes:
- /etc/letsencrypt:/etc/letsencryptSELF_SIGNED_CERT should be set to false in this case.