Wikidata:SPARQL query service/WDQS backend update/Migrate

Wikidata Query Service Backend Migration

This page provides instructions for how to start using WDQS v2 endpoints, and guidance about rewriting your queries to be compatible with SPARQL1.1. and the new QLever backend.

Start using new endpoints

edit

Access to both the v1 and v2 services will be available until at least February 2027, as outlined in the project timeline.

v1 (current) tool URL v2 (new) tool URL
query.wikidata.org query-next.wikidata.org
query-scholarly.wikidata.org query-scholarly-next.wikidata.org

If you submit queries via HTTP requests directly to the SPARQL endpoint, follow the same URL pattern outlined in the table above:

Example GET request with v1 URL:

https://query.wikidata.org/sparql?query=(SPARQL_query)

Example GET request with v2 URL:

https://query-next.wikidata.org/sparql?query=(SPARQL_query)

The API functionality in v2 is the same as v1, with one exception: QLever does not support the application/x-binary-rdf-results-table response format.

To test query migration to QLever, you can also use the QLever-maintained Wikidata query endpoint. The API is https://qlever.dev/api/wikidata?format=json&query=(SPARQL_query) (Note "format" is required).

Rewrite queries

edit

In order for queries to run on the new QLever backend, you must rewrite them to stop using Blazegraph-specific services, functions, or syntax. Query syntax must also be SPARQL 1.1 compliant. The information in the following sections provides tools, examples, and guidance to help you rewrite your queries.

The following table links to details of how (or whether) the v1 functionality is supported in v2.

Reference table: WDQS v2 query support
Service, function, or syntax being deprecated v2 support
Non-SPARQL 1.1. date and time parsing QLever adheres to the SPARQL 1.1 standard, which requires xsd:dateTime to include a time part, not just a date. Duration calculations should use operators like xsd:dayTimeDuration instead of xsd:duration.
Label service(wikibase:label) Use OPTIONAL clauses with rdfs:label, schema:description, skos:altLabel
MediaWiki API service(wikibase:mwapi) Not supported in v2: Get API results separately and input into SPARQL, or consider using Wikidata APIs instead of SPARQL
Named subqueries Rewrite queries to reorder clauses and update syntax
Query hints Rewrite queries to remove hints, use subqueries to control block execution order
Geospatial services and functions
  • wikibase:around
  • wikibase:box
  • geof:distance
Rewrite queries to use features of Wikidata RDF and GeoSPARQL
bd:slice Rewrite queries to consider using ORDER BY, and OFFSET / LIMIT, or range
  • May no longer be needed with QLever performance improvements
bd:sample Rewrite queries to use LIMIT and ORDER BY
  • May no longer be needed with QLever performance improvements
wikibase:decodeURI() Not supported in v2: Use programming language's built-in decoder, or retrieve different values for readability
wikibase:isSomeValue Rewrite queries to FILTER on the skolemized IRI for "someValue"
GAS service Rewrite queries, recommended to use QLever's Path Search

Date and time parsing

edit

SPARQL 1.1 defines no arithmetic on dates and durations. The following differences between Blazegraph and QLever impact date and time parsing, and operations like filtering, comparing, and calculating duration:

  • Blazegraph's date parser is more relaxed. QLever adheres strictly to the SPARQL1.1 standard, so you must express dates and times with more precision.
  • Blazegraph returns the result of date calculations as numbers. This makes it possible to do comparisons like >= 30. QLever supports date-based arithmetic through the XPath operators xsd:dayTimeDuration and xsd:yearMonthDuration, not for plain xsd:duration. QLever strictly requires both sides of the comparison to be the same type, e.g. "P1D"^^xsd:dayTimeDuration.

For example, on QLever, an expression to check whether ?x falls between two dates would take the form:

?x >= "2001-01-01T00:00:00"^^xsd:dateTime && ?x < "2001-01-02T00:00:00"^^xsd:dateTime
Comparison of basic date expressions
Blazegraph QLever
"2001-08-25"^^xsd:dateTime "2001-08-25T00:00:00"^^xsd:date
or
"2001-08-25T00:00:00"^^xsd:dateTime
"P60D"^^xsd:duration "P60D"^^xsd:dayTimeDuration

For additional information see https://docs.qlever.dev/datetime/#standard-sparql-functions.

Label service

edit

Query rewrite

The label service (wikibase:label) enables you to fetch entity labels, aliases, or descriptions, with language fallbacks. It reduces the complexity of the syntax necessary to retrieve labels, and it helps compensate for Blazegraph inefficiency in using rdfs:label directly on a graph the size of Wikidata. QLever is more efficient at processing queries using rdfs:label, even with billions of triples.

Because the label service implements complex fallback semantics, your updated queries will likely appear more complex: rewrites must create an OPTIONAL block for each label variable for each language, so a list of three label variables in three languages would become nine OPTIONAL blocks, where in the label service call it was just a list of three languages.

To rewrite queries that used the label service, you must:

  1. Remove the call to the wikibase:label service.
  2. Determine each entity variable and its related label/description/alias variable names.
  3. Create OPTIONAL blocks: use BIND to first check for a bound variable, and then the OPTIONAL clause to request the label in each of the specified languages, by priority.

For full instructions, see the label service query rewrite guide.

MediaWiki API service

edit

Not supported in v2

The MediaWikiAPI service (wikibase:mwapi) is a Blazegraph-specific extension to WDQS that calls the MediaWiki Action API, and binds its results to SPARQL variables. The service is not part of the SPARQL 1.1 standard, and is not supported in WDQS v2, since it is not possible to call a REST API within a query. Consequently, the only reliable and efficient way to rewrite SPARQL queries that use this service is to separately call the MediaWiki Action API (or another Wikimedia API), and then feed the results into a subsequent SPARQL query. You may want to explore whether other data access methods could work for your use case. For example, you may be able to use the Wikidata REST API with MediaWiki REST API to accomplish similar tasks more efficiently than you can by combining SPARQL and the MediaWiki Action API.

For full instructions, see the MWAPI service migration guide.

Named subqueries

edit

Query rewrite

Named subqueries is a Blazegraph extension that adds order or structure, to help optimize queries. SPARQL 1.1 supports subqueries, but they are not nameable: the %name syntax is not SPARQL 1.1. QLever's syntax requires that any named subquery be defined before it is referenced. To revise queries:

  1. Move the WITH clause outside of the main top-level SELECT, and name it by using WITH % AS { … }.
  2. Reference the subquery by using the pattern SELECT … WHERE { INCLUDE % }.

For full instructions see the named subqueries rewrite guide.

Query hints

edit

Query rewrite

Blazegraph hint directives include: hint:Query, hint:Prior, hint:Group, hint:GroupAndSubGroups, and hint:SubQuery. SPARQL 1.1 and QLever don't include a query hint system. Queries that include hint:* directives will still run on v2, but the hints will have no impact. If you were using query hints to control execution order, be aware that the structure of your query influences QLever's optimizer: to force early evaluation of a portion of a query, consider using subqueries.

For full instructions, see the query hints rewrite guide.

Geospatial services and functions

edit

Query rewrite

Blazegraph provides several geospatial extensions and helper functions that have no direct SPARQL 1.1 equivalents: wikibase:around, wikibase:box, and geof:distance. GeoSPARQL and Wikidata's RDF coordinate properties can reproduce most of the same functionality on the QLever backend.

For full instructions, see the geospatial services and functions rewrite guide.

bd:slice

edit

Query rewrite

bd:slice is a Blazegraph service that provides efficient, stable pagination over a single triple pattern. Its purpose is to keep otherwise-expensive queries under the WDQS timeout by working on a subset of the data, instead of the full data.

For many queries, you may not need this type of optimization anymore: QLever’s indexes and performance are often sufficient to successfully complete queries that would time out on Blazegraph. If you do still need to slice or paginate your results, consider using ORDER BY, and OFFSET / LIMIT, or range.

For full instructions, see the bd:slice query rewrite guide.

bd:sample

edit

Query rewrite

bd:sample is a Blazegraph service that selects a bounded subset of solutions from a graph pattern without fully materializing and sorting the entire set of results. Its purpose is to keep otherwise-expensive queries under the WDQS timeout, by working on a sample instead of the full data.

You may not need this type of optimization anymore: QLever can often successfully complete queries that would time out on Blazegraph. If you do still need similar functionality:

  1. Rewrite your queries to use an inner SELECT … WHERE { }, projecting the variables present in the outer query.
  2. Append LIMIT N to replicate bd:sample.limit. Or, if the original query didn't specify a limit, use the default value (100).
  3. Add an ORDER BY (depending on bd:sampleType).

For full instructions, see the bd:sample query rewrite guide.

wikibase:decodeURI()

edit

Not supported in v2

The decodeURI function reverses percent-encoding to make sitelink URLs into human-readable strings in query output. SPARQL 1.1 has no equivalent functionality. Most programming languages have URL decoding components you can use instead. For output readability, you could retrieve the “title” from the value of the predicate, schema:name instead.

For full instructions, see the wikibase:decodeURI() migration guide.

wikibase:isSomeValue()

edit

Query rewrite

The wikibase:isSomeValue function detects and filters statements that have an "unknown value" or "some value" rather than a specific, concrete data item. To rewrite queries that use isSomeValue(), FILTER the skolemized IRI for "someValue" instead.

For full instructions, see the wikibase:isSomeValue() query rewrite guide.

GAS service

edit

Query rewrite

The Blazegraph Gather, Apply, and Scatter (GAS) service provides graph traversal algorithms that enable a series of relations to be followed through the graph. The recommended rewrite strategy is to use QLever's Path Search service instead.

For full instructions, see the GAS service rewrite guide.

Update documentation and examples

edit

Tutorials, notebooks, example queries, and all other documentation must eventually be updated to use the new endpoints, and to reflect the additional query revisions outlined above. Please update any content you maintain, and request help if you need it.

  • For help with rewriting queries, first ask on Wikidata:Request_a_query.
  • To help with updating Wikidata documentation and examples, or to request that page content be updated to reflect v2 changes, comment on a page's Talk page.

Alternative data access methods

edit

Wikidata REST API

edit

The Wikidata REST API has endpoints that can retrieve item labels, property labels, and descriptions in a specific language with fallbacks:

Depending on your use case, you may be able to use the Wikidata REST API to achieve the same goals for which you were using WDQS v1 services like wikibase:label and wikibase:mwapi.

Wikibase API

edit

The MediaWiki Action API has a Wikibase module you can use to retrieve content from Wikidata, and integrate it into your application. See the tutorial and the API docs.