Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

An **Explain** result view for [YASGUI](https://github.com/TriplyDB/Yasgui) and [MatGUI](https://github.com/Matdata-eu/MatGUI) that shows how a SPARQL query is executed:

- **Execution plan**: the operations the triple store ran for your query, with the rows and the time of each, asked from the endpoint after the query completed. Supported engines: QLever, GraphDB, Stardog, Blazegraph and Amazon Neptune.
- **Execution plan**: the operations the triple store ran for your query, with the rows and the time of each, asked from the endpoint after the query completed. Supported engines: QLever, GraphDB, Stardog, Blazegraph, Amazon Neptune and TripleTrail.
- **Algebra**: the query translated to SPARQL algebra (section 18 of the SPARQL specification), computed in the browser for any endpoint, as a collapsible tree and as the s-expression used by ARQ and the specification.

**[Live demo](https://matdata-eu.github.io/yasgui-explain-plugin/)**
Expand Down Expand Up @@ -53,18 +53,19 @@ Run a query, then open the **Explain** tab. The algebra is there at once. The pl
| Stardog | `/{db}/explain` with `profile=true` | the profiled plan as text: rows, wall time and memory per operator |
| Blazegraph | the query with `explain=details` | Blazegraph's explain page, in a sandboxed frame |
| Neptune | the query with `explain=details` | Neptune's explain table as text |
| TripleTrail | `/{ds}/explain` (beside `/{ds}/sparql` or `/{ds}/query`), recognised from a `Server` header naming TripleTrail | the operation tree with the time and rows of each step, the row count, planning time and spatial index decisions |

Virtuoso, Jena Fuseki, Oxigraph, RDF4J and Comunica publish no plan over the SPARQL protocol: the view then shows the algebra only.

Only MatGUI's YASR runs the plan request with the authentication and request settings of the tab. In other hosts the plugin sends the request itself, without them. Stardog, Blazegraph and Neptune need extra request parameters, which MatGUI passes on from version 6.9 (`yasr.hostApi.queryRequest`); older hosts send those requests directly too.
Only MatGUI's YASR runs the plan request with the authentication and request settings of the tab. In other hosts the plugin sends the request itself, without them. Stardog, Blazegraph, Neptune and TripleTrail need extra request parameters, which MatGUI passes on from version 6.9 (`yasr.hostApi.queryRequest`); older hosts send those requests directly too.

## Options

Options go in the plugin constructor, or in `yasr.config.plugins.Explain.dynamicConfig`:

| Option | Default | Meaning |
|---|---|---|
| `engine` | detect | Force an engine (`qlever`, `graphdb`, `stardog`, `blazegraph`, `neptune`) or `none` for every endpoint |
| `engine` | detect | Force an engine (`qlever`, `graphdb`, `stardog`, `blazegraph`, `neptune`, `tripletrail`) or `none` for every endpoint |
| `engines` | all | The engines offered in the list, in order |
| `autoFetch` | `true` | Fetch the plan when the view opens; with `false`, the Refresh button does |
| `getQuery` | the host's query link | A function returning the query text |
Expand All @@ -84,7 +85,7 @@ npm run build # dist/
npm run build:demo # demo/dist, published to GitHub Pages
```

The demo opens on a QLever endpoint, with buttons for public QLever, GraphDB, Blazegraph and Stardog endpoints; the sample buttons show recorded plans without an endpoint (`?sample=qlever|graphdb|stardog`).
The demo opens on a QLever endpoint, with buttons for public QLever, GraphDB, Blazegraph and Stardog endpoints; the sample buttons show recorded plans without an endpoint (`?sample=qlever|graphdb|stardog|tripletrail`).

## License

Expand Down
2 changes: 1 addition & 1 deletion demo/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
YASGUI Explain Plugin - demo

A full Yasgui with the Explain plugin registered. Run a query against a
QLever, GraphDB, Stardog, Blazegraph or Neptune endpoint to see its
QLever, GraphDB, Stardog, Blazegraph, Neptune or TripleTrail endpoint to see its
execution plan, or load one of the sample plans to try the view without
an endpoint. The algebra works for any query.
Run locally with: npm run dev
Expand Down
73 changes: 73 additions & 0 deletions demo/samples.js
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,72 @@ Slice(offset=0, limit=10) [#10], results: 10, wall time: 0 ms (0.0%), memory: 0
+─ Scan[PSOC](?s, :label, ?label) [#5.0K], results: 4927, wall time: 1 ms (1.2%), memory: 0 B
\`─ Scan[PSOC](?s, :price, ?price) [#5.0K], results: 4927, wall time: 1 ms (1.2%), memory: 0 B`;

const tripletrailQuery = `PREFIX : <http://example.org/>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>

SELECT ?station ?name WHERE {
?station a :Station ;
:name ?name ;
geo:hasGeometry/geo:asWKT ?wkt .
FILTER(geof:sfWithin(?wkt, "POLYGON((4.2 50.7, 4.6 50.7, 4.6 51.0, 4.2 51.0, 4.2 50.7))"^^geo:wktLiteral))
}
ORDER BY ?name`;

const tripletrailResponse = {
rows: 42,
spatial: {
filters: 1,
joins: 0,
candidates: 57,
lookup_ms: 0.38,
decisions: ['sfWithin on ?wkt answered from the spatial index (bounding box of the polygon)'],
},
plan: {
'planning duration in seconds': 0.00052,
plan: {
name: 'Project(?station, ?name)',
'number of results': 42,
'duration in seconds': 0.00021,
children: [
{
name: 'OrderBy(?name)',
'number of results': 42,
'duration in seconds': 0.00034,
children: [
{
name: 'Filter(sfWithin(?wkt, POLYGON))',
'number of results': 42,
'duration in seconds': 0.00112,
children: [
{
name: 'Join(?station)',
'number of results': 57,
'duration in seconds': 0.00087,
children: [
{
name: 'QuadPattern(?station geo:hasGeometry/geo:asWKT ?wkt)',
'number of results': 57,
'duration in seconds': 0.00041,
children: [],
},
{
name: 'QuadPattern(?station :name ?name)',
'number of results': 1840,
'duration in seconds': 0.00163,
children: [],
},
],
},
],
},
],
},
],
},
},
};

export const samples = {
qlever: {
label: 'QLever plan',
Expand All @@ -216,4 +282,11 @@ export const samples = {
contentType: 'text/plain',
body: stardogPlan,
},
tripletrail: {
label: 'TripleTrail plan',
engine: 'tripletrail',
query: tripletrailQuery,
contentType: 'application/json',
body: JSON.stringify(tripletrailResponse),
},
};
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "@matdata/yasgui-explain-plugin",
"private": false,
"version": "0.1.0",
"description": "Explain view for YASGUI and MatGUI: the SPARQL algebra of a query, and the execution plan with timings from QLever, GraphDB, Stardog, Blazegraph and Neptune",
"description": "Explain view for YASGUI and MatGUI: the SPARQL algebra of a query, and the execution plan with timings from QLever, GraphDB, Stardog, Blazegraph, Neptune and TripleTrail",
"main": "dist/yasgui-explain-plugin.cjs.js",
"module": "dist/yasgui-explain-plugin.esm.js",
"unpkg": "dist/yasgui-explain-plugin.min.js",
Expand Down Expand Up @@ -36,7 +36,8 @@
"qlever",
"graphdb",
"stardog",
"blazegraph"
"blazegraph",
"tripletrail"
],
"author": "Matdata",
"license": "Apache-2.0",
Expand Down
4 changes: 2 additions & 2 deletions publiccode.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ description:
shortDescription: Shows how a SPARQL endpoint executes a query, with the execution plan of
the engine and the SPARQL algebra, in YASGUI and MatGUI.
longDescription: |
This plugin adds an Explain view to the results of a SPARQL query. It shows the execution plan of the triple store, asked from the endpoint after the query ran: the operations the engine executed, with the rows and the time of each, for QLever, GraphDB, Stardog, Blazegraph and Amazon Neptune. For any endpoint it also shows the SPARQL algebra of the query, as a tree and as the s-expression used by the SPARQL specification, computed in the browser.
This plugin adds an Explain view to the results of a SPARQL query. It shows the execution plan of the triple store, asked from the endpoint after the query ran: the operations the engine executed, with the rows and the time of each, for QLever, GraphDB, Stardog, Blazegraph, Amazon Neptune and TripleTrail. For any endpoint it also shows the SPARQL algebra of the query, as a tree and as the s-expression used by the SPARQL specification, computed in the browser.

It helps to understand why a query is slow, which join or scan takes the time, and how the engine reads the query. It works in YASGUI and in MatGUI, where it is included by default.
documentation: https://github.com/Matdata-eu/yasgui-explain-plugin#readme
features:
- Execution plan with time and rows per operation (QLever)
- Execution plan with time and rows per operation (QLever, TripleTrail)
- Execution plans of GraphDB, Stardog, Blazegraph and Neptune
- SPARQL algebra of any query, as a tree and as s-expression
- Engine detection from the endpoint URL, with a manual choice remembered per endpoint
Expand Down
4 changes: 2 additions & 2 deletions src/ExplainPlugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* A YASR result view with two parts:
* - the execution plan of the query, asked from the endpoint's engine with one
* background request after the query ran (QLever, GraphDB, Stardog,
* Blazegraph, Neptune), and
* Blazegraph, Neptune, TripleTrail), and
* - the SPARQL algebra of the query, computed in the browser for any endpoint.
*/

Expand Down Expand Up @@ -252,7 +252,7 @@ export class ExplainPlugin {
planSection.body.appendChild(
message(
resolved.source === 'none'
? 'No engine recognised for this endpoint. Pick it in the Engine list when it is QLever, GraphDB, Stardog, Blazegraph or Neptune; other engines publish no plan over the SPARQL protocol.'
? 'No engine recognised for this endpoint. Pick it in the Engine list when it is QLever, GraphDB, Stardog, Blazegraph, Neptune or TripleTrail; other engines publish no plan over the SPARQL protocol.'
: 'No plan is asked for this endpoint. Pick an engine in the Engine list to fetch one.'
)
);
Expand Down
20 changes: 18 additions & 2 deletions src/engines/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,25 @@ import { graphdb } from './graphdb';
import { stardog } from './stardog';
import { blazegraph } from './blazegraph';
import { neptune } from './neptune';
import { tripletrail } from './tripletrail';

export const ENGINES: Record<EngineId, Engine> = { qlever, graphdb, stardog, blazegraph, neptune };
export const ENGINES: Record<EngineId, Engine> = {
qlever,
graphdb,
stardog,
blazegraph,
neptune,
tripletrail,
};

export const ENGINE_IDS: EngineId[] = ['qlever', 'graphdb', 'stardog', 'blazegraph', 'neptune'];
export const ENGINE_IDS: EngineId[] = [
'qlever',
'graphdb',
'stardog',
'blazegraph',
'neptune',
'tripletrail',
];

export function isEngineId(value: unknown): value is EngineId {
return typeof value === 'string' && value in ENGINES;
Expand All @@ -36,6 +51,7 @@ export function detectEngine(
if (server.includes('graphdb')) return 'graphdb';
if (server.includes('stardog')) return 'stardog';
if (server.includes('blazegraph') || server.includes('bigdata')) return 'blazegraph';
if (server.includes('tripletrail')) return 'tripletrail';

if (!endpoint) return undefined;
let url: URL | undefined;
Expand Down
92 changes: 92 additions & 0 deletions src/engines/tripletrail.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/**
* TripleTrail
*
* TripleTrail explains a query on `GET|POST /{ds}/explain`, beside the
* dataset's `/{ds}/sparql` and `/{ds}/query`, with the same `query`
* parameter. It runs the query and answers JSON: the row count, the plan it
* ran under `plan.plan` (each step with its `name`, `number of results`,
* `duration in seconds` and `children`), the planning time under
* `plan["planning duration in seconds"]`, and the decisions of its spatial
* index under `spatial`. A caller who may read only some graphs gets the row
* count alone, with `restricted: true`.
*/

import type { Engine, Plan, PlanNode } from '../types';

/** `https://host/ds/sparql` (or `/query`, `/update`) becomes `https://host/ds/explain` */
export function explainUrl(endpoint: string): string {
const index = endpoint.search(/[?#]/);
const [base, suffix] =
index < 0 ? [endpoint, ''] : [endpoint.slice(0, index), endpoint.slice(index)];
const trimmed = base.replace(/\/+$/, '');
if (/\/(sparql|query|update)$/i.test(trimmed))
return trimmed.replace(/\/(sparql|query|update)$/i, '/explain') + suffix;
if (/\/explain$/i.test(trimmed)) return trimmed + suffix;
return `${trimmed}/explain${suffix}`;
}

const isNumber = (value: unknown): value is number =>
typeof value === 'number' && Number.isFinite(value);

const ms = (seconds: unknown) => (isNumber(seconds) ? seconds * 1000 : undefined);

function toNode(step: any): PlanNode {
return {
label: String(step?.name || 'step')
.replace(/\s+/g, ' ')
.trim(),
rows: isNumber(step?.['number of results']) ? step['number of results'] : undefined,
timeMs: ms(step?.['duration in seconds']),
children: Array.isArray(step?.children) ? step.children.map(toNode) : [],
};
}

export function parseTripleTrailResponse(body: string): Plan {
const text = body.trim();
if (!text) throw new Error('TripleTrail returned an empty response.');
let json: any;
try {
json = JSON.parse(text);
} catch {
throw new Error('TripleTrail did not return JSON. Is this endpoint a TripleTrail server?');
}
if (!json || typeof json !== 'object' || json.head || json.results || 'boolean' in json)
throw new Error(
'The endpoint returned results instead of a plan. Is this endpoint a TripleTrail server?'
);
const summary: string[] = [];
if (isNumber(json.rows)) summary.push(`Result size: ${json.rows.toLocaleString('en-US')} rows`);
const planning = ms(json.plan?.['planning duration in seconds']);
if (planning !== undefined) summary.push(`Query planning: ${planning.toFixed(2)} ms`);
const spatial = json.spatial && typeof json.spatial === 'object' ? json.spatial : {};
if (spatial.filters || spatial.joins) {
const lookup = isNumber(spatial.lookup_ms)
? `, index lookup ${spatial.lookup_ms.toFixed(1)} ms`
: '';
summary.push(
`Spatial: ${spatial.filters || 0} filters, ${spatial.joins || 0} joins, ${spatial.candidates || 0} candidates${lookup}`
);
}
if (Array.isArray(spatial.decisions))
for (const decision of spatial.decisions) summary.push(`Spatial rewrite: ${String(decision)}`);
const root = json.plan && (json.plan.plan || (json.plan.name ? json.plan : undefined));
if (root && typeof root === 'object') return { kind: 'tree', root: toNode(root), summary };
if (json.restricted)
summary.push(
'The plan is left out: you may read only some of the graphs, and the plan is chosen from all of them.'
);
return { kind: 'text', text: summary.join('\n') || 'No plan.' };
}

export const tripletrail: Engine = {
id: 'tripletrail',
label: 'TripleTrail',
description: 'The plan TripleTrail ran, with the rows and time of each step.',
needsRequestOptions: true,
buildRequest(query, endpoint) {
return { query, accept: 'application/json', endpoint: explainUrl(endpoint) };
},
parse(body) {
return parseTripleTrailResponse(body);
},
};
2 changes: 1 addition & 1 deletion src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ export interface Yasr {
}

/** The identifiers of the engines the plugin can ask for a plan */
export type EngineId = 'qlever' | 'graphdb' | 'stardog' | 'blazegraph' | 'neptune';
export type EngineId = 'qlever' | 'graphdb' | 'stardog' | 'blazegraph' | 'neptune' | 'tripletrail';

/** The engine setting of an endpoint: detect it, one of the engines, or no plan at all */
export type EngineChoice = 'auto' | EngineId | 'none';
Expand Down
Loading