Persistently identify semantic represenations of benchmarks and tool configurations

[doigl]

A semantic representation of benchmarks shall - among other aspects - specify in a machine readable way all input parameters and post-processing metrics. A representation of a tool configuration could adress a simulation code in a specific version with its dependencies and configurations.

This information should be placed in a knowledge graph for machine readability. To also offer some human readable information, we could provide landing pages basing on markdown/github pages

For identification, a prefix for persistent identifiers of the different benchmark scenarios should be used. My suggestion: a https://w3id.org/ in the namespace from nfdi4ing like w3id.org/nfdi4ing/benchmarks.

This would allow to support Content-Negotiation to redirect text/html requests to the human readable landing page and text/turtle requests to the rdf representation of the benchmark.

[joergfunger]

what is that status with that issue? @doigl and @M-Jafarkhani

[doigl]

@joergfunger : The status of the semantic representation is actually a JSON-LD file for each benchmark. This is available for both benchmarks that we have so far.

It would be possible to add the benchmarks also to the ORKG, but I only see the possibility to represent there a subset of the information, that we need. (according to the ORKG benchmark data model, like the benchmark itself, and the metrics, but without the configurations and the virtual sensor information of the metrics).

The status of the PIDs via w3id.org: I'm waiting for someone, who also likes the idea ;-) That could be done very fast.

[joergfunger]

Thanks for the update, I thought the semantic representation is first done locally within the benchmark ROCrate (that just stores the benchmark definition, not the results). Where can I find that representation?
As for the PID, I'm not the expert here, so I think directly adding that would IMO be possible.

[doigl]

@joergfunger : Yes, the idea is, that the benchmark representation is also a file within the RO-Crate and can be referenced from the ro-crate-metadata.json. The other possibility would be to add the information directly to the ro-crate-metadata.json.

@M-Jafarkhani : Is the semantic representation already included in the RO-Crate and also pushed to the github somewhere?

[M-Jafarkhani]

One sample representation could look like the following:

benchmark.json

[joergfunger]

How do you generate that file?