Publish The nml-tools Schema Dialect And Meta-Schema

[MuellerSeb]

Define the nml-tools schema language explicitly as a Fortran-oriented dialect
based on JSON Schema, publish a machine-readable meta-schema at a stable
versioned URI, and bundle the same resource with the Python package for offline
validation.

The first published dialect should include the existing focused JSON Schema
subset, all supported x-fortran-* keywords, nml-tools operational-default and
reference policies, and the proposed first-class type: complex extension.

The canonical meta-schema should be published together with the full Read the
Docs site described in #58.
Complex support is tracked in #56.

Motivation

nml-tools began as a JSON Schema-based validation layer around f90nml JSON
conversion. It now owns substantially more behavior:

• Fortran module and helper generation;
• operational defaults and generated sentinel storage;
• runtime dimensions and constants;
• native and imported derived types;
• f2py and Python wrappers;
• Fortran-specific shape, kind, length, module, and ownership metadata;
• local schema resolution and use-site composition rules; and
• schema-aware namelist parsing and validation.

The schema language should continue using JSON/YAML documents and familiar JSON
Schema concepts, but strict Draft 2020-12 compatibility should not force
unnatural representations of Fortran concepts. type: complex is the first
proposed direct extension of the standard type vocabulary.

A published dialect makes that divergence explicit, gives editors a source for
completion and diagnostics, and provides a versioned contract for users and
downstream tooling.

Dialect Policy

Describe nml-tools schemas as:

A Fortran-native schema dialect based on a focused subset of JSON Schema
Draft 2020-12.

Use these compatibility rules:

• Preserve standard JSON Schema keyword names and semantics where nml-tools
  supports them naturally.
• Continue using number, integer, boolean, string, array, and object.
• Add first-class Fortran types only where JSON Schema has no natural scalar
  equivalent, beginning with complex.
• Keep representation and generator controls under x-fortran-*.
• Keep schema documents serializable as ordinary JSON and YAML.
• Do not claim that an nml-tools schema containing custom type values is valid
  under the standard Draft 2020-12 meta-schema.
• Treat the nml-tools dialect as implicit when $schema is omitted, preserving
  compatibility with existing project schemas.
• When $schema explicitly names standard Draft 2020-12, reject nml-tools
  custom type values with a diagnostic suggesting the dialect URI.

The first version should be a project-specific meta-schema without a formal
$vocabulary declaration. Redefining the semantics of the standard type
keyword conflicts with the standard validation vocabulary, so presenting this
as an additive vocabulary would be misleading. A formal nml-tools validation
vocabulary can be designed later if interoperability demand justifies it.

Published URI

Publish an immutable schema-language version at a canonical HTTPS URI, for
example:

https://nml-tools.readthedocs.io/en/latest/schema/v1/nml-tools.schema.json

The exact host depends on the registered Read the Docs project name, but the
path must include an independent dialect version such as v1. The v1
resource must remain semantically immutable even though the documentation's
latest build changes.

The meta-schema should identify itself with the same URI:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://nml-tools.readthedocs.io/en/latest/schema/v1/nml-tools.schema.json"
}

User schemas may then opt in explicitly:

$schema: https://nml-tools.readthedocs.io/en/latest/schema/v1/nml-tools.schema.json

x-fortran-namelist: solver
type: object
properties:
  impedance:
    type: complex
    x-fortran-kind: dp

Do not use an unversioned or mutable latest dialect identifier without a
stable v1 path component. Publish v2 for breaking schema-language changes.
Package releases may continue using the same dialect version while remaining
backward compatible.

Meta-Schema Contents

Create one canonical checked-in meta-schema and use it for both package and
documentation output. It should recursively describe the actual accepted
nml-tools schema positions rather than presenting every node as an unrestricted
generic JSON Schema.

At minimum, define reusable meta-schema nodes for:

• the root namelist schema;
• scalar property schemas;
• intrinsic array schemas and their items;
• inline and referenced derived definitions;
• $defs entries and $ref use sites;
• complex scalar schemas;
• annotation keywords;
• defaults and examples;
• bounds and enums;
• x-fortran-kind, x-fortran-len, and x-fortran-shape;
• flexible-tail and default-control keywords; and
• derived type/module ownership keywords.

The type syntax should include:

{
  "enum": [
    "object",
    "array",
    "string",
    "integer",
    "number",
    "boolean",
    "complex"
  ]
}

The complex meta-schema branch should require two numeric entries for defaults
and examples and reject incompatible representation keywords.

The meta-schema should mirror current accepted behavior. Do not silently make
unknown-key rejection stricter than the Python loader without treating that as
a separate compatibility change. Known keywords should nevertheless receive
complete types, enums, descriptions, examples, and conditional structure so
editors provide useful completion.

Package Integration

Bundle the canonical meta-schema under the Python package, for example:

src/nml_tools/schemas/v1/nml-tools.schema.json

Use importlib.resources to access it so installed wheels, editable installs,
and source trees behave consistently.

Update schema loading to:

• recognize the canonical nml-tools dialect URI;
• validate a schema document against the bundled meta-schema without network
  access;
• continue recognizing the supported standard Draft 2020-12 URI;
• treat missing $schema as the nml-tools dialect for backward compatibility;
• report unsupported dialect URIs clearly;
• distinguish meta-schema syntax errors from nml-tools semantic errors that
  require constants, dimensions, reference resolution, or cross-field checks;
  and
• avoid downloading remote meta-schemas or references during normal operation.

The existing jsonschema dependency can perform meta-schema validation, while
the nml-tools resolver and validators continue implementing domain semantics.
A meta-schema can describe the syntax of type: complex; it does not teach a
generic instance validator how a Fortran complex value behaves.

Documentation Publication

Expose the bundled resource from the documentation build at the canonical
schema/v1/nml-tools.schema.json path. Avoid maintaining a second hand-edited
copy. The documentation build should copy the package resource or verify that a
generated copy is byte-identical.

Add human-readable pages covering:

• dialect goals and compatibility policy;
• supported standard JSON Schema keywords;
• intentional deviations;
• every x-fortran-* extension;
• schema-language versioning;
• how $schema affects validation;
• editor configuration; and
• how the machine-readable meta-schema differs from runtime semantic
  validation.

SchemaStore And Editor Discovery

SchemaStore registration is optional and should follow publication of the
stable URI. SchemaStore can register a self-hosted Read the Docs URL and help
language servers select it automatically.

Current nml-tools schemas may use arbitrary filenames, so a broad *.yml or
*.json match would create false positives. Before requesting automatic
registration, either:

• establish an optional naming convention such as *.nml-schema.yml,
  *.nml-schema.yaml, and *.nml-schema.json; or
• register the schema with no automatic fileMatch, leaving users to select it
  manually.

Explicit $schema remains the authoritative and most portable association.

Versioning And Compatibility

• Version the schema language independently from the package version.
• Keep a dialect URI immutable after publication.
• Add new optional keywords and compatible type refinements within v1 only if
  old conforming documents remain valid and keep their meaning.
• Publish v2 for removed keywords, changed defaults, changed type semantics,
  or newly invalid existing documents.
• Keep old meta-schema resources online indefinitely.
• Document which package versions implement each dialect version.
• Use the existing required-version config policy for package compatibility;
  do not overload it as the schema dialect identifier.

Tests And CI

Add tests that:

• validate the meta-schema itself against Draft 2020-12;
• validate every committed example schema against the bundled dialect;
• cover positive and negative documents for every supported schema position;
• cover type: complex and its default/example pair syntax;
• verify standard $schema, nml-tools $schema, omitted $schema, and unknown
  dialect behavior;
• verify schema loading performs no network access;
• verify the package resource is included in wheels and source distributions;
• compare the bundled resource with the documentation-published copy; and
• verify old dialect URLs remain available when a new dialect is introduced.

Add a documentation build check that fails on broken internal references to
the dialect URI or missing static schema output. External link checking may run
separately because it requires network access.

Acceptance Criteria

• nml-tools describes its schema language as an explicit Fortran-native
  dialect based on JSON Schema.
• A versioned meta-schema is bundled with the package and published at a stable
  Read the Docs URI.
• Schemas can select the dialect with $schema.
• Existing schemas without $schema continue to load under the implicit
  nml-tools dialect.
• type: complex is represented and documented as an intentional deviation.
• Meta-schema validation and nml-tools semantic validation produce distinct,
  useful diagnostics.
• Runtime schema loading is fully offline.
• Editors can use the published resource for completion and validation.
• Dialect versioning and compatibility rules are documented.

Out Of Scope

• Teaching arbitrary third-party JSON Schema validators the runtime semantics
  of type: complex.
• Remote $ref retrieval by nml-tools.
• A formal vocabulary registry in the first dialect version.
• SchemaStore registration before a safe filename/discovery policy exists.
• Translating nml-tools schemas into strict standard JSON Schemas. Such an
  exporter may be added separately if a real interoperability need emerges.