feat(data-index): give relaton-data-ietf a Pages identity

data-index/README.adoc

@@ -13,7 +13,8 @@ directory's `configs.yml` for the page's `title`, `favicon` and `description`.
 This directory once also generated a per-repo Jekyll `_config.yml`
 (`bin/gen-data-index-config` into `generated/`). support#58
 (relaton/relaton#83) replaced the Jekyll build with `relaton index`, and a sweep
-of all 30 repos found that none had ever carried a committed `_config.yml`, so
+of every `relaton-data-*` repo found that none had ever carried a committed
+`_config.yml`, so
 the generator, its snapshot and the `paginate` / `pubid_class` / `pubid_require`
 keys that fed it were removed. Git history holds them if they are ever wanted.
 ====
@@ -152,7 +153,7 @@ for r in $(gh repo list relaton --limit 200 --json name -q '.[].name' | grep '^r
 done
 ----
 
-PR-time validation is `check-index.yml`, mapped into all 30 repos: a PR touching
+PR-time validation is `check-index.yml`, mapped into all 31 repos: a PR touching
 `data/**` builds the whole index and fails if it does not parse. `check_data.yml`
 keeps its own `push`/`pull_request` triggers on top of that in 3 repos (bipm,
 iho, iso) as a cheaper schema check.
@@ -171,7 +172,7 @@ a run carrying fresh data.
 
 [NOTE]
 ====
-Cimas replaces `deploy.yml` and `check-index.yml` wholesale in all 30 mapped
+Cimas replaces `deploy.yml` and `check-index.yml` wholesale in all 31 mapped
 repos, so nothing per-repo can live in a caller's `with:` block and survive. That
 is also why a run's *kind* is derived centrally from its ref rather than from a
 `validate-only` input: the input would have to arrive in exactly that block.
@@ -308,10 +309,16 @@ Small flavors (bipm, iana) finish in under two minutes.
 
 == Status (verified 2026-08-03)
 
-`configs.yml` covers *30 repos*: the 26 rollout targets plus the 3 already-live
+`configs.yml` covers *31 repos*: the 26 rollout targets plus the 3 already-live
 (*ids, oasis, w3c*), folded in so this file is the single source for every site,
-plus *iala* (added 2026-08-17; `main`-default, `index-v2.yaml`). The 3 live ones
-carry per-repo `favicon`/`description` overrides.
+plus *iala* (added 2026-08-17; `main`-default, `index-v2.yaml`) and *ietf* (added
+2026-08-24). The 3 live ones carry per-repo `favicon`/`description` overrides.
+
+`ietf` closes the last gap between this file and `cimas.yml`: Cimas had always
+synced `deploy.yml` into `relaton-data-ietf`, but the repo published no index, so
+it carried no row and its site fell back to a bare `IETF Index`. It now crawls
+with `Relaton::Ietf::DataFetcher`, which commits `index-v1.yaml` (177,230
+records) on `main`. Every repo Cimas syncs `deploy.yml` into now has a row.
 
 [WARNING]
 ====
@@ -357,7 +364,7 @@ Re-check current state any time with:
 
 [source,sh]
 ----
-bin/check-data-pages              # site + raw-index for all 30
+bin/check-data-pages              # site + raw-index for all 31
 bin/check-data-pages --site-only  # just the Pages sites (the 404s to fix)
 ----
 
@@ -387,7 +394,7 @@ cimas open-prs -f $CIMAS_CONFIG -r $CIMAS_WD -b data-pages -m 'Pages deploy' -g
 ----
 +
 NOTE: `check-index.yml` triggers on `pull_request`, so `open-prs` starts a
-`Check index` run in every repo it opens a PR in — ~30 full builds at once,
+`Check index` run in every repo it opens a PR in — ~31 full builds at once,
 2–3 min each on the largest flavors. That is a free fleet-wide smoke test of the
 build, not a loop. Inspect one repo's synced tree in `$CIMAS_WD` before pushing
 all of them: for a target that has never had this file, the sync is an *add*.

data-index/configs.yml

@@ -14,7 +14,7 @@
 #                          (`baseurl` + `source`) both return 200.
 #
 # Branding lives here, and not in each caller's `with:` block, because
-# cimas.yml maps `.github/workflows/deploy.yml` as a whole-file copy for 30
+# cimas.yml maps `.github/workflows/deploy.yml` as a whole-file copy for 31
 # repos: a `with:` block is wiped on the next `cimas sync` and the site silently
 # loses its favicon and description.
 #
@@ -99,6 +99,30 @@ repos:
     description: >-
       Combined bibliographic index of ITU-R and ITU-T publications in Relaton
       format.
+  # Cimas has always synced deploy.yml into relaton-data-ietf. Until that repo
+  # migrated to `Relaton::Ietf::DataFetcher` it published no index, so it
+  # deliberately had no row here and its site fell back to a bare "IETF Index"
+  # with no favicon and no description. The crawl now commits index-v1.yaml
+  # (177,230 records) on main, so the site has a real index and this row gives
+  # it an identity.
+  #
+  # `source` FOLLOWS the publish here, unlike the iana row above. relaton-ietf's
+  # DataFetcher is moving index-v1 -> index-v2 as a straight replacement, never
+  # both; lib/relaton_ietf/data_fetcher.rb still declares index-v1. Flip this
+  # key only after a crawl has committed index-v2.yaml, or bin/check-data-pages
+  # reports a 404.
+  #
+  # The rfcs/rfcsubseries/ids rows below stay: every released relaton reads
+  # index-v1 from those three repos. Retire them the staged way the itu-r -> itu
+  # comment above describes, once relaton's Ietf::Scraper reads this index.
+  - repo: ietf
+    display: IETF
+    source: index-v1.yaml
+    branch: main
+    favicon: 'https://www.ietf.org/static/img/ietf-logo.dc9646b3c716.svg'
+    description: >-
+      Bibliographic data for IETF RFCs, Internet-Drafts and RFC sub-series in
+      Relaton format
   - { repo: jcgm,         display: JCGM,          source: index-v1.yaml, branch: main }
   - { repo: jis,          display: JIS,           source: index-v2.yaml, branch: v2 }
   - { repo: nist,         display: NIST,          source: index-v2.yaml, branch: v2 }
@@ -110,7 +134,7 @@ repos:
   - { repo: xsf,          display: XSF,           source: index-v1.yaml, branch: v2 }
 
   # Already-live repos (their sites returned 200 before this file existed).
-  # Folded in so this file covers all 30; they carry per-repo favicon/
+  # Folded in so this file covers all 31; they carry per-repo favicon/
   # description overrides. branch: v2 is
   # each repo's real default (index-v1.yaml is served from v2).
   - repo: ids

lib/data_index_config.rb

@@ -53,16 +53,19 @@ def entry(repo)
   # The branding `relaton index` renders into the Pages site — title, favicon and
   # `<meta name="description">` — resolved centrally rather than passed by each
   # caller. cimas.yml maps `.github/workflows/deploy.yml` as a whole-file copy for
-  # 30 repos, so a `with:` block carrying these values is wiped on the next
+  # 31 repos, so a `with:` block carrying these values is wiped on the next
   # `cimas sync` and the site silently loses them.
   #
   # Precedence: an explicit non-blank argument (a caller's workflow input) beats
   # this repo's configs.yml entry, which beats the shared default.
   #
-  # Deliberately never raises, unlike #entry: Cimas syncs deploy.yml into
-  # relaton-data-ietf, which publishes no document index and so has no configs.yml
-  # row. An unknown repo falls back to what the workflow's own shell derivation
-  # produced before this method existed — "<FLAVOR> Index" and no branding.
+  # Deliberately never raises, unlike #entry. Every repo Cimas syncs deploy.yml
+  # into now carries a configs.yml row, so nothing exercises the fallback today —
+  # it is a safety net, not a live path. It stays because a repo added to
+  # cimas.yml before its row lands would otherwise fail its own Pages build on a
+  # missing row. Such a repo falls back to what the workflow's own shell
+  # derivation produced before this method existed — "<FLAVOR> Index" and no
+  # branding. spec/cimas_data_pages_spec.rb is what makes the gap loud.
   #
   # => { "title" => String, "favicon" => String, "description" => String }
   def branding(repo, title: nil, favicon: nil, description: nil)
@@ -122,7 +125,8 @@ def entry_description(entry)
   end
 
   # What the workflow's retired shell step produced for a repo configs.yml does
-  # not cover: the slug, upcased. Keeps relaton-data-ietf building unchanged.
+  # not cover: the slug, upcased. Keeps such a repo building unchanged rather
+  # than failing its deploy on a missing row.
   def derived_title(repo)
     "#{self.class.flavor(repo).upcase} Index"
   end

spec/cimas_data_pages_spec.rb

@@ -64,7 +64,7 @@
         # The inverse of the guard this example replaces. `relaton index` needs
         # no Gemfile: `source: gem` installs relaton-cli, `source: git` writes
         # $RUNNER_TEMP/Gemfile.index and points BUNDLE_GEMFILE at it. A restored
-        # mapping would re-create the retired Jekyll bundle in 30 repos and
+        # mapping would re-create the retired Jekyll bundle in 31 repos and
         # re-establish the "this repo builds with Jekyll" signal the migration
         # removes — harmless to the build, which is exactly why it would stick.
         expect(repositories.fetch(cimas_key).fetch("files")).not_to have_key("Gemfile.deploy")
@@ -74,13 +74,14 @@
 
   it "every relaton-data-* repo in the `data` group with a deploy.yml is covered by configs.yml" do
     # The reverse direction: a data repo that Cimas pushes deploy.yml into but that
-    # has no configs.yml row would deploy the theme with no per-repo _config.yml.
-    # Excludes are the deliberate non-index repos (see data-pages-rollout hand-off).
+    # has no configs.yml row deploys a site with no title, favicon or description.
     known = configs.repos.map { |e| "relaton-data-#{e['repo']}" }
-    # ietf gets deploy.yml but is deliberately not a Pages index (no document
-    # index published); every other data-group repo is covered by configs.yml,
-    # including the already-live ids/oasis/w3c now folded in.
-    excluded = %w[relaton-data-ietf]
+    # Empty, and that is the point: configs.yml now covers every data-group repo
+    # that gets deploy.yml. relaton-data-ietf was the last exclusion — it had no
+    # document index until it migrated to Relaton::Ietf::DataFetcher, which
+    # commits index-v1.yaml as it crawls. The list stays so a future deliberate
+    # non-index repo has somewhere to go, and so removing one is a visible edit.
+    excluded = %w[]
     deploys_pages =
       data_group.select do |name|
         files = repositories.fetch(name, {}).fetch("files", nil) || {}
@@ -93,11 +94,11 @@
   end
 
   it "syncs deploy.yml and check-index.yml together, or neither" do
-    # Not folded into the per-repo examples above: those iterate configs.yml,
-    # which deliberately has no relaton-data-ietf row (it gets deploy.yml but
-    # publishes no document index). A repo that merges data with no PR-time
-    # build is exactly the gap this file exists to make loud, whether or not it
-    # has a page — so drive this one off cimas.yml instead.
+    # Not folded into the per-repo examples above: those iterate configs.yml, so
+    # they are blind to a repo cimas.yml already syncs but configs.yml has not
+    # caught up with. A repo that merges data with no PR-time build is exactly
+    # the gap this file exists to make loud, whether or not it has a row yet —
+    # so drive this one off cimas.yml instead.
     #
     # Both directions, because both halves fail silently and the per-repo
     # examples above can only see repos configs.yml already knows about. A repo
@@ -120,7 +121,7 @@
   end
 
   # The failure mode that made this whole change necessary is entirely silent: a
-  # per-repo value in a Cimas-synced template is copied verbatim into 30 repos,
+  # per-repo value in a Cimas-synced template is copied verbatim into 31 repos,
   # and a per-repo value hand-added to a *synced destination* is reverted on the
   # next sync with nothing red in CI. These two guards make either a test
   # failure, for every template cimas.yml syncs — not just deploy.yml.

spec/data_deploy_caller_template_spec.rb

@@ -63,8 +63,9 @@
     it "accepts every default branch name in use across the fleet" do
       # `workflow_run`'s branch filter matches the *triggering* run's branch, so
       # this list has to cover every default branch the synced repos actually
-      # use (23 of the 30 are on v2, six on main, one on master) for the file to
-      # stay repo-agnostic. It is only a pre-filter: a repo carrying more than
+      # use (23 of the 31 are on v2, eight on main; none is on master today, and
+      # the list keeps that name because over-covering costs nothing) for the file
+      # to stay repo-agnostic. It is only a pre-filter: a repo carrying more than
       # one of those names lets a crawl on a dormant branch through, so
       # data-deploy.yml gates on `workflow_run.head_branch` being the repo's
       # real default branch, and publication on the run's own ref.
@@ -117,7 +118,7 @@
     it "leaves no push or pull_request trigger in Deploy or Crawler" do
       # Stated separately from the two `contain_exactly` examples above because
       # this is the invariant that fails silently: re-adding either trigger just
-      # burns Actions minutes fleet-wide, in 30 repos at once, with nothing red.
+      # burns Actions minutes fleet-wide, in 31 repos at once, with nothing red.
       #
       # Note what re-adding `pull_request` *here* would not be: a restored
       # pre-merge check. That lives in check-index.yml, which reaches the same
@@ -168,7 +169,7 @@
       # nothing user-visible to notice either.
       expect(File.exist?(check_index_path)).to be(true),
                                                "the pre-merge check caller is missing; without it a PR " \
-                                               "touching data/ runs no build at all in 30 repos"
+                                               "touching data/ runs no build at all in 31 repos"
     end
 
     it "triggers on pull requests and nothing else" do
@@ -235,7 +236,7 @@
 
     it "passes no inputs at all" do
       # THE reason branding moved into data-index/configs.yml. cimas.yml maps
-      # this file into 30 repos as a whole-file copy, so anything in a `with:`
+      # this file into 31 repos as a whole-file copy, so anything in a `with:`
       # here is either wrong for the other 29 or — once a repo hand-edits it —
       # silently reverted by the next `cimas sync`. Branding failed silently (the
       # page just loses its favicon), which is why it moved to configs.yml.
@@ -261,7 +262,7 @@
     #
     # Narrowing this to `contents: read` — the reflexive "minimal permissions"
     # edit — does not fail at parse time. It fails minutes into the build, at the
-    # deploy step, in all 30 repos at once, with:
+    # deploy step, in all 31 repos at once, with:
     #
     #   The workflow is requesting 'pages: write', but is only allowed 'pages: none'.
     #

spec/data_deploy_workflow_spec.rb

@@ -165,7 +165,7 @@
   describe "resolving branding from relaton/support" do
     it "checks out relaton/support, the branding source of truth" do
       # Branding cannot live in the caller: cimas.yml maps
-      # .github/workflows/deploy.yml as a whole-file copy for 30 repos, so a
+      # .github/workflows/deploy.yml as a whole-file copy for 31 repos, so a
       # `with:` block is wiped on the next sync and the page silently loses its
       # favicon and description.
       expect(support_checkout).not_to be_nil

spec/data_index_config_spec.rb

@@ -20,7 +20,7 @@ def branding(repo)
     it "applies the default favicon to repos without an override" do
       # The convention is the SDO's own icon where there is a stable URL for one,
       # so this roster grows. Adding a `favicon:` to a row means adding it here.
-      overridden = %w[iana ids itu oasis w3c]
+      overridden = %w[iana ids ietf itu oasis w3c]
       default_favicon = config.defaults.fetch("favicon")
 
       config.repos.reject { |e| overridden.include?(e["repo"]) }.each do |e|
@@ -84,17 +84,21 @@ def branding(repo)
   end
 
   describe "configs.yml data" do
-    it "covers exactly the 30 repos with no duplicates" do
+    it "covers exactly the 31 repos with no duplicates" do
       repos = config.repos.map { |e| e["repo"] }
-      expect(repos.size).to eq(30)
-      expect(repos.uniq.size).to eq(30)
+      expect(repos.size).to eq(31)
+      expect(repos.uniq.size).to eq(31)
       expect(repos).to include("iso", "ieee", "jis", "adobe", "easc", "gost", "jcgm", "oiml", "iala")
       expect(repos).to include("ids", "oasis", "w3c") # already-live, folded in
       # Both ITU rows: `itu` is the combined ITU-R + ITU-T corpus, `itu-r` the
       # ITU-R-only repo the relaton gem still consumes. Neither is a typo for
       # the other, and dropping either silently 404s a site.
       expect(repos).to include("itu", "itu-r")
-      expect(repos).not_to include("sdo", "ietf", "misc")
+      # ietf was the one repo Cimas synced deploy.yml into with no row here. It
+      # publishes an index since migrating to Relaton::Ietf::DataFetcher, so the
+      # exclusion is lifted and this file now covers the whole synced fleet.
+      expect(repos).to include("ietf")
+      expect(repos).not_to include("sdo", "misc")
     end
 
     it "gives every entry the required fields" do

spec/index_branding_spec.rb

@@ -7,8 +7,8 @@
 # of truth for these values, and the caller template carries no `with:` at all.
 #
 # The load-bearing invariants here are the fallbacks: the workflow passes all
-# three flags unconditionally, and Cimas syncs deploy.yml into one repo
-# (relaton-data-ietf) that configs.yml deliberately does not cover.
+# three flags unconditionally, and a repo added to cimas.yml before its
+# configs.yml row lands must still resolve rather than fail its own deploy.
 require "English" # $CHILD_STATUS
 require "shellwords"
 
@@ -101,11 +101,14 @@
     end
 
     it "falls back to the derived title and no branding for a repo configs.yml omits" do
-      # relaton-data-ietf gets deploy.yml from Cimas but publishes no document
-      # index, so it has no configs.yml row (see cimas_data_pages_spec.rb).
+      # No live repo exercises this any more: configs.yml now covers every repo
+      # Cimas syncs deploy.yml into, relaton-data-ietf included. relaton-data-sdo
+      # is a deliberate stand-in — it appears in neither file, so it stays a
+      # stable witness. The fallback stays guarded because a repo added to
+      # cimas.yml before its row lands would otherwise fail its own Pages build.
       # Its result must match what the retired shell derivation produced.
-      expect(config.branding("relaton/relaton-data-ietf"))
-        .to eq("title" => "IETF Index", "favicon" => "", "description" => "")
+      expect(config.branding("relaton/relaton-data-sdo"))
+        .to eq("title" => "SDO Index", "favicon" => "", "description" => "")
     end
 
     it "does not raise for an unknown repo, unlike #entry" do
@@ -136,7 +139,7 @@
 
   # The unit examples above all bypass the executable the workflow actually runs.
   # Without these, renaming a method on GithubOutput or DataIndexConfig would
-  # leave the whole suite green and break the resolve step in all 30 repos.
+  # leave the whole suite green and break the resolve step in all 31 repos.
   describe "bin/index-branding" do
     bin = File.join(repo_root, "bin/index-branding")
 
@@ -155,11 +158,14 @@
     end
 
     it "resolves a repo configs.yml does not cover" do
-      out = `#{bin.shellescape} relaton/relaton-data-ietf 2>/dev/null`
+      # Same stand-in as the unit example above: relaton-data-sdo is in neither
+      # configs.yml nor cimas.yml, so it exercises the fallback the executable
+      # must not fail on.
+      out = `#{bin.shellescape} relaton/relaton-data-sdo 2>/dev/null`
 
       expect($CHILD_STATUS).to be_success
       expect(parse.call(out))
-        .to eq("title" => "IETF Index", "favicon" => "", "description" => "")
+        .to eq("title" => "SDO Index", "favicon" => "", "description" => "")
     end
 
     it "applies the flags the workflow always passes, blanks included" do