Conversation
- infrastructure/db-table-notes.json (new): curated semantics keyed schema.table — units, product variant/run, record start — the facts introspection can't see. Seeded with public.imerg (v7 LATE run, not gauge-adjusted Final; mean = daily zonal mm; record from 1998-06) and public.seas5 (issued+leadtime=valid), both verified against the pipeline code / live data. - gen_db_schema.py: merge those notes into each table's details cell (📝) and point to the file from the page header. Snapshot stays generated; the notes file is the only hand-edited part. - blob-io skill: never list a blob container without name_starts_with (full-container listings hang for minutes), and a recipe for building a seasonal zonal series from the daily per-pcode stats tables with area-weighted admin units (the code everyone re-derives per analysis). Motivated by a live lookup: IMERG semantics (which run? what units?) required a second page-hop to raster-pipelines.md, and the zone reconstruction was rewritten from scratch.
|
Docs check — this PR touches machinery whose describing doc didn't change:
If the docs genuinely don't need an update (internal refactor, comment fix), ignore this — it's a reminder, not a gate. Capture-as-you-go: the author has the context now; the drift bots only catch it weeks later. |
| cl = cols_by[(s, t)] | ||
| pk = pk_by[(s, t)] | ||
| colstr = ", ".join((f"**{c}**" if c in pk else c) + f" `{dt}`" for c, dt in cl) | ||
| note = notes.get(f"{s}.{t}") |
There was a problem hiding this comment.
Orphaned note keys vanish silently.
notes.get(f"{s}.{t}") renders a note only when its key matches a live table. A key stranded by a rename (the 2026-09 aa unification just renamed/dropped several tables) or a typo (public.imrg) silently stops rendering — hand-curated semantics disappear from the generated snapshot with no signal, and the page still looks complete. A 3-line check (set(notes) - {f"{s}.{t}" for ...} → fail loudly or emit into the page) makes the parallel file self-checking.
Secondary points: json.loads(NOTES_FILE.read_text()) (line 104) is unguarded — one trailing comma in this hand-edited JSON kills the daily db-schema.yml run with a traceback that doesn't name the file; a note keyed to a zero-column table never renders (the note is appended only inside the if cl details cell); and the public.imerg note says "record from 1998-06" while pipelines/raster-pipelines.md says coverage from 1998-01-01 — if both are true (source coverage vs DB rows), the note should say which fact it states.
| 🤖 I can help with this PR if wanted — mention `@kb-steward` with a question and I'll answer, or ask me explicitly to make a change (resolve conflicts, apply review feedback) and I'll push it. Otherwise I'll stay out of the way. |
This branch has not been deployed
1 participant
What
infrastructure/db-table-notes.json(new, hand-curated) — one-line semantics per DB table: units, product variant/run, record start. Seeded with the two tables verified this session:public.imerg— v7 late run (satellite-only in-season, not the gauge-adjusted Final),mean= zonal-mean daily mm, record from 1998-06, seasonal totals = SUM over days (watch partial seasons).public.seas5—issued_date+leadtime=valid_date.scripts/gen_db_schema.py— merges those notes into each table's<details>cell (📝) and mentions the file in the page header. The snapshot stays fully generated; the notes file is the only curated part. Tested against the live prod DB (📝 renders on both rows); regenerated snapshot deliberately not committed — the dailydb-schema.ymlrun will pick it up.claude/plugins/data-access/skills/blob-io/SKILL.md— two additions from real friction: never list a blob container withoutname_starts_with(a full-container listing hangs for minutes), and a compact recipe for a seasonal zonal series from the daily per-pcode stats tables using area-weighted admin units (code otherwise re-derived per analysis).Why
During a live Niger-drought lookup, finding the IMERG table took one grep — but its semantics (which run? which units? record start?) needed a second hop to
pipelines/raster-pipelines.md, and the run variant (late vs Final) materially changes interpretation (gauge-adjusted or not). Colocating verified semantics on the schema row makes the next lookup one hop, and the recipe removes the most-rewritten boilerplate.check_claude_assets.pypasses (desc budget unchanged — no skill descriptions touched).