Repository navigation
Conversation
The output configuration will be a YAML file, so CABLE needs a YAML parser. yaFyaml is used (the other Fortran YAML library tried could not represent lists of mappings). It is found with find_package if installed, and otherwise fetched with FetchContent together with gFTL and gFTL-shared, pinned to exact tags. cable_yaml_mod wraps the library: the parsed document is copied into a plain tree of cable_yaml_node_t values (scalar, mapping or sequence), so no yaFyaml types are visible outside the module and the library can be replaced later. The node type has its own deep-copy assignment because gfortran 13 mishandles intrinsic assignment of recursive allocatable components. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
def_var gains two optional arguments, shuffle and deflate_level. The nf90 implementation applies them to variables with dimensions (scalars cannot be compressed) when the level is above zero; this needs a NetCDF-4 file. The stub ignores them. The ParallelIO implementation accepts them but stops with a clear message if compression is requested, because its deflate call could not be checked here. The test writes a NetCDF-4 file and reads the settings back with the raw netCDF library. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
The design settles on a single name for each concept, with no mapping between
internal and user-facing names:
frequency: timestep (was all), 3hrly, daily, monthly, yearly
aggregation: instant (was point), mean, max, min, sum
reduction: none, grid_cell_average, first_tile_on_cell (was
first_patch_in_grid_cell), dominant_tile
cable_timing_mod now owns the frequency list and a rank function (finest to
coarsest), supports 3hrly and yearly, drops the custom userNNN intervals, and
has a date helper that turns a start date and a number of days into
YYYY-MM-DD. The old is_greater_than comparison is replaced by comparing ranks.
dominant_tile is a new tile reduction (the largest area fraction in a cell; the
first tile wins ties) for every type and rank, wired into the write dispatch.
The original output module and diagnostics are updated for the renames so the
code stays consistent. The old output%averaging values all, daily and monthly are
still accepted and translated in one place.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
…tions What an output variable is, and where its data comes from, used to be one hand-written Fortran call per variable in cable_diagnostics, mixed with the logic that decided whether it was written. It is now split in two: * output_catalogue.yaml describes each variable: name, dimensions, type, units, long name, group, module, restart name and the like. The text is embedded into the executable at build time by cmake/EmbedText.cmake, so there is no file to find at run time. * cable_output_bindings*.F90 supply what only Fortran can: the model variable (through an aggregator), the unit conversion, the valid range and whether the variable exists in the current model configuration. Model state and settings are passed in as arguments, not read from module variables. cable_output_catalogue_mod joins the two by name into cable_output_variable_definition_t values and stops, listing every problem, if a name has no partner on the other side. The field name users see is the NetCDF name the variable always had; restart files keep their existing variable names through a separate restart name. Variables the model cannot provide stay in the catalogue as unavailable. Nothing uses the new code yet; cable_diagnostics is still the source of output variables. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
cable_output_config_mod reads the YAML output configuration: streams (a file and
a write frequency), and variables, groups and modules directed to them. It
expands modules, then groups, then variables into a flat list of variable uses,
fills in the {field_name}, {frequency}, {aggregation}, {reduction}, {start_date}
and {end_date} templates, and applies the design's two rules (no two variables
with the same NetCDF name in a stream, and no two identical variables in a
stream).
Everything is a function of its arguments and every problem found is collected, so
one run reports all the mistakes in a file. It also rejects unknown keys and names,
reductions that make no sense for a variable (integers, scalars, variables with no
tile dimension), writing a variable more often than the model updates it, two
streams writing one file, and attributes that clash with the automatic ones.
Variables that are unavailable are skipped inside a group or module, and are an
error only when requested by name.
The stream and variable settings types live in cable_output_mod so that the
engine can take them. The parser is not connected to the drivers yet.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
The serial and MPI master drivers now set output up in five steps: build the variable definitions from the catalogue and bindings, register them, work out the run's first and last dates, read the output configuration file named by filename%output_config, and create the files. The output engine is rewritten to write several streams: output_streams replaces the single global stream, every variable use owns its own aggregator, and each stream has its own file, frequency and compression. update, write, write_parameters and write_restart loop over the streams. A stream with compression above zero is a NetCDF-4 file, otherwise classic. The restart file is written from the definitions that have a restart name, as before and independent of the configuration. Breaking changes: * filename%output_config is required. A run without it stops at start-up. * The output%met, output%flux, ... group switches, the per-variable output%<name> switches, output%averaging and all patchout% switches are removed. cable_iovars keeps output%restart and output%grid. A namelist that still sets a removed switch stops with a message that says what to do. * cable_diagnostics and cable_diagnostics_casa are removed; the catalogue and bindings replace them. * output%balances no longer controls the balance printout at the end of a run, which now depends on verbose only. * Files written for a separate file per variable go in the directory part of the stream's file_name. The example cable.nml in src/offline is updated, with a matching output_config.yaml. Output of the original code was reproduced exactly (apart from the new cell_methods attributes) for all four test sites, at monthly, daily and every time step, in serial and under MPI, using YAML equivalent to the old switches. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
cable_output_build_streams works out which files a configuration produces and which variables go in each, without creating any. The tests check the file and variable counts for several streams, that each use takes its settings from the definition and the configuration, the order of attributes (definition attributes, then cell_methods, then the user's), that stream compression and global attributes are carried over, one file per variable with the directory of file_name, and that a stream with no variables makes no file. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
The documentation already uses the mkdocs macros plugin. main.py is its hook and registers a macro that builds the catalogue page every time the documentation is built, so the page cannot go out of date. catalogue_docs.py reads three source files: output_catalogue.yaml (the metadata of each variable), the Fortran bindings (which model variable each name is bound to, the unit conversion and when it exists) and the valid ranges in cable_checks.F90. For each variable the page shows its name, long name, units, dimensions and group, the model variable it is bound to with the type of the structure it lives in, and notes on conversion, valid range, availability and restart name. Restart-only variables have their own table. A catalogue entry with no binding is shown as a warning rather than hidden. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
The Output files section replaces its placeholder with an overview (summary first, the namelist settings that concern output, and a table for converting the removed output% and patchout% switches), a reference for every setting and rule of the configuration file, and ten worked examples. Every example was run through the model. The cable.nml reference loses the removed switches and gains filename%output_config. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
Flow charts (as Mermaid diagrams) of how a definition, a use and a file relate, what happens at start-up, on every time step and at the end of a run, with a table of where each part lives and the things worth knowing when changing it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hNmrg1iRG449BqCN25afu
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CABLE
Thank you for submitting a pull request to the CABLE Project.
Description
This development was driven by me, but the actual code was written by Claude.
This implements the new output API as specified in the forum post and the PDF
CABLE-output-config.pdf. In essence, it gives the user much more fine grained control over how variables are output from the model.
This includes a YAML parser, rewriting of the CABLE diagnostics into a catalogue of variables and integration into the cable driver, both in MPI and serial.
I have directed Claude to chunk the commits into logical parts, to make it easier to review.
Type of change
[x] Structural development
Checklist
Testing
Are the changes bitwise-compatible with the main branch? If working on an optional feature, are the results bitwise-compatible when this feature is off? If yes, copy benchcab output showing successful completion of the bitwise compatibility tests or equivalent results below this line.
Are the changes non bitwise-compatible with the main branch because of a bug fix or a feature being newly implemented or improved? If yes, add the link to the modelevaluation.org analysis versus the main branch or equivalent results below this line.
Please add a reviewer when ready for review.
📚 Documentation preview 📚: https://cable--744.org.readthedocs.build/en/744/