diff --git a/CHANGELOG.md b/CHANGELOG.md index 383b09ed..96c8e1cc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,9 @@ it releases that version ([RELEASING.md](https://github.com/energy-models/mathsp ## Upcoming version +- feat(language)!: relation rows index an entry and a sum ([#7](https://github.com/fluxopt/mathspec/pull/7)) + - Breaks a file: the `where:` of a parameter or a variable no longer takes a relation's bare name. Write `in: [relation]`. + - `sum(x, over=)` sums over the relation's key. - feat(language): a dimension may alias another's labels ([#1020](https://github.com/energy-models/mathspec/pull/1020)) - feat!: mappings give labels, relations give rows ([#1015](https://github.com/energy-models/mathspec/pull/1015)) - Breaks a file: `relations:` takes `dims:`, `missing:` and `description` only. Each value column becomes a mapping under `mappings:`, keyed by its subject, with `dim:` naming its dimension and always written, and is read as `subject[mapping]`, such as `generator[bus]`. diff --git a/docs/about/limits.md b/docs/about/limits.md index f18d8ae2..38628f71 100644 --- a/docs/about/limits.md +++ b/docs/about/limits.md @@ -110,13 +110,14 @@ limit does not depend on what solvers take. mathspec refuses a request when a construct that exists already states the same thing. -| Request | Why refused | What you write instead | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| A `where` comparing the label that each row has ([a mapping](../reference/language/relations.md#mappings)) against the dimension it refers to, such as `generator[bus] == bus` | Two dimensions cannot be compared, so mathspec refuses the comparison. The mapping already gives each row its label. | A sum or a lookup through the mapping. `sum(x, over=d, by=s[m])` adds each value of `x` to the label that the mapping gives it, and `at(x, by=s[m])` takes the value at that label ([operators](../reference/language/operators.md#sum)). | -| A vocabulary for tracked metrics: `impacts:`, `effects:`, a `costs` dimension | A tracked metric is already a [named expression](../reference/language/named.md#expressions). | An `impact` dimension and one named expression. A constraint can cap the expression, the objective can weight it, and a tool reads it back after the solver runs. | -| Helpers for one domain, such as `reduce_carrier_dim` | A macro already states a helper. | A [component library](../examples/library/index.md) of macros. Its macros use only the operators that exist. | -| A Python application programming interface (API) for building specs | The file is already the API, and the file is what you review and diff. | YAML, or a `dict` with the same keys. A tool can combine dicts before it calls `to_spec`. | -| A `join` operator apart from `sum` and `at`, such as `sum(join(p, generator[bus]), over=generator)` | [`sum(by=)` and `at(by=)` already state a join](relations-as-linear-maps.md#why-the-language-has-no-join-operator). | A sum or a lookup with `by=`, as in the first row. | +| Request | Why refused | What you write instead | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| A `where` comparing the label that each row has ([a mapping](../reference/language/relations.md#mappings)) against the dimension it refers to, such as `generator[bus] == bus` | Two dimensions cannot be compared, so mathspec refuses the comparison. The mapping already gives each row its label. | A sum or a lookup through the mapping. `sum(x, over=d, by=s[m])` adds each value of `x` to the label that the mapping gives it, and `at(x, by=s[m])` takes the value at that label ([operators](../reference/language/operators.md#sum)). | +| A vocabulary for tracked metrics: `impacts:`, `effects:`, a `costs` dimension | A tracked metric is already a [named expression](../reference/language/named.md#expressions). | An `impact` dimension and one named expression. A constraint can cap the expression, the objective can weight it, and a tool reads it back after the solver runs. | +| Helpers for one domain, such as `reduce_carrier_dim` | A macro already states a helper. | A [component library](../examples/library/index.md) of macros. Its macros use only the operators that exist. | +| A Python application programming interface (API) for building specs | The file is already the API, and the file is what you review and diff. | YAML, or a `dict` with the same keys. A tool can combine dicts before it calls `to_spec`. | +| A `join` operator apart from `sum` and `at`, such as `sum(join(p, generator[bus]), over=generator)` | [`sum(by=)` and `at(by=)` already state a join](relations-as-linear-maps.md#why-the-language-has-no-join-operator). | A sum or a lookup with `by=`, as in the first row. | +| A dimension of pairs, such as one index over the wired `(entity, bus)` pairs, or one assumption per parameter that refuses data at a pair that is not wired | A relation already lists the pairs, and its `missing:` already sets what a pair with no row means. | A relation over the pairs, named under [`in:`](../reference/language/parameters-variables-constraints.md#in) by each parameter or variable that it indexes. A tool refuses a data row of that parameter at a pair the relation does not list. | ### Data preparation diff --git a/docs/examples/sparse.md b/docs/examples/sparse.md index 1c13fec4..f183e2d9 100644 --- a/docs/examples/sparse.md +++ b/docs/examples/sparse.md @@ -17,9 +17,9 @@ the sets in one way: - `balance` sums over every row of `connection`, grouped by the bus each port feeds. -`efficiency` and `flow` exist only where `connection` lists a port, so -`missing: 1` fills only those ports, and a data row for a port that -`connection` does not list is dropped. +`efficiency` and `flow` list `connection` under `in:`, so they exist only +where it lists a port, `missing: 1` fills only those ports, and a data row +for a port that `connection` does not list is refused. [#1019](https://github.com/energy-models/mathspec/issues/1019) compares how three designs for sparse sets write this spec. @@ -50,14 +50,14 @@ mappings: bus: { dim: bus, description: the bus a port feeds } parameters: - efficiency: { dims: [unit, port, period], where: connection, missing: 1, description: share of the output a port delivers } + efficiency: { dims: [unit, port, period], in: [connection], missing: 1, description: share of the output a port delivers } capacity: { dims: [unit], description: the most a unit produces in a period } ramp: { dims: [unit], description: the most a unit's output rises from one period to the next } load: { dims: [bus, period], description: demand to be met } variables: - output: { dims: [unit, period], where: active, bounds: { lower: 0 }, description: what a unit produces } - flow: { dims: [unit, port, period], where: connection, bounds: { lower: 0 }, description: what a port takes of it } + output: { dims: [unit, period], in: [active], bounds: { lower: 0 }, description: what a unit produces } + flow: { dims: [unit, port, period], in: [connection], bounds: { lower: 0 }, description: what a port takes of it } constraints: port_limit: @@ -75,7 +75,7 @@ constraints: balance: description: what the ports deliver to a bus meets its load dims: [bus, period] - expression: sum(efficiency * flow, over=[unit, port], by=connection[bus]) >= load + expression: sum(efficiency * flow, over=connection, by=connection[bus]) >= load objective: sense: minimize @@ -98,7 +98,7 @@ Units that feed buses through the ports they have, and run only in the periods t | Symbol | Meaning | |---|---| | $`\mathrm{capacity}`$ | `capacity` over $`\mathcal{U}`$ — the most a unit produces in a period | -| $`\mathrm{efficiency}`$ | `efficiency` over $`\mathcal{T} \times \mathcal{O} \times \mathcal{U}`$, only where $`\left( o,\ u \right) \in \mathrm{connection}`$, $`1`$ where the data has no row — share of the output a port delivers | +| $`\mathrm{efficiency}`$ | `efficiency` over $`\left( o,\ u \right) \in \mathrm{connection},\ t \in \mathcal{T}`$, $`1`$ where the data has no row — share of the output a port delivers | | $`\mathrm{load}`$ | `load` over $`\mathcal{B} \times \mathcal{T}`$ — demand to be met | | $`\mathrm{ramp}`$ | `ramp` over $`\mathcal{U}`$ — the most a unit's output rises from one period to the next | @@ -106,21 +106,21 @@ Units that feed buses through the ports they have, and run only in the periods t | Symbol | Meaning | |---|---| -| $`\mathit{flow}`$ | `flow` over $`\mathcal{T} \times \mathcal{O} \times \mathcal{U}`$ — what a port takes of it | -| $`\mathit{output}`$ | `output` over $`\mathcal{T} \times \mathcal{U}`$ — what a unit produces | +| $`\mathit{flow}`$ | `flow` over $`\left( o,\ u \right) \in \mathrm{connection},\ t \in \mathcal{T}`$ — what a port takes of it | +| $`\mathit{output}`$ | `output` over $`\left( t,\ u \right) \in \mathrm{active}`$ — what a unit produces | #### Variable domains **`flow`** ```math -\mathit{flow}_{t,o,u} \ge 0 \qquad \forall\, t \in \mathcal{T},\ o \in \mathcal{O},\ u \in \mathcal{U} \,:\, \left( o,\ u \right) \in \mathrm{connection} +\mathit{flow}_{t,o,u} \ge 0 \qquad \forall\, \left( o,\ u \right) \in \mathrm{connection},\ t \in \mathcal{T} ``` **`output`** ```math -\mathit{output}_{t,u} \ge 0 \qquad \forall\, t \in \mathcal{T},\ u \in \mathcal{U} \,:\, \left( t,\ u \right) \in \mathrm{active} +\mathit{output}_{t,u} \ge 0 \qquad \forall\, \left( t,\ u \right) \in \mathrm{active} ``` #### Objective diff --git a/docs/examples/times/capacity.md b/docs/examples/times/capacity.md index c8824589..41686104 100644 --- a/docs/examples/times/capacity.md +++ b/docs/examples/times/capacity.md @@ -18,7 +18,8 @@ without storage, with the upper bound only. The GAMS conditions map to the spec as follows: - `RTP_VINTYR` and `PRC_TS` are [relations](../../reference/language/relations.md), - and `act` exists only at the coordinates that both list. + and `act` lists both under `in:`, so it exists only at the coordinates that + both list. - `TS_MAP(R,S,TS)` is the relation `ts_map`, which names `timeslice` twice under two column names. - `COEF_AF(…,'UP')` is the `where:` of `eq_capact_up`. @@ -70,7 +71,7 @@ variables: act: description: activity of a vintage of a process in a timeslice (VAR_ACT) dims: [region, vintage, period, process, timeslice] - where: rtp_vintyr AND prc_ts + in: [prc_ts, rtp_vintyr] bounds: { lower: 0 } constraints: @@ -121,7 +122,7 @@ Capacity transfer and capacity–activity from TIMES: EQ\_CPT and the upper form | Symbol | Meaning | |---|---| -| $`\mathit{act}`$ | `act` over $`\mathcal{T} \times \mathcal{P} \times \mathcal{R} \times \mathcal{S} \times \mathcal{V}`$ — activity of a vintage of a process in a timeslice (VAR\_ACT) | +| $`\mathit{act}`$ | `act` over $`\left( p,\ r,\ s \right) \in \mathrm{prc\_ts},\ \left( t,\ p,\ r,\ v \right) \in \mathrm{rtp\_vintyr}`$ — activity of a vintage of a process in a timeslice (VAR\_ACT) | | $`\mathit{cap}`$ | `cap` over $`\mathcal{T} \times \mathcal{P} \times \mathcal{R}`$ — installed capacity (VAR\_CAP) | | $`\mathit{ncap}`$ | `ncap` over $`\mathcal{P} \times \mathcal{R} \times \mathcal{V}`$ — new capacity (VAR\_NCAP) | @@ -130,7 +131,7 @@ Capacity transfer and capacity–activity from TIMES: EQ\_CPT and the upper form **`act`** ```math -\mathit{act}_{t,p,r,s,v} \ge 0 \qquad \forall\, t \in \mathcal{T},\ p \in \mathcal{P},\ r \in \mathcal{R},\ s \in \mathcal{S},\ v \in \mathcal{V} \,:\, \left( t,\ p,\ r,\ v \right) \in \mathrm{rtp\_vintyr} \wedge \left( p,\ r,\ s \right) \in \mathrm{prc\_ts} +\mathit{act}_{t,p,r,s,v} \ge 0 \qquad \forall\, \left( p,\ r,\ s \right) \in \mathrm{prc\_ts},\ \left( t,\ p,\ r,\ v \right) \in \mathrm{rtp\_vintyr} ``` **`cap`** diff --git a/docs/examples/times/trade.md b/docs/examples/times/trade.md index e96066bf..8a03acd6 100644 --- a/docs/examples/times/trade.md +++ b/docs/examples/times/trade.md @@ -14,10 +14,10 @@ which TIMES writes with a second name for each set, `ALIAS(REG,R)` in The spec does the same with an [alias](../../reference/language/dimensions.md#alias): `from_region` has the labels of `region`, and `from_commodity` those of `commodity`. The relation `top_ire` lists the processes that carry each link, -as TIMES' `TOP_IRE` does, and `ire_flo`, `trade_cost` and `export` exist only -at its rows. A row of `ire_flo` outside `top_ire` is dropped, not refused. -The data of `ire_flo` has the five columns that the TIMES data already has, -and no label is invented. +as TIMES' `TOP_IRE` does, and `ire_flo`, `trade_cost` and `export` name it +under `in:`, so a data row outside its rows is refused. The data of `ire_flo` +has the five columns that the TIMES data already has, and no label is +invented. `balance` adds the imports of each region and commodity over the far end of each link. It reads the exports at the near end: it sums them to each @@ -50,20 +50,20 @@ parameters: ire_flo: description: the share of an export that arrives as an import (IRE_FLO) dims: [from_region, from_commodity, region, commodity, process] - where: top_ire + in: [top_ire] demand: { dims: [commodity, region], description: what a region needs of a commodity } supply_cost: { dims: [commodity, region], description: the cost of a unit produced in a region } trade_cost: description: the cost of a unit exported along a link dims: [from_region, from_commodity, region, commodity, process] - where: top_ire + in: [top_ire] variables: supply: { dims: [commodity, region], bounds: { lower: 0 }, description: what a region produces of a commodity } export: description: what leaves along a link through a process (VAR_IRE) dims: [from_region, from_commodity, region, commodity, process] - where: top_ire + in: [top_ire] bounds: { lower: 0 } constraints: @@ -99,15 +99,15 @@ Trade between regions from TIMES: IRE\_FLO, the efficiency of a flow from one re | Symbol | Meaning | |---|---| | $`\mathrm{demand}`$ | `demand` over $`\mathcal{C} \times \mathcal{R}`$ — what a region needs of a commodity | -| $`\mathrm{ire\_flo}`$ | `ire_flo` over $`\mathcal{C} \times \mathcal{C} \times \mathcal{R} \times \mathcal{P} \times \mathcal{R}`$, only where $`\left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire}`$ — the share of an export that arrives as an import (IRE\_FLO) | +| $`\mathrm{ire\_flo}`$ | `ire_flo` over $`\left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire}`$ — the share of an export that arrives as an import (IRE\_FLO) | | $`\mathrm{supply}^{\mathrm{cost}}`$ | `supply_cost` over $`\mathcal{C} \times \mathcal{R}`$ — the cost of a unit produced in a region | -| $`\mathrm{trade\_cost}`$ | `trade_cost` over $`\mathcal{C} \times \mathcal{C} \times \mathcal{R} \times \mathcal{P} \times \mathcal{R}`$, only where $`\left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire}`$ — the cost of a unit exported along a link | +| $`\mathrm{trade\_cost}`$ | `trade_cost` over $`\left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire}`$ — the cost of a unit exported along a link | #### Variables | Symbol | Meaning | |---|---| -| $`\mathit{export}`$ | `export` over $`\mathcal{C} \times \mathcal{C} \times \mathcal{R} \times \mathcal{P} \times \mathcal{R}`$ — what leaves along a link through a process (VAR\_IRE) | +| $`\mathit{export}`$ | `export` over $`\left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire}`$ — what leaves along a link through a process (VAR\_IRE) | | $`\mathit{supply}`$ | `supply` over $`\mathcal{C} \times \mathcal{R}`$ — what a region produces of a commodity | #### Variable domains @@ -115,7 +115,7 @@ Trade between regions from TIMES: IRE\_FLO, the efficiency of a flow from one re **`export`** ```math -\mathit{export}_{c,c',r',p,r} \ge 0 \qquad \forall\, c \in \mathcal{C},\ c' \in \mathcal{C},\ r' \in \mathcal{R},\ p \in \mathcal{P},\ r \in \mathcal{R} \,:\, \left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire} +\mathit{export}_{c,c',r',p,r} \ge 0 \qquad \forall\, \left( c,\ c',\ r',\ p,\ r \right) \in \mathrm{top\_ire} ``` **`supply`** diff --git a/docs/howto/declare-a-column.md b/docs/howto/declare-a-column.md index 747ab03d..e27d718a 100644 --- a/docs/howto/declare-a-column.md +++ b/docs/howto/declare-a-column.md @@ -51,7 +51,7 @@ variables: dispatch: { dims: [snapshot, generator], bounds: { lower: 0, upper: availability } } reserve: dims: [snapshot, generator, bus] - where: connection + in: [connection] bounds: { lower: 0, upper: availability } constraints: diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 99b10600..3a1009f7 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -153,6 +153,14 @@ A group is the set of rows of a dimension or a relation to which one `sum_back` or `position()` work separately in each group, such as each season of a year ([partitions](language/relations.md#partitions)). +## Index set + +The index set of a parameter or a variable is the coordinates its `in:` lists: +the combinations of labels where each of its relations has a row. A mapping is +not an index set. A tool refuses a row of the data of a parameter +outside its index set +([`in`](language/parameters-variables-constraints.md#in)). + ## Label A label is one member of a dimension, such as `wind` in `generator`. The diff --git a/docs/reference/language/absence.md b/docs/reference/language/absence.md index a77e6fab..49287829 100644 --- a/docs/reference/language/absence.md +++ b/docs/reference/language/absence.md @@ -51,19 +51,20 @@ contain. ## What creates absence -Eight constructs make a coordinate absent: - -| Construct | What is absent | -| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | -| `where:` on a variable | the variable, at the coordinates the condition excludes ([`where`](#where)) | -| `where:` on a parameter | the parameter, at the coordinates the condition excludes ([`where` on a parameter](parameters-variables-constraints.md#where)) | -| `where:` on a constraint | the row ([`where`](#where)) | -| `shift(x, along=d, offset=n)` without `edge=` | the expression, at the edge coordinate ([shift](operators.md#shift)) | -| a key with no row in a `missing: absent` relation | the relation, at that key ([relations](relations.md#missing)) | -| a row with no label in a `missing: absent` mapping | the mapping, at that row ([mappings](relations.md#missing)) | -| a missing row of a `missing: absent` parameter | the parameter, at that coordinate ([`missing`](#missing)) | -| `within=` at a coordinate in no group | the expression, at that coordinate ([`within=`](operators.md#within)) | -| a quotient whose divisor is zero | the quotient, at that coordinate ([reported values](#reported-values)) | +Nine constructs make a coordinate absent: + +| Construct | What is absent | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `where:` on a variable | the variable, at the coordinates the condition excludes ([`where`](#where)) | +| `where:` on a parameter | the parameter, at the coordinates the condition excludes ([`where` on a parameter](parameters-variables-constraints.md#where)) | +| `where:` on a constraint | the row ([`where`](#where)) | +| `shift(x, along=d, offset=n)` without `edge=` | the expression, at the edge coordinate ([shift](operators.md#shift)) | +| a key with no row in a `missing: absent` relation | the relation, at that key ([relations](relations.md#missing)) | +| a row with no label in a `missing: absent` mapping | the mapping, at that row ([mappings](relations.md#missing)) | +| a key with no row in a relation in `in:` | the parameter or variable, at that key ([`in`](parameters-variables-constraints.md#in)) | +| a missing row of a `missing: absent` parameter | the parameter, at that coordinate ([`missing`](#missing)) | +| `within=` at a coordinate in no group | the expression, at that coordinate ([`within=`](operators.md#within)) | +| a quotient whose divisor is zero | the quotient, at that coordinate ([reported values](#reported-values)) | No value shifts in at the edge, so a `shift` without `edge=` has no row there: @@ -228,8 +229,9 @@ as [missing rows](../reading.md#missing-rows) describes. ### `missing` on a variable -Outside its `where:`, a variable is absent by default. Its `missing:` key can -set a number there instead, such as `0`. Under `absent`, a row that needs that +Outside its `where:` and its [`in:`](parameters-variables-constraints.md#in-on-a-variable), +a variable is absent by default. Its `missing:` key can set a number there +instead, such as `0`. Under `absent`, a row that needs that coordinate is absent. A file declares `missing: 0` for a quantity that is zero outside its condition: diff --git a/docs/reference/language/dimensions.md b/docs/reference/language/dimensions.md index 5af7b16c..641da6eb 100644 --- a/docs/reference/language/dimensions.md +++ b/docs/reference/language/dimensions.md @@ -171,7 +171,7 @@ dimensions: relations: trade: { dims: [from_region, region], missing: absent } parameters: - ire_flo: { dims: [from_region, region], where: trade } + ire_flo: { dims: [from_region, region], in: [trade] } variables: exp: { dims: [region] } imp: { dims: [region] } diff --git a/docs/reference/language/expressions.md b/docs/reference/language/expressions.md index 963dcfb7..3936f5cb 100644 --- a/docs/reference/language/expressions.md +++ b/docs/reference/language/expressions.md @@ -221,15 +221,15 @@ mathspec suggests the rewrite: a new name for one of the two entries. The place of a name in a string sets which kinds of name it may be: -| Place | Kinds of name it accepts | -| -------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -| expression (`p * cost`) | a variable, or a parameter whose values are numbers ([dtype](parameters-variables-constraints.md#parameters)) | -| dimension argument (`over=`, `along=`) | a dimension, a list of them for `over=`, or with `by=` a column of the table in `by=` where one has that name | -| column argument (`by=`, `within=`) | mappings or columns of one subject, written `subject[column]` or `subject[column, …]` | -| condition | a parameter, variable, dimension, relation, mapping or named condition ([conditions](#conditions)) | -| `bounds.lower` / `bounds.upper` | a parameter name, or a number | -| the `edge` key of `shift` | `'wrap'` in quotes, or a bare number | -| `dual` argument (`dual(c)`) | a constraint, resolved against the constraints alone ([named expressions](named.md#dual)) | +| Place | Kinds of name it accepts | +| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| expression (`p * cost`) | a variable, or a parameter whose values are numbers ([dtype](parameters-variables-constraints.md#parameters)) | +| dimension argument (`over=`, `along=`) | a dimension, a list of them for `over=`, a relation for `over=` (its dimensions), or with `by=` a column of the table in `by=` where one has that name | +| column argument (`by=`, `within=`) | mappings or columns of one subject, written `subject[column]` or `subject[column, …]` | +| condition | a parameter, variable, dimension, relation, mapping or named condition ([conditions](#conditions)) | +| `bounds.lower` / `bounds.upper` | a parameter name, or a number | +| the `edge` key of `shift` | `'wrap'` in quotes, or a bare number | +| `dual` argument (`dual(c)`) | a constraint, resolved against the constraints alone ([named expressions](named.md#dual)) | A bare word in the value of a keyword argument is a name, and mathspec looks it up. The key of a keyword argument is never a name, so `over` in @@ -322,26 +322,26 @@ QUOTED ::= "'" chars "'" | '"' chars '"' A name written alone is a bare name, and what it tests depends on the kind of its entry. -| Written as | Kind of name | Meaning | -| ---------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `name` (bare) | parameter | For a number, it is true where the data has a row, whatever the `missing:`. For a `bool`, it is its value, and the `missing:` sets the value at a missing row ([`missing`](parameters-variables-constraints.md#missing)). `0.0` is a row, and `inflow != 0` tests for non-zero | -| `name` (bare) | variable | It is true where the variable exists | -| `name` (bare) | relation | It is true where the relation has a row. A `missing: absent` relation has no row for some combinations ([`missing`](relations.md#missing)), and the bare name selects the combinations it lists | -| `subject[mapping]` (bare) | mapping | It is true where the mapping gives the row of its subject a label. A `missing: absent` mapping gives some rows no label ([`missing`](relations.md#missing)) | -| `name` (bare) | dimension | mathspec refuses it at load | -| `name` (bare) | named condition | It tests the [named condition](named.md#conditions). A given condition is true or false at each coordinate, as a `bool` parameter is | -| `name OP value` | parameter | It compares element by element. A missing row with no `missing:` value compares false | -| `name OP value` | dimension | It filters the frame's own coordinate column | -| `subject[mapping] OP value` | mapping | It compares the label that the mapping gives each row of its subject. A comparison reads one mapping. Where the mapping gives no label, the comparison is false for every operator, `!=` included, so `NOT` of it is true there | -| `s[a] OP t[b]` | two mappings | mathspec accepts it where both subjects are over the same dimensions and both mappings refer to one dimension. `line[bus0] != line[bus1]` excludes a self-loop. Where either mapping gives no label, the comparison is false | -| `expression OP expression` | arithmetic over parameters | It compares coordinate by coordinate, over every dimension either side is over ([arithmetic in a comparison](#arithmetic-in-a-comparison)). A side with no value at a coordinate compares false | -| `position(name) OP i` | dimension | It tests where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | -| `position(name, within=s[m])` | dimension | It tests the same, counted within each group the mapping makes | -| `count(condition, over=name) OP i` | condition | It compares how many coordinates along the dimension the condition admits ([`count`](#count)) | -| `shift(condition, along=name, offset=i)` | condition | It is the condition `i` coordinates back, and false at the first `i` coordinates | -| `at(condition, by=s[m])` | condition | It is the condition through the mapping ([`at` in a condition](#at-in-a-condition)), and false where the mapping gives no label | -| `AND` `OR` `NOT` | | They are case-insensitive. `NOT` applies before `AND`, and `AND` before `OR` | -| `True` / `False` | | `True` is the same as no `where`. `False` leaves an entry with no rows | +| Written as | Kind of name | Meaning | +| ---------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `name` (bare) | parameter | For a number, it is true where the data has a row, whatever the `missing:`. For a `bool`, it is its value, and the `missing:` sets the value at a missing row ([`missing`](parameters-variables-constraints.md#missing)). `0.0` is a row, and `inflow != 0` tests for non-zero | +| `name` (bare) | variable | It is true where the variable exists | +| `name` (bare) | relation | It is true where the relation has a row. A `missing: absent` relation has no row for some combinations ([`missing`](relations.md#missing)), and the bare name selects the combinations it lists. The `where:` of a parameter or a variable cannot hold it, because [`in:`](parameters-variables-constraints.md#in) states those rows | +| `subject[mapping]` (bare) | mapping | It is true where the mapping gives the row of its subject a label. A `missing: absent` mapping gives some rows no label ([`missing`](relations.md#missing)) | +| `name` (bare) | dimension | mathspec refuses it at load | +| `name` (bare) | named condition | It tests the [named condition](named.md#conditions). A given condition is true or false at each coordinate, as a `bool` parameter is | +| `name OP value` | parameter | It compares element by element. A missing row with no `missing:` value compares false | +| `name OP value` | dimension | It filters the frame's own coordinate column | +| `subject[mapping] OP value` | mapping | It compares the label that the mapping gives each row of its subject. A comparison reads one mapping. Where the mapping gives no label, the comparison is false for every operator, `!=` included, so `NOT` of it is true there | +| `s[a] OP t[b]` | two mappings | mathspec accepts it where both subjects are over the same dimensions and both mappings refer to one dimension. `line[bus0] != line[bus1]` excludes a self-loop. Where either mapping gives no label, the comparison is false | +| `expression OP expression` | arithmetic over parameters | It compares coordinate by coordinate, over every dimension either side is over ([arithmetic in a comparison](#arithmetic-in-a-comparison)). A side with no value at a coordinate compares false | +| `position(name) OP i` | dimension | It tests where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | +| `position(name, within=s[m])` | dimension | It tests the same, counted within each group the mapping makes | +| `count(condition, over=name) OP i` | condition | It compares how many coordinates along the dimension the condition admits ([`count`](#count)) | +| `shift(condition, along=name, offset=i)` | condition | It is the condition `i` coordinates back, and false at the first `i` coordinates | +| `at(condition, by=s[m])` | condition | It is the condition through the mapping ([`at` in a condition](#at-in-a-condition)), and false where the mapping gives no label | +| `AND` `OR` `NOT` | | They are case-insensitive. `NOT` applies before `AND`, and `AND` before `OR` | +| `True` / `False` | | `True` is the same as no `where`. `False` leaves an entry with no rows | With a dimension `bp` declared, mathspec prints for `where: "bp"`: diff --git a/docs/reference/language/parameters-variables-constraints.md b/docs/reference/language/parameters-variables-constraints.md index 0aeb5f5e..8d4c15d7 100644 --- a/docs/reference/language/parameters-variables-constraints.md +++ b/docs/reference/language/parameters-variables-constraints.md @@ -280,6 +280,7 @@ because `dims: []` declares a scalar. | `dims` | required. The dimensions it is indexed over. `[]` means a scalar | | | `dtype` | `float`, `int`, `bool` or `str` | `float` | | `missing` | what a missing row of the data means ([`missing`](#missing)) | `refused` | +| `in` | the relations of its index set ([`in`](#in)) | `[]` | | `where` | the condition that limits its coordinates ([`where`](#where)) | `null` | With data attached, a tool requires the column to match the `dtype`: @@ -315,7 +316,8 @@ A `str` parameter takes any label. A dimension with `missing:` sets the meaning of a coordinate of the `dims` at which the data has no row. It takes `refused` (the default), `absent`, or a value of the `dtype` of the parameter. On a parameter with a [`where:`](#where), `missing:` applies -only where the condition is true. +only where the condition is true. On a parameter with an [`in:`](#in), it +applies only where each relation has a row. ```yaml dimensions: @@ -449,17 +451,25 @@ snapshot of the example, and drops every other row. A tool drops a row of the data outside the condition without refusing the data. An [assumption](assumptions.md) on the parameter does not test that row -either, because the parameter has no row there. +either, because the parameter has no row there. To have a tool refuse a row at +a pair that is not wired, write [`in: [connection]`](#in). A relation name +alone in the `where:` would state the same rows a second time, so mathspec +refuses it. For `where: connection` on `efficiency`, the message is: + +```text +Parameter 'efficiency': `where: connection` tests the rows of 'connection', which `in:` states. Write `in: [connection]` in place of the `where:`. +``` A tool evaluates the condition from the data alone, before it reads the parameter. So the condition may contain [dimensions](dimensions.md), -[relations](relations.md) and parameters without a `where:`. +[relations](relations.md) and parameters with neither a `where:` nor an +`in:`. mathspec refuses four conditions at load: - a condition that contains a variable - a condition that contains the parameter itself -- a condition that contains a parameter with a `where:` of its own +- a condition that contains a parameter with a `where:` or an `in:` of its own - a condition over a dimension that the parameter does not have For a `where:` on `efficiency` that contains the variable `p`, the message @@ -475,6 +485,136 @@ The typeset legend lists the condition beside the dimensions, after `only where`. For `efficiency`, the legend entry ends in `only where` $\mathrm{role}(b,\ e) = \text{'}\mathrm{output}\text{'}$. +### `in` + +`in:` lists the relations that make up the index set of the parameter. The +parameter has coordinates only where each relation has a row, and a tool +refuses a row of the data anywhere else. A [`where:`](#where) drops that row +instead. In this example, `efficiency` and `p_nom_max` have values only at the +pairs that `connection` lists: + +```yaml +dimensions: + entity: { dtype: str } + bus: { dtype: str } + snapshot: { dtype: int } +relations: + connection: { dims: [entity, bus], missing: absent } +parameters: + efficiency: { dims: [entity, bus, snapshot], in: [connection], missing: 1 } + p_nom_max: { dims: [entity, bus], in: [connection], missing: .inf } + distance: { dims: [entity, bus] } +variables: + p: { dims: [entity, snapshot] } +constraints: + balance: + dims: [bus, snapshot] + expression: sum(efficiency * p, over=entity) == 0 +``` + +Each parameter names its own relations. `distance` is also over `entity` and +`bus`, names none, and needs a row at every pair. + +With data attached, a tool decides the coordinates of `efficiency` in three +steps: + +1. Where `connection` has no row, `efficiency` is [absent](absence.md), and + the tool refuses any row of the data of `efficiency` there. +2. Where the [`where:`](#where) of `efficiency` is false, the tool drops the + row. `efficiency` has no `where:`, so the tool drops nothing. +3. At each remaining coordinate with no row, `missing: 1` sets the value. + +So `missing: 1` fills only the wired pairs, and a data row for an entity at a +bus it is not wired to is refused. + +A parameter in several relations has coordinates only where each of them has +a row. + +A relation in `in:` gives its rows, and its [mappings](relations.md#mappings) play no +part in the index set. They stay readable as `connection[bus]` in a `where:` +and in a call. In this example, `connection` has a row per port of a unit, and +its mappings `bus` and `role` give each port a bus and a role: + +```yaml +dimensions: + unit: { dtype: str } + port: { dtype: int } + bus: { dtype: str } + role: { dtype: str } + snapshot: { dtype: int } +relations: + connection: { dims: [unit, port], missing: absent } +mappings: + connection: { bus: { dim: bus }, role: { dim: role } } +parameters: + efficiency: + dims: [unit, port, snapshot] + in: [connection] + where: "connection[role] == 'output'" + missing: 1 +variables: + flow: { dims: [unit, snapshot], bounds: { lower: 0 } } +constraints: + delivered: + dims: [bus, snapshot] + expression: sum(efficiency * flow, over=[unit, port], by=connection[bus]) >= 0 +``` + +`efficiency` has coordinates at each port that `connection` lists, and a tool +refuses a data row for a port that `connection` does not list. The `where:` +then drops the input ports. `efficiency` is not over `bus` or `role`, the +dimensions that the mappings refer to. An entry over one of them has a coordinate +at every label of that dimension, not only at the label that the mapping gives the +port. + +mathspec refuses seven index sets at load: + +- a name that is not a relation, such as a dimension +- a relation named twice +- a mapping, written as `subject[mapping]`, in place of a relation +- a relation under `missing: refused`, which has a row at every combination + and so leaves nothing out +- a relation with two columns over one dimension +- a relation over a dimension that the parameter does not have +- a `where:` that reads the bare name of a relation of the `in:` + +For `p_nom_max: { dims: [entity], in: [connection] }`, the message is: + +```text +Parameter 'p_nom_max': in: names 'connection', over ['bus', 'entity'], and 'p_nom_max' is not over ['bus']. Add ['bus'] to its dims:, or delete 'connection' from in:. +``` + +mathspec suggests both rewrites. For `in: ["connection[bus]"]` on +`efficiency` in the second example, the message is: + +```text +Parameter 'efficiency': in: names 'connection[bus]', a mapping of 'connection', and an index set is a relation. To keep the rows the mapping gives a label, write where: connection[bus]. +``` + +A `where:` on a parameter with an `in:` narrows the coordinates inside the +index set, and the tool drops the rows outside it, as in step 2. Inside the +index set, the bare name of each relation of the `in:` is true. So mathspec +refuses a `where:` that reads one, and names what the `where:` is left with. +`where: connection` on `efficiency` in the first example is always true: + +```text +Parameter 'efficiency': in: gives 'efficiency' coordinates only where 'connection' has a row, so `where: connection` narrows nothing. Delete the `where:`, or the relation from it. +``` + +A `where:` that is always false, such as `where: NOT connection`, leaves the +parameter no coordinate, and mathspec suggests a `where:` that holds somewhere +inside the relation. For `where: "connection AND active"`, where `active` is a +`bool` parameter, the message ends in +`` `where: connection AND active` repeats the relation. Delete the relation from it. `` +A test of a mapping, such as `connection[role] == 'output'`, is not the bare name +of the relation, so it loads. + +The typeset legend prints the index set as the domain of the parameter, after +`over`. For `efficiency` in the first example, the legend entry starts with +`efficiency` over $(b, e) \in \mathrm{connection},\ t \in \mathcal{T}$. In +the second example, it starts with `efficiency` over +$(p, u) \in \mathrm{connection},\ t \in \mathcal{T}$, without the mappings. + ## `variables` A **variable** is a quantity that the solver decides, with one value per @@ -501,6 +641,7 @@ capacity of its generator. | Key | What it sets | Default | | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------ | | `dims` | required. The dimensions it is indexed over | | +| `in` | the relations of its index set ([`in`](#in-on-a-variable)) | `[]` | | `where` | which coordinates exist ([absence](absence.md)) | `null` | | `bounds.lower` / `bounds.upper` | a finite number, or the name of a `float` or `int` parameter. `null` leaves that side open | `null` | | `domain` | `continuous`, `integer` or `binary`. `binary` has fixed bounds of 0 and 1 | `continuous` | @@ -527,6 +668,54 @@ bounds pin a variable to one value ([fix a quantity](../../howto/pin-a-variable.md)), and a pinned variable is still a variable. +### `in` on a variable + +A variable takes [`in:`](#in) as a parameter does: it exists only where each +relation has a row. In this example, `flow` exists at the pairs that +`connection` lists, at every snapshot: + +```yaml +dimensions: + generator: { dtype: str } + bus: { dtype: str } + snapshot: { dtype: int } +relations: + connection: { dims: [generator, bus], missing: absent } +parameters: + load: { dims: [bus, snapshot] } +variables: + flow: + dims: [generator, bus, snapshot] + in: [connection] + bounds: { lower: 0 } +constraints: + balance: + dims: [bus, snapshot] + expression: sum(flow, over=generator) == load +``` + +$$\mathit{flow}_{b,g,t} \ge 0 \qquad \forall\, \left( b,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T}$$ + +A variable has no data, so nothing is refused, and `in: [connection]` gives +`flow` the coordinates that `where: connection` gives it. mathspec refuses +the same index sets at load as on a parameter, and a `where:` on the variable +narrows inside the index set. A `missing:` number stands for the variable +outside its `in:`, as outside a `where:`. + +mathspec refuses a `where:` that tests the rows of a relation, as on a +parameter, because `in:` states them. For `where: connection` on `flow`, the +message is: + +```text +Variable 'flow': `where: connection` tests the rows of 'connection', which `in:` states. Write `in: [connection]` in place of the `where:`. +``` + +For `where: "connection AND active"`, the rewrite is `in: [connection]`, with +`active` kept in the `where:`. Under `OR` or `NOT`, `in:` cannot state the +rows, and mathspec suggests a `bool` parameter in their place. A +[mapping](relations.md#mappings) tested alone, such as `connection[bus]`, asks for a +label rather than a row, so it loads. + ## `objective` The **objective** is the one number that the solver minimises or maximises. diff --git a/docs/reference/language/relations.md b/docs/reference/language/relations.md index 4fbae06a..981e6554 100644 --- a/docs/reference/language/relations.md +++ b/docs/reference/language/relations.md @@ -54,13 +54,17 @@ dimension, and a mapping or a condition states that. With Relation 'r' has one column under 'dims:', and a relation needs at least two. To give each generator a label, declare a mapping under mappings: generator:. To mark some of them, declare a bool parameter or a condition. ``` -A relation is not a dimension, so its name does not go in `dims:`, `over=` or +A relation is not a dimension, so its name does not go in `dims:` or `along=`. With `flow: { dims: [connection] }`, mathspec prints: ```text -Variable 'flow' names the relation 'connection', and a relation is not a dimension. Name its dimensions, ['bus', 'generator'], under dims:, and confine the entry with where: connection. +Variable 'flow' names the relation 'connection', and a relation is not a dimension. Name its dimensions, ['bus', 'generator'], under dims:, and confine the entry with in: [connection]. ``` +In `over=`, a relation stands for its dimensions. `sum(flow, over=connection)` +is `sum(flow, over=[bus, generator])`, and a dimension of the relation that is +also named in the list is named twice and refused. + The order of the columns under `dims:` changes nothing. The typesetter sorts the columns by name, so `connection` above prints as $`\mathrm{connection} \subseteq \mathcal{B} \times \mathcal{G}`$. @@ -199,6 +203,13 @@ A relation and a mapping have no `where:`, so their rows and labels come from th data alone. A tool therefore knows them before it evaluates any condition, such as the [`where:` of a parameter](parameters-variables-constraints.md#where). +The rows of a relation can also be the coordinates of a parameter or a +variable, its [index set](../glossary.md#index-set). An entry with +`in: [connection]` has coordinates only at the pairs that `connection` lists, +and a tool refuses a row of the data of a parameter at any other pair +([`in`](parameters-variables-constraints.md#in)). A mapping is not an index set, +because it adds no row to its subject. + An [assumption](assumptions.md) can require rows of a relation or labels of a mapping. For `line[bus1]` [above](#missing), `{holds: "line[bus1]", where: "closed"}` requires an end for every closed line, where `closed` is a `bool` parameter @@ -274,7 +285,10 @@ Five more rules apply to a sum through a table: - The operand is over every column that the call joins on, and over none that it groups by. A column that the operand is over is joined on. -- A name in `over=` is a column of the table, or else a dimension. With the +- A name in `over=` is a column of the table, or else a dimension. A relation + in `over=` stands for its dimensions, so + `sum(x, over=connection, by=connection[bus])` joins on each column of + `connection` that is over one of them. With the mapping `rep` of `snapshot`, `sum(x, over=snapshot, by=snapshot[rep])` joins on the key column `snapshot` and groups by `rep`. Where two columns are over one dimension, the name selects which one, so @@ -382,8 +396,9 @@ Expected end of text, found '[' (at char 8), (line:1, col:9) A mapping alone in a condition, such as `line[bus1]`, is true where the mapping gives the row a label. A relation name alone is true where the relation has a row. -On a variable, the condition leaves the variable absent at every pair that the -relation does not list: +On a parameter or a variable, [`in:`](parameters-variables-constraints.md#in) +states those rows, so mathspec refuses the name in its `where:`. With `in:`, a +variable is absent at every pair that the relation does not list: ```yaml title="flow.yaml" dimensions: @@ -397,7 +412,7 @@ parameters: variables: flow: dims: [generator, bus, snapshot] - where: connection + in: [connection] bounds: { lower: 0 } constraints: balance: @@ -409,8 +424,13 @@ constraints: --8<-- "docs/static/relations-where-bare.svg" -`flow` exists at the pairs that `connection` lists and is [absent](absence.md) -at every other pair, so `balance` adds up the generators wired to each bus. +`flow` exists at the pairs that `connection` lists and is +[absent](absence.md) at every other pair, so `balance` adds up the generators +wired to each bus. `where: connection` in place of the `in:` is refused: + +```text +Variable 'flow': `where: connection` tests the rows of 'connection', which `in:` states. Write `in: [connection]` in place of the `where:`. +``` `python -m mathspec markdown flow.yaml` prints the domain of `flow` and the constraint: diff --git a/docs/reference/notation.md b/docs/reference/notation.md index 9d18cd9d..a0806d37 100644 --- a/docs/reference/notation.md +++ b/docs/reference/notation.md @@ -37,8 +37,9 @@ dimensions: bp: { dtype: int, ordered: true } # the breakpoints every curve below runs through relations: - gen_zone: { dims: [generator, snapshot] } # a relation over two dimensions, with a mapping: a call sums one away and joins on the other - connection: { dims: [generator, bus], missing: absent } # a relation with no mapping: many-to-many, read only by sum with both ends named + gen_zone: { dims: [generator, snapshot] } # a relation over two dimensions, with a map: a call sums one away and joins on the other + connection: { dims: [generator, bus], missing: absent } # a relation with no map: many-to-many, read only by sum with both ends named + bus_zone: { dims: [bus, zone], missing: absent } # the pairs that exist mappings: generator: { bus: { dim: bus }, technology: { dim: technology } } # two mappings of one subject, read to both at once @@ -58,7 +59,8 @@ parameters: is_flexible: { dims: [generator], dtype: bool, missing: false } zone_cap: { dims: [zone] } tech_cap: { dims: [bus, technology], missing: absent } - link_eff: { dims: [generator, bus], where: connection, missing: 1 } + link_eff: { dims: [generator, bus], in: [connection], missing: 1 } # over the pairs a relation lists; a `where:` may not test them + zone_share: { dims: [bus, zone], in: [bus_zone], missing: 1 } # over the pairs that exist: data refused outside them, with no `where:` of its own susceptance: { dims: [from_bus, bus], missing: 0 } # over one set twice, through its alias area_weight: { dims: [bus], where: "bus[area]" } # a mapping's label as a condition: the buses that have an area min_up: { dims: [generator], dtype: int } @@ -77,13 +79,13 @@ parameters: | Symbol | Meaning | |---|---| | $`\mathcal{B}`$ | index $`b`$ — `bp` | -| $`\mathcal{U}`$ | index $`u`$ — `bus` with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U}`$, `connection` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label | +| $`\mathcal{U}`$ | index $`u`$ — `bus` with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{bus\_zone} \subseteq \mathcal{U} \times \mathcal{Z},\ \mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U}`$, `bus_zone` is `absent` where the data has no row, `connection` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label | | $`\mathcal{U}`$ | index $`u'`$ — `from_bus`, an alias of `bus` | | $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{gen\_zone} \subseteq \mathcal{G} \times \mathcal{T},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U},\ \mathrm{technology}: \mathcal{G} \to \mathcal{E}`$, `connection` is `absent` where the data has no row | | $`\mathcal{S}`$ | index $`s`$ — `season` with $`\mathrm{season}: \mathcal{T} \to \mathcal{S}`$ | | $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{gen\_zone} \subseteq \mathcal{G} \times \mathcal{T},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z},\ \mathrm{rep}: \mathcal{T} \to \mathcal{T},\ \mathrm{season}: \mathcal{T} \to \mathcal{S}`$ | | $`\mathcal{E}`$ | index $`e`$ — `technology` with $`\mathrm{technology}: \mathcal{G} \to \mathcal{E}`$ | -| $`\mathcal{Z}`$ | index $`z`$ — `zone` (labels `north`, `south`) with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z}`$, `bus[area]` is `absent` where the data gives no label | +| $`\mathcal{Z}`$ | index $`z`$ — `zone` (labels `north`, `south`) with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{bus\_zone} \subseteq \mathcal{U} \times \mathcal{Z},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z}`$, `bus_zone` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label | #### Parameters @@ -100,7 +102,7 @@ parameters: | $`\mathrm{growth}`$ | `growth` (scalar) | | $`\mathrm{is\_flexible}`$ | `is_flexible` over $`\mathcal{G}`$, `false` where the data has no row | | $`\mathrm{lead}`$ | `lead` over $`\mathcal{G}`$ | -| $`\mathrm{link\_eff}`$ | `link_eff` over $`\mathcal{U} \times \mathcal{G}`$, only where $`\left( u,\ g \right) \in \mathrm{connection}`$, $`1`$ where the data has no row | +| $`\mathrm{link\_eff}`$ | `link_eff` over $`\left( u,\ g \right) \in \mathrm{connection}`$, $`1`$ where the data has no row | | $`\mathrm{load}`$ | `load` over $`\mathcal{U} \times \mathcal{T}`$ | | $`\mathrm{min\_up}`$ | `min_up` over $`\mathcal{G}`$ | | $`\mathrm{p}^{\mathrm{max}}`$ | `p_max` over $`\mathcal{G}`$, $`\infty`$ where the data has no row | @@ -108,6 +110,7 @@ parameters: | $`\mathrm{susceptance}`$ | `susceptance` over $`\mathcal{U} \times \mathcal{U}`$, $`0`$ where the data has no row | | $`\mathrm{tech\_cap}`$ | `tech_cap` over $`\mathcal{U} \times \mathcal{E}`$, `absent` where the data has no row | | $`\mathrm{zone\_cap}`$ | `zone_cap` over $`\mathcal{Z}`$ | +| $`\mathrm{zone\_share}`$ | `zone_share` over $`\left( u,\ z \right) \in \mathrm{bus\_zone}`$, $`1`$ where the data has no row | #### Conditions @@ -120,6 +123,7 @@ parameters: | Symbol | Meaning | |---|---| +| $`\mathit{flow}`$ | `flow` over $`\left( u,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T}`$ | | $`\mathit{fuel}`$ | `fuel` over $`\mathcal{G} \times \mathcal{T}`$ | | $`\mathit{headroom}`$ | `headroom` (scalar), $`0`$ where the mask leaves it out | | $`\mathit{heat}`$ | `heat` over $`\mathcal{G} \times \mathcal{T}`$ | @@ -292,6 +296,23 @@ variables: \mathit{headroom} \ge 0 \qquad \text{where } \mathrm{budget} \text{ is defined} ``` +#### Variable over the rows of a relation + +a domain of the pairs `connection` lists, at every snapshot, and a `where:` inside it + +```yaml +variables: + flow: + dims: [snapshot, generator, bus] + in: [connection] + where: "p_max > 0" + bounds: { lower: 0 } +``` + +```math +\mathit{flow}_{u,g,t} \ge 0 \qquad \forall\, \left( u,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T} \,:\, \mathrm{p}^{\mathrm{max}}_{g} > 0 +``` + #### Variable in a special ordered set the family a sos runs along diff --git a/docs/reference/reading.md b/docs/reference/reading.md index 41908025..56da6fb1 100644 --- a/docs/reference/reading.md +++ b/docs/reference/reading.md @@ -185,13 +185,29 @@ comparison in a condition. Under `'absent'`, a tool leaves a bound open where the parameter has no row. `program.parameters[name].where` is a [`Mask`](#nodes-and-conditions) for the -`where:` of the parameter, or `None` when the parameter has every coordinate -of its `dims`. With data attached, an engine applies the condition before -`missing:`, as +coordinates of the parameter, or `None` when the parameter has every +coordinate of its `dims`. It contains each relation of the `in:` of the +parameter and its `where:`. With data attached, an engine applies the +condition before `missing:`, as [`where` on a parameter](language/parameters-variables-constraints.md#where) describes. The condition contains no variable and no parameter with a `where:` -of its own. So an engine can evaluate every such condition before it reads any -parameter that has a `where:`. +or an `in:` of its own. So an engine can evaluate every such condition before +it reads any parameter that has one. + +`program.parameters[name].in_` lists the relations of the `in:` of the +parameter, sorted by name. The attribute ends in `_` because `in` is a Python +keyword. With data attached, an engine refuses a row of the parameter at a +combination that one of them has no row for, and reads no mapping. It does this +before it applies the `where`, and so before `missing:`, as +[`in`](language/parameters-variables-constraints.md#in) describes. An engine +drops a row outside the rest of the `where`, and refuses a row outside the +relations. + +`program.variables[name].where` is the same kind of `Mask` for the coordinates +of the variable, with the relations of its `in:` in it, and +`program.variables[name].in_` lists those relations. A variable has no data, so +an engine builds it where the `where` is true and needs nothing more from +`in_`. With data attached, an engine must refuse a divisor that is `0` in a row it builds, and name the divisor. The same applies to a term of a sum that it diff --git a/docs/static/relations-where-bare.svg b/docs/static/relations-where-bare.svg index 9d1806b5..09eae35c 100644 --- a/docs/static/relations-where-bare.svg +++ b/docs/static/relations-where-bare.svg @@ -3,8 +3,8 @@ SPDX-FileCopyrightText: mathspec contributors SPDX-License-Identifier: CC-BY-4.0 --> - A bare relation in a where condition keeps the pairs it lists - The bare relation connection is a table with three rows: wind and north, gas and north, and gas and south. The variable flow, declared with where connection, is a table with the same three pairs of generator and bus. The pair wind and south, which connection does not list, has no row, so flow is absent there. + A relation in in: gives the pairs it lists + The relation connection is a table with three rows: wind and north, gas and north, and gas and south. The variable flow, declared with in connection, is a table with the same three pairs of generator and bus. The pair wind and south, which connection does not list, has no row, so flow is absent there. @@ -36,8 +36,8 @@ SPDX-License-Identifier: CC-BY-4.0 - where: connection - keeps the pairs + in: [connection] + gives the pairs the relation lists diff --git a/examples/sparse.yaml b/examples/sparse.yaml index cc242523..93d6c6f7 100644 --- a/examples/sparse.yaml +++ b/examples/sparse.yaml @@ -27,14 +27,14 @@ mappings: bus: { dim: bus, description: the bus a port feeds } parameters: - efficiency: { dims: [unit, port, period], where: connection, missing: 1, description: share of the output a port delivers } + efficiency: { dims: [unit, port, period], in: [connection], missing: 1, description: share of the output a port delivers } capacity: { dims: [unit], description: the most a unit produces in a period } ramp: { dims: [unit], description: the most a unit's output rises from one period to the next } load: { dims: [bus, period], description: demand to be met } variables: - output: { dims: [unit, period], where: active, bounds: { lower: 0 }, description: what a unit produces } - flow: { dims: [unit, port, period], where: connection, bounds: { lower: 0 }, description: what a port takes of it } + output: { dims: [unit, period], in: [active], bounds: { lower: 0 }, description: what a unit produces } + flow: { dims: [unit, port, period], in: [connection], bounds: { lower: 0 }, description: what a port takes of it } constraints: port_limit: @@ -52,7 +52,7 @@ constraints: balance: description: what the ports deliver to a bus meets its load dims: [bus, period] - expression: sum(efficiency * flow, over=[unit, port], by=connection[bus]) >= load + expression: sum(efficiency * flow, over=connection, by=connection[bus]) >= load objective: sense: minimize diff --git a/examples/times/capacity.yaml b/examples/times/capacity.yaml index f8b54494..3437f595 100644 --- a/examples/times/capacity.yaml +++ b/examples/times/capacity.yaml @@ -42,7 +42,7 @@ variables: act: description: activity of a vintage of a process in a timeslice (VAR_ACT) dims: [region, vintage, period, process, timeslice] - where: rtp_vintyr AND prc_ts + in: [prc_ts, rtp_vintyr] bounds: { lower: 0 } constraints: diff --git a/examples/times/trade.yaml b/examples/times/trade.yaml index 5401620c..14b82c83 100644 --- a/examples/times/trade.yaml +++ b/examples/times/trade.yaml @@ -24,20 +24,20 @@ parameters: ire_flo: description: the share of an export that arrives as an import (IRE_FLO) dims: [from_region, from_commodity, region, commodity, process] - where: top_ire + in: [top_ire] demand: { dims: [commodity, region], description: what a region needs of a commodity } supply_cost: { dims: [commodity, region], description: the cost of a unit produced in a region } trade_cost: description: the cost of a unit exported along a link dims: [from_region, from_commodity, region, commodity, process] - where: top_ire + in: [top_ire] variables: supply: { dims: [commodity, region], bounds: { lower: 0 }, description: what a region produces of a commodity } export: description: what leaves along a link through a process (VAR_IRE) dims: [from_region, from_commodity, region, commodity, process] - where: top_ire + in: [top_ire] bounds: { lower: 0 } constraints: diff --git a/schema/mathspec.schema.json b/schema/mathspec.schema.json index 78668ec0..f38bf59d 100644 --- a/schema/mathspec.schema.json +++ b/schema/mathspec.schema.json @@ -718,6 +718,14 @@ "title": "Dtype", "type": "string" }, + "in": { + "default": [], + "items": { + "type": "string" + }, + "title": "In", + "type": "array" + }, "missing": { "anyOf": [ { @@ -896,7 +904,7 @@ }, "RelationSpec": { "additionalProperties": false, - "description": "A named relation: the combinations of its dimensions that exist, one row each.\n\n``dims:`` is a list of dimensions, or a mapping of column name to\ndimension where two columns share one. A row is its own identity, so the\nrelation is a subset of the product of its dimensions::\n\n relations:\n connection: {dims: [unit, port], missing: absent}\n corridor: {dims: {from: bus, to: bus}}\n\nA relation is not a dimension. Its name stands in a condition, where it\nis true at the rows it lists, and before a\nbracket, where it reads one of its mappings (``mappings:``, [`MappingSpec`][]). The\nrows are data, and arrive under the relation's name, one column per\ncolumn name, with a column per mapping beside them.", + "description": "A named relation: the combinations of its dimensions that exist, one row each.\n\n``dims:`` is a list of dimensions, or a mapping of column name to\ndimension where two columns share one. A row is its own identity, so the\nrelation is a subset of the product of its dimensions::\n\n relations:\n connection: {dims: [unit, port], missing: absent}\n corridor: {dims: {from: bus, to: bus}}\n\nA relation is not a dimension. Its name stands under an entry's ``in:``,\nin a condition, where it is true at the rows it lists, and before a\nbracket, where it reads one of its mappings (``mappings:``, [`MappingSpec`][]). The\nrows are data, and arrive under the relation's name, one column per\ncolumn name, with a column per mapping beside them.", "properties": { "description": { "anyOf": [ @@ -1024,6 +1032,14 @@ "title": "Domain", "type": "string" }, + "in": { + "default": [], + "items": { + "type": "string" + }, + "title": "In", + "type": "array" + }, "missing": { "anyOf": [ { diff --git a/src/mathspec/_advising.py b/src/mathspec/_advising.py index 546c42bf..faca5357 100644 --- a/src/mathspec/_advising.py +++ b/src/mathspec/_advising.py @@ -35,7 +35,8 @@ def advice(spec: str | Path | Mapping[str, object] | Spec | Program) -> tuple[Ad Returns: The never-an-axis advice sorted by dimension name, then one note per - entry the program reads and does not build, then the + entry the program reads and does not build, then one per variable + whose ``where:`` names a relation ``in:`` could name, then the unboundedness advice; ``str()`` of each is its sentence. Raises: diff --git a/src/mathspec/_condition_resolver.py b/src/mathspec/_condition_resolver.py index d5e25342..017ee7ea 100644 --- a/src/mathspec/_condition_resolver.py +++ b/src/mathspec/_condition_resolver.py @@ -157,6 +157,7 @@ def _bare_name(self, node: NameNode) -> Condition | UnresolvedConditionNode: and declared.missing == 'refused' and declared.where is None and declared.dtype in ('int', 'str') + and not declared.in_ ): rewrite = ( f'Declare `missing: absent`, or compare the label: where: "{node.name} == \'...\'".' diff --git a/src/mathspec/_expression_resolver.py b/src/mathspec/_expression_resolver.py index 5ba84f8e..47b9cd0f 100644 --- a/src/mathspec/_expression_resolver.py +++ b/src/mathspec/_expression_resolver.py @@ -249,7 +249,8 @@ def _call(self, node: FunctionCallNode) -> Expression | None: along = self._dim_ref(value, node.name, key) unread |= along is None else: - over = self._over(value, node.name, relation='by' in node.kwargs) + by = node.kwargs.get('by') + over = self._over(value, node.name, by=by.relation if isinstance(by, ColumnsNode) else None) unread |= over is None case 'value': amounts[key] = self._amount(value, node.name, key) @@ -487,7 +488,7 @@ def _dual(self, node: FunctionCallNode) -> Dual | None: declared = {**self.ns.spec.constraints, **self.ns.spec.given.constraints}[value.name] return Dual(value.name, dims=sorted_dims(declared.dims)) - def _over(self, value: ArithmeticNode, operator: str, *, relation: bool) -> tuple[str, ...] | None: + def _over(self, value: ArithmeticNode, operator: str, *, by: str | None) -> tuple[str, ...] | None: """``over=``: a dimension or a list of them, and beside ``by=`` the columns of the relation too, each a bare name. Beside ``by=`` a name is read against the relation in [`_grouping`][], @@ -510,8 +511,9 @@ def _over(self, value: ArithmeticNode, operator: str, *, relation: bool) -> tupl return None if any(n in self.formals for n in names): return None + names = self._relations_expanded(names, by) found = len(self.errors) - if not relation: + if by is None: for n in names: if n not in self.ns.dimensions: self.errors.append(_undeclared_dim(self.context, operator, f'over={n}', n, self.ns, self.formals)) @@ -522,6 +524,23 @@ def _over(self, value: ArithmeticNode, operator: str, *, relation: bool) -> tupl ) return names if len(self.errors) == found else None + def _relations_expanded(self, names: tuple[str, ...], by: str | None) -> tuple[str, ...]: + """*names* with each relation that is no dimension replaced by its key: its columns, and none of its mappings. + + A sum over a relation is a sum over its key, so ``over=connection`` + and ``over=[unit, port]`` lower to one node, and nothing after the + load sees the relation's name in ``over=``. Beside ``by=`` the same + relation, the key stays as its columns, which the call reads by name; + anywhere else each column becomes the dimension it is over. + """ + ns = self.ns + + def key(name: str) -> tuple[str, ...]: + shape = ns.relations[name] + return shape.key if name == by else tuple(shape.dim(r) for r in shape.key) + + return tuple(d for n in names for d in (key(n) if n in ns.relations and n not in ns.dimensions else (n,))) + def columns_ref(self, value: ArithmeticNode, operator: str, key: str) -> ColumnsNode | None: """A kwarg naming columns of one relation, ``relation[column, …]``, each a column the relation declares. diff --git a/src/mathspec/_lowering.py b/src/mathspec/_lowering.py index 1f078f4f..997709ac 100644 --- a/src/mathspec/_lowering.py +++ b/src/mathspec/_lowering.py @@ -15,6 +15,7 @@ from __future__ import annotations +import functools from typing import TYPE_CHECKING from mathspec._dimensions import check_spec @@ -41,6 +42,7 @@ from mathspec._validation import emitted_name_errors, reference_errors from mathspec.errors import SchemaError, did_you_mean, prefixed from mathspec.program import ( + And, Assumption, BooleanLiteral, Constant, @@ -57,6 +59,7 @@ Parameter, ParameterReference, Program, + RelationDefined, Sos, Variable, VariableDefined, @@ -68,7 +71,7 @@ if TYPE_CHECKING: from collections.abc import Iterable, Mapping - from mathspec.program import Expression, Gate, JoinColumns + from mathspec.program import Condition, Expression, Gate, JoinColumns from mathspec.spec import AssumptionSpec, Spec @@ -150,8 +153,9 @@ def lower(spec: Spec) -> Program: variables = {} for vname, vdef in spec.variables.items(): - where = resolve_condition_text(vdef.where, ns, f"Variable '{vname}'", errors, self_variable=vname) - if where is not None and (refusal := self_existence(where, vname, f"Variable '{vname}'")) is not None: + context = f"Variable '{vname}'" + where = resolve_condition_text(vdef.where, ns, context, errors, self_variable=vname) + if where is not None and (refusal := self_existence(where, vname, context)) is not None: errors.append(refusal) if vdef.domain == 'binary': lower_bound, upper_bound = Constant(0.0, dims=()), Constant(1.0, dims=()) @@ -159,12 +163,13 @@ def lower(spec: Spec) -> Program: lower_bound, upper_bound = _bound(vdef.bounds.lower, ns), _bound(vdef.bounds.upper, ns) variables[vname] = Variable( vdef.dims, - where=mask_of(where), + where=_index_mask(vname, vdef.in_, vdef.where, where, ns, context, errors), lower=lower_bound, upper=upper_bound, domain=vdef.domain, missing=vdef.missing, description=vdef.description, + in_=vdef.in_, ) constraints: dict[str, Constraint] = {} @@ -224,7 +229,14 @@ def lower(spec: Spec) -> Program: program = Program( parameters={ - name: Parameter(pdef.dims, pdef.dtype, pdef.missing, pdef.description, parameter_masks.get(name)) + name: Parameter( + pdef.dims, + pdef.dtype, + pdef.missing, + pdef.description, + parameter_masks.get(name), + pdef.in_, + ) for name, pdef in spec.parameters.items() }, variables=variables, @@ -391,34 +403,112 @@ def _bound(value: float | str | None, ns: Namespace) -> Constant | ParameterRefe def _parameter_masks(spec: Spec, ns: Namespace, errors: list[str]) -> dict[str, Mask | None]: - """Each parameter's ``where:`` typed, a refusal appended for each mask that reads what it may not. + """Each parameter's ``in:`` and ``where:`` as one mask, a refusal appended for each ``where:`` that reads what it may not. A tool evaluates a parameter's mask from the data alone, before any variable has a value and before any mask has been applied. So a mask reads - dimensions, relations and unmasked parameters, and no masks can wait on - one another. + dimensions, relations and parameters with neither ``in:`` nor ``where:``, + and no masks can wait on one another. """ - masked = {name for name, pdef in spec.parameters.items() if pdef.where is not None} + masked = {name for name, pdef in spec.parameters.items() if pdef.where is not None or pdef.in_} masks: dict[str, Mask | None] = {} for pname in sorted(masked): - context = f"Parameter '{pname}'" - mask = masks[pname] = mask_of(resolve_condition_text(spec.parameters[pname].where, ns, context, errors)) - if mask is None: + pdef, context = spec.parameters[pname], f"Parameter '{pname}'" + where = resolve_condition_text(pdef.where, ns, context, errors) + masks[pname] = _index_mask(pname, pdef.in_, pdef.where, where, ns, context, errors) + if (mask := mask_of(where)) is None: continue errors.extend( f"{context}: its `where:` contains variable '{v}', and a tool evaluates the `where:` of a parameter " f'from the data alone, before any variable has a value. Test a parameter or a mapping instead.' for v in sorted(mask.names_read & {*spec.variables, *spec.given.variables}) ) - errors.extend( + errors.extend(_masked_read(context, pname, read, spec) for read in sorted(mask.names_read & masked)) + return masks + + +def _masked_read(context: str, pname: str, read: str, spec: Spec) -> str: + """The refusal for a parameter's ``where:`` that reads *read*, a parameter whose own mask decides its rows.""" + if read == pname: + return ( f"{context}: its `where:` contains '{pname}' itself, and a tool evaluates that `where:` before it " f"reads '{pname}'. Test a relation or another parameter." - if read == pname - else f"{context}: its `where:` contains '{read}', which has a `where:` of its own. Test the " - f"condition in the `where:` of '{read}', or a relation." - for read in sorted(mask.names_read & masked) ) - return masks + if spec.parameters[read].where is None: + return ( + f"{context}: its `where:` contains '{read}', which has an `in:` of its own. Test a relation that the " + f"`in:` of '{read}' names instead." + ) + return ( + f"{context}: its `where:` contains '{read}', which has a `where:` of its own. Test the condition in the " + f"`where:` of '{read}', or a relation." + ) + + +def _index_mask( + name: str, + in_: tuple[str, ...], + text: str | None, + where: Condition | None, + ns: Namespace, + context: str, + errors: list[str], +) -> Mask | None: + """The key of each relation in *in_* and the resolved *where* as one mask, or ``None`` where neither limits anything. + + A refusal is appended where the relations already decide some of the + ``where:``: inside them each bare relation name is true, so reading one + there says nothing. A mapping read as ``subject[mapping]`` is a test, + and stays. + """ + if where is not None and in_ and (written := Mask(where)).within(in_) != written: + errors.append(_decided_by_index_set(context, name, text, in_, written.within(in_).root)) + elif where is not None and (problem := _rows_tested(context, text, Mask(where), in_, ns)) is not None: + errors.append(problem) + rows: list[Condition] = [ + RelationDefined(r, tuple(sorted(ns.relations[r].dim(k) for k in ns.relations[r].key))) for r in in_ + ] + conjuncts = [*rows, *(() if where is None else (where,))] + return mask_of(functools.reduce(And, conjuncts)) if conjuncts else None + + +def _rows_tested(context: str, text: str | None, written: Mask, in_: tuple[str, ...], ns: Namespace) -> str | None: + """The refusal for a ``where:`` that tests the rows of a relation, which ``in:`` states, or ``None``. + + ``in:`` and a bare relation name in ``where:`` would say one thing twice, so + only ``in:`` is taken. A mapping read as ``subject[mapping]`` is a test of a label, + and stays. + """ + tested = sorted({a.name for a in written.atoms if isinstance(a, RelationDefined) and a.column is None}) + if not tested: + return None + named = ' and '.join(f"'{r}'" for r in tested) + head = f'{context}: `where: {text}` tests the rows of {named}' + if everywhere := [r for r in tested if ns.relations[r].missing == 'refused']: + return ( + f"{head}, and under missing: refused '{everywhere[0]}' has a row everywhere. Delete it from the `where:`." + ) + top = {c.name for c in written.conjuncts if isinstance(c, RelationDefined) and c.column is None} + if not set(tested) <= top: + return f'{head} under OR or NOT, which `in:` cannot state. Declare those rows as a bool parameter, and test that in its place.' + rest = ', and keep the rest in the `where:`.' if len(written.conjuncts) > len(top) else ' in place of the `where:`.' + return f'{head}, which `in:` states. Write `in: [{", ".join(sorted({*in_, *tested}))}]`{rest}' + + +def _decided_by_index_set(context: str, name: str, text: str | None, in_: tuple[str, ...], inside: Condition) -> str: + """The refusal for a ``where:`` that reads a relation of the entry's own ``in:``, by what is left of it inside.""" + named = ' and '.join(f"'{r}'" for r in in_) + head = ( + f"{context}: in: gives '{name}' coordinates only where {named} " + f'{"has" if len(in_) == 1 else "have"} a row, so `where: {text}`' + ) + match inside: + case BooleanLiteral(value=True): + return f'{head} narrows nothing. Delete the `where:`, or the relation from it.' + case BooleanLiteral(): + return f"{head} leaves '{name}' no coordinate. Write a `where:` that holds somewhere inside the relation." + case _: + return f'{head} repeats the relation. Delete the relation from it.' def _assumption(name: str, entry: AssumptionSpec, ns: Namespace, errors: list[str]) -> Assumption | None: diff --git a/src/mathspec/_piecewise.py b/src/mathspec/_piecewise.py index 6be1a214..6dbaac87 100644 --- a/src/mathspec/_piecewise.py +++ b/src/mathspec/_piecewise.py @@ -449,7 +449,7 @@ def leaves_ungated(gate: VariableSpec | Variable, walk: JoinColumns | None = Non Under ``missing: refused`` every curve has one. """ absent = walk is not None and walk.relation.missing_at(walk.dropped) == 'absent' - return absent or (gate.where is not None and gate.missing != 0) + return absent or ((gate.where is not None or bool(gate.in_)) and gate.missing != 0) def gate_of(text: str) -> str | None: diff --git a/src/mathspec/_resolution.py b/src/mathspec/_resolution.py index c57ea1df..d47a0585 100644 --- a/src/mathspec/_resolution.py +++ b/src/mathspec/_resolution.py @@ -110,7 +110,9 @@ def __init__(self, spec: Spec) -> None: } self.relations: dict[str, Relation] = subject_tables(spec) self.defaults: dict[str, bool | float] = { - p: pd.missing for p, pd in spec.parameters.items() if not isinstance(pd.missing, str) and pd.where is None + p: pd.missing + for p, pd in spec.parameters.items() + if not isinstance(pd.missing, str) and pd.where is None and not pd.in_ } self.leaf_dims: dict[str, tuple[str, ...]] = { **{p: pd.dims for p, pd in parameters.items()}, diff --git a/src/mathspec/_typesetting/__init__.py b/src/mathspec/_typesetting/__init__.py index 4fed1203..4cc2c231 100644 --- a/src/mathspec/_typesetting/__init__.py +++ b/src/mathspec/_typesetting/__init__.py @@ -153,7 +153,10 @@ def typeset( blocks = [format_.note(format_.escape(program.description))] if program.description else [] if legend: - explained, noticed = Legend(program, walk.symbols, format_, walk.parameter_masks()), notice(program) + explained, noticed = ( + Legend(program, walk.symbols, format_, walk.parameter_masks(), walk.index_sets()), + notice(program), + ) blocks += [ format_.section(title, format_.glossary(items)) for title, items in explained.glossaries(noticed, walk.defined()) diff --git a/src/mathspec/_typesetting/legend.py b/src/mathspec/_typesetting/legend.py index 3cd3800d..ecc911b1 100644 --- a/src/mathspec/_typesetting/legend.py +++ b/src/mathspec/_typesetting/legend.py @@ -135,6 +135,7 @@ class Legend: symbols: Symbols format: Format parameter_masks: Mapping[str, str] = field(default_factory=dict) + index_sets: Mapping[str, str] = field(default_factory=dict) def _op(self, name: OperatorName) -> str: return self.format.operators[name] @@ -158,7 +159,7 @@ def glossaries(self, noticed: Noticed, defined: Iterable[str]) -> list[tuple[str parameters = [ self._item( self.symbols.name[p], - f'{fmt.mono(p)}{self._over(list(entry.dims))}{self._only_where(p)}{self._missing(entry.missing)}', + f'{fmt.mono(p)}{self._over(list(entry.dims), p)}{self._only_where(p)}{self._missing(entry.missing)}', entry.description, ) for p, entry in program.parameters.items() @@ -166,7 +167,7 @@ def glossaries(self, noticed: Noticed, defined: Iterable[str]) -> list[tuple[str variables = [ self._item( self.symbols.name[v], - f'{fmt.mono(v)}{self._over(list(entry.dims))}{self._missing(entry.missing, "absent", _MASKED_OUT)}', + f'{fmt.mono(v)}{self._over(list(entry.dims), v)}{self._missing(entry.missing, "absent", _MASKED_OUT)}', entry.description, ) for v, entry in program.variables.items() @@ -239,7 +240,10 @@ def _item(self, symbol: str, what: str, description: str | None) -> LegendItem: meaning = f'{what} {self.format.dash} {self.format.escape(description)}' if description else what return LegendItem(symbol, meaning) - def _over(self, dims: list[str]) -> str: + def _over(self, dims: list[str], entry: str | None = None) -> str: + """The domain of a legend row: *entry*'s index set where it has one, else the product of its sets.""" + if entry in self.index_sets: + return f' over {self.format.math(self.index_sets[entry])}' if not dims: return ' (scalar)' product = self.format.joined([self.symbols.set[d] for d in dims], self._op('times')) diff --git a/src/mathspec/_typesetting/walk.py b/src/mathspec/_typesetting/walk.py index 4a449b82..49064ce9 100644 --- a/src/mathspec/_typesetting/walk.py +++ b/src/mathspec/_typesetting/walk.py @@ -66,7 +66,7 @@ from mathspec._expression_parser import BinaryOperator from mathspec._typesetting.format import Format from mathspec._typesetting.symbols import Symbols - from mathspec.program import Link, Piecewise, Program + from mathspec.program import Link, Parameter, Piecewise, Program, Variable _PRECEDENCE: dict[BinaryOperator, int] = {'+': 1, '-': 1, '*': 2, '/': 2, '**': 3} _ATOM = 5 @@ -219,6 +219,14 @@ def _unsigned(node: Expression) -> Expression | None: return None +def _inside_on(entry: Parameter | Variable) -> Mask | None: + """*entry*'s mask with its ``in:`` read as true, since the domain states it: ``None`` where nothing is left.""" + if entry.where is None: + return None + inside = entry.where.within(entry.in_) + return None if isinstance(inside.root, BooleanLiteral) else inside + + class Walk: """Walks a program, emitting [`Line`][]s in one format. @@ -647,10 +655,10 @@ def _mask_condition(self, ctx: _Context, mask: Mask | None) -> str: """ return '' if mask is None else self._condition(mask.root, ctx) - def _quantifier(self, dims: list[str], condition: str) -> str: + def _quantifier(self, dims: list[str], condition: str, *, memberships: list[str] | None = None) -> str: if not dims and not condition: return '' - over = self.format.joined([self._membership(d) for d in dims], '') + over = self.format.joined(memberships or [self._membership(d) for d in dims], '') if not condition: return f'{self._op("forall")} {over}' if not over: @@ -719,13 +727,42 @@ def _definitions(self) -> list[Line]: return [self.definition(name) for name in self.defined()] def parameter_masks(self) -> dict[str, str]: - """Each masked parameter's mask, printed over the parameter's own indices for its legend row.""" + """Each masked parameter's ``where:``, inside its ``in:``, printed over the parameter's own indices for its legend row.""" return { - name: self._condition(entry.where.root, self._context(entry.dims)) + name: self._condition(mask.root, self._context(entry.dims)) for name, entry in self.program.parameters.items() - if entry.where is not None + if (mask := _inside_on(entry)) is not None } + def index_sets(self) -> dict[str, str]: + """Each parameter's and variable's ``in:``, printed as the domain its legend row is over.""" + entries = {**self.program.parameters, **self.program.variables} + return { + name: self.format.joined(self._index_set(entry.dims, entry.in_, self._context(entry.dims)), '') + for name, entry in entries.items() + if entry.in_ + } + + def _index_set(self, dims: tuple[str, ...], in_: tuple[str, ...], ctx: _Context) -> list[str]: + """The rows of each relation in *in_*, then each other dimension of *dims*, as its set. + + A relation with mappings prints by its own columns too, ``(p, u) ∈ connection``: + its mappings play no part in the index set. + """ + relations = self.program.relations + keyed = {relations[r].dim(k) for r in in_ for k in relations[r].key} + return [ + *(self._key_in(r, ctx) for r in in_), + *(self._membership(d) for d in dims if d not in keyed), + ] + + def _key_in(self, name: str, ctx: _Context) -> str: + """The key of relation *name* at the frame's own indices, as a member of the relation: ``(b, g) ∈ connection``.""" + at = self._frame_key(name, ctx) + key = [at[k] for k in self.program.relations[name].key] + member = key[0] if len(key) == 1 else self.format.parenthesise(self.format.joined(key, '')) + return f'{member} {self._op("in")} {self.format.upright(name)}' + def defined(self) -> list[str]: """The named expressions that print under their own symbol: every one, or only the unsubstitutable when inlining. @@ -830,7 +867,11 @@ def _variable(self, name: str) -> Line: entry = self.program.variables[name] ctx = self._context(frame=entry.dims) symbol = ctx.indexed(self.symbols.name[name], list(entry.dims)) - condition = self._quantifier(list(entry.dims), self._mask_condition(ctx, entry.where)) + condition = self._quantifier( + list(entry.dims), + self._mask_condition(ctx, _inside_on(entry)), + memberships=self._index_set(entry.dims, entry.in_, ctx), + ) lower, upper = entry.lower, entry.upper if entry.domain == 'binary': diff --git a/src/mathspec/_validation.py b/src/mathspec/_validation.py index dd259143..c339cc2a 100644 --- a/src/mathspec/_validation.py +++ b/src/mathspec/_validation.py @@ -34,7 +34,7 @@ from mathspec.spec import NUMERIC_DTYPES, Spec, labels_of if TYPE_CHECKING: - from collections.abc import Iterable, Iterator + from collections.abc import Iterable, Iterator, Sequence from pathlib import Path from mathspec.program import Program @@ -111,6 +111,7 @@ def reference_errors(spec: Spec) -> list[str]: *_frame_dimensions(spec), *_relation_targets(spec), *_mapping_targets(spec), + *_index_sets(spec), *_aliases(spec), *_bound_names(spec), *_sos_shapes(spec), @@ -128,7 +129,7 @@ def undeclared_dimension(kind: str, name: str, dimension: str, spec: Spec, *, al if along: rewrite = f'Name one of its dimensions, {dims}.' elif kind in ('Parameter', 'Variable'): - rewrite = f'Name its dimensions, {dims}, under dims:, and confine the entry with where: {dimension}.' + rewrite = f'Name its dimensions, {dims}, under dims:, and confine the entry with in: [{dimension}].' else: rewrite = f'Name its dimensions, {dims}.' return f"{kind} '{name}' names the relation '{dimension}', and a relation is not a dimension. {rewrite}" @@ -280,6 +281,51 @@ def _mapping_targets(spec: Spec) -> Iterator[str]: ) +def _index_sets(spec: Spec) -> Iterator[str]: + """Each entry of an ``in:`` is a relation over dimensions of the entry, leaves a combination out, and is named once.""" + entries = [ + *((f"Parameter '{name}'", name, p.dims, p.in_) for name, p in spec.parameters.items()), + *((f"Variable '{name}'", name, v.dims, v.in_) for name, v in spec.variables.items()), + ] + for context, name, dims, in_ in entries: + head = f'{context}: in: names' + yield from (f"{head} '{r}' twice. Delete one." for r, count in Counter(in_).items() if count > 1) + for rname in dict.fromkeys(in_): + if (lk := spec.relations.get(rname)) is None: + yield _not_a_relation(head, rname, spec, dims) + continue + if lk.missing == 'refused': + yield ( + f"{head} '{rname}', which under missing: refused has a row at every combination, so it " + f"leaves nothing out. Write missing: absent on '{rname}', or delete it from in:." + ) + over = Counter(d for _, d in lk.pairs) + if twice := [d for d, count in over.items() if count > 1]: + yield ( + f"{head} '{rname}', which has two columns over '{twice[0]}', and '{name}' has " + f"'{twice[0]}' once. Name a relation over each dimension once." + ) + elif outside := [d for d in over if d not in dims]: + yield ( + f"{head} '{rname}', over {sorted(over)}, and '{name}' is not over {outside}. Add " + f"{outside} to its dims:, or delete '{rname}' from in:." + ) + + +def _not_a_relation(head: str, written: str, spec: Spec, dims: Sequence[str]) -> str: + """The refusal for an ``in:`` entry that names no relation: a mapping, or a subject with mappings, apart.""" + column = re.fullmatch(r'(\w+)\[(.+)\]', written) + if column is not None and column[1] in spec.mappings: + return ( + f"{head} '{written}', a mapping of '{column[1]}', and an index set is a relation. To keep the rows the mapping " + f'gives a label, write where: {written}.' + ) + if written in spec.dimensions: + rewrite = f"Delete '{written}' from in:, since dims: lists it." if written in dims else 'List it under dims:.' + return f"{head} '{written}', which is a dimension, and an index set is a relation. {rewrite}" + return f"{head} '{written}', which is not a relation. {did_you_mean(written, spec.relations, label='Relations')}" + + def _aliases(spec: Spec) -> Iterator[str]: """An alias names a declared dimension that is no alias itself, and no mapping stands where its own mapping does.""" for name, d in spec.dimensions.items(): diff --git a/src/mathspec/program.py b/src/mathspec/program.py index b009aa4e..11f329ce 100644 --- a/src/mathspec/program.py +++ b/src/mathspec/program.py @@ -736,9 +736,14 @@ class Parameter: """ description: str | None = None where: Mask | None = None - """The condition that limits the coordinates of the parameter, or ``None`` when it has every coordinate of its - ``dims``. Where the condition is false, the parameter is absent and a tool drops its rows of data there. ``missing`` - applies where the condition is true. + """Where the parameter has coordinates, or ``None`` when it has every coordinate of its ``dims``. It holds each + relation in [`in_`][mathspec.program.Parameter.in_] and the file's ``where:``. Where it is false, the parameter is + absent and a tool drops its rows of data there. ``missing`` applies where it is true. + """ + in_: tuple[str, ...] = () + """The relations of the parameter's index set, written ``in:``, sorted by name. A tool attaching the data refuses a + row at a key combination one of them has no row for, before ``where`` drops anything and before ``missing`` fills + anything. A value column of the relation plays no part. """ @@ -746,6 +751,9 @@ class Parameter: class Variable: dims: tuple[str, ...] where: Mask | None = None + """Where the variable exists, or ``None`` when it exists at every coordinate of its ``dims``. It holds each relation + in [`in_`][mathspec.program.Variable.in_] and the file's ``where:``. + """ lower: Expression | None = None """A number, a parameter, or ``None`` when that side is open. An engine decides how an open side enters the model. """ @@ -754,6 +762,8 @@ class Variable: domain: VariableDomain = 'continuous' missing: VariableMissing = 'absent' description: str | None = None + in_: tuple[str, ...] = () + """The relations of the variable's index set, written ``in:``, sorted by name.""" @dataclass(frozen=True) @@ -1742,6 +1752,23 @@ def _fold(node: Condition) -> Condition: return node +def _within(node: Condition, relations: frozenset[str]) -> Condition: + """*node* with each bare read of *relations* true, and a named condition left whole where nothing in it changes.""" + match node: + case RelationDefined(name=name, column=None) if name in relations: + return BooleanLiteral(value=True) + case Not(operand=operand): + return Not(_within(operand, relations)) + case And(left=left, right=right): + return And(_within(left, relations), _within(right, relations)) + case Or(left=left, right=right): + return Or(_within(left, relations), _within(right, relations)) + case ConditionReference(body=body): + return node if (inside := _within(body, relations)) == body else inside + case _: + return node + + @dataclass(frozen=True) class Mask: """A resolved ``where`` and the questions the language answers about it. @@ -1790,6 +1817,13 @@ def dims(self) -> tuple[str, ...]: """ return tuple(sorted({dim for atom in self.atoms for dim in _atom_dims(atom)})) + def within(self, relations: Iterable[str]) -> Mask: + """The mask read inside the rows of *relations*, where each of them, read at the mask's own coordinates, is true. + + A relation read anywhere else, as through ``shift`` or ``at``, is left as it is. + """ + return Mask(_within(self.root, frozenset(relations))) + def __invert__(self) -> Mask: """The mask admitting exactly the rows this one refuses — construction folds a double negation or a literal flip.""" return Mask(Not(self.root)) diff --git a/src/mathspec/spec.py b/src/mathspec/spec.py index 354ca46b..1b010386 100644 --- a/src/mathspec/spec.py +++ b/src/mathspec/spec.py @@ -133,7 +133,8 @@ def _set_fields(self, handler: SerializerFunctionWrapHandler) -> dict[str, objec The canonical form dumps with ``exclude_defaults``, which leaves a default out whether the input set it or not. """ - return _as_set(self, cast('dict[str, object]', handler(self))) + keys = _written_keys(type(self)) + return {keys[k]: v for k, v in _as_set(self, cast('dict[str, object]', handler(self))).items()} @model_validator(mode='before') @classmethod @@ -145,7 +146,7 @@ def _reject_unknown_keys(cls, data: object) -> object: """ if not isinstance(data, dict): return data - known = set(cls.model_fields) + known = set(_written_keys(cls).values()) unknown = [k for k in data if isinstance(k, str) and k not in known] if unknown: raise ValueError( @@ -184,6 +185,11 @@ def _reject_unknown_keys(cls, data: object) -> object: _WORDS = ', '.join(get_args(RelationMissing)) +def _written_keys(cls: type[BaseModel]) -> dict[str, str]: + """Each field of *cls* by name, mapped to the key a file writes it under: its alias where the name is a keyword.""" + return {name: field.alias or name for name, field in cls.model_fields.items()} + + def _as_set(model: BaseModel, written: dict[str, object]) -> dict[str, object]: """*written* cut to the fields *model*'s input set, less a null and an empty mapping, which say nothing. @@ -254,8 +260,8 @@ class RelationSpec(_StrictSpec): connection: {dims: [unit, port], missing: absent} corridor: {dims: {from: bus, to: bus}} - A relation is not a dimension. Its name stands in a condition, where it - is true at the rows it lists, and before a + A relation is not a dimension. Its name stands under an entry's ``in:``, + in a condition, where it is true at the rows it lists, and before a bracket, where it reads one of its mappings (``mappings:``, [`MappingSpec`][]). The rows are data, and arrive under the relation's name, one column per column name, with a column per mapping beside them. @@ -418,9 +424,14 @@ class ParameterSpec(_StrictSpec): dims: Dims dtype: ParameterDtype = 'float' + in_: Dims = Field((), alias='in') + """The parameter's index set, written ``in:``: relations, by name, each over dimensions of the parameter and leaving + some combination out. The parameter has coordinates only where each of them has a row, and a tool refuses a row of + its data anywhere else. A mapping of the relation plays no part. + """ where: str | None = None - """The condition that limits the coordinates of the parameter, in the condition grammar. Where it is false, the - parameter is absent and a tool drops its rows of data there. ``missing:`` applies where it is true. + """The condition that limits the coordinates of the parameter inside its ``in:``, in the condition grammar. Where it + is false, the parameter is absent and a tool drops its rows of data there. ``missing:`` applies where it is true. """ missing: Annotated[RelationMissing | bool | int | float, WithJsonSchema(_MISSING_SCHEMA)] = 'refused' description: str | None = None @@ -514,7 +525,10 @@ class VariableSpec(_StrictSpec): _label: ClassVar[str] = 'a variable entry' dims: Dims + in_: Dims = Field((), alias='in') + """The variable's index set, written ``in:`` as on a parameter: it exists only where each relation has a row.""" where: str | None = None + """The condition that limits where the variable exists inside its ``in:``, in the condition grammar.""" bounds: BoundsSpec = BoundsSpec() domain: VariableDomain = 'continuous' missing: Annotated[Literal['absent'] | float, WithJsonSchema(_VARIABLE_MISSING_SCHEMA)] = 'absent' @@ -528,7 +542,7 @@ def _absent_or_a_number(cls, v: object, info: ValidationInfo[object]) -> object: The number stands for the variable where its mask leaves it out, so a mask must leave something out. The number lies in the variable's domain. The bounds hold only inside the mask, so the number may lie outside them. It loads as a float, as a bound does, so ``0`` and ``0.0`` are one spec. The checks run on the number as written, - so a refusal quotes it, and skip a ``where:`` or ``domain:`` that failed to load. + so a refusal quotes it, and skip an ``in:``, ``where:`` or ``domain:`` that failed to load. """ if v is None: raise _missing_null('a variable', 'Write a number, or leave the key out for absent.') @@ -539,10 +553,11 @@ def _absent_or_a_number(cls, v: object, info: ValidationInfo[object]) -> object: f'missing: {_as_yaml(v)} on a variable, which takes absent or a finite number. A variable has no ' f'data, so refused and a flag apply only to a parameter.' ) - elif 'where' in info.data and info.data['where'] is None: + elif {'in_', 'where'} <= info.data.keys() and info.data['where'] is None and not info.data['in_']: msg = ( - f'missing: {_as_yaml(v)} needs a `where:`. A variable with no `where:` exists at every ' - f'coordinate of its dimensions, so no coordinate is missing. Add a `where:`, or delete `missing:`.' + f'missing: {_as_yaml(v)} needs a `where:` or an `in:`. A variable with neither exists at every ' + f'coordinate of its dimensions, so no coordinate is missing. Add a `where:` or an `in:`, or delete ' + f'`missing:`.' ) elif info.data.get('domain') == 'binary' and v not in (0, 1): msg = f'missing: {_as_yaml(v)} on a binary variable, which takes 0 or 1.' diff --git a/tests/test_alias.py b/tests/test_alias.py index b1db7d6b..cb17dee2 100644 --- a/tests/test_alias.py +++ b/tests/test_alias.py @@ -98,6 +98,42 @@ def test_an_alias_that_says_more_or_less_than_its_dimension_is_refused(patch: di assert message in str(caught.value) +def test_ire_flo_is_over_two_regions_and_two_commodities_at_the_links_that_exist(): + """TIMES' IRE_FLO(REG,COM,R,C): an alias for each set named twice, and `in:` for the links that exist.""" + spec = to_spec( + { + 'dimensions': { + 'region': {}, + 'commodity': {}, + 'from_region': {'alias': 'region'}, + 'from_commodity': {'alias': 'commodity'}, + }, + 'relations': { + 'top_ire': {'dims': ['from_region', 'from_commodity', 'region', 'commodity'], 'missing': 'absent'} + }, + 'parameters': { + 'ire_flo': {'dims': ['from_region', 'from_commodity', 'region', 'commodity'], 'in': ['top_ire']} + }, + 'variables': { + 'export': {'dims': ['from_region', 'from_commodity', 'region', 'commodity'], 'in': ['top_ire']}, + 'supply': {'dims': ['region', 'commodity']}, + }, + 'constraints': { + 'balance': { + 'dims': ['region', 'commodity'], + 'expression': ( + 'supply + sum(ire_flo * export, over=[from_region, from_commodity]) ' + '- sum(sum(sum(export, over=[region, commodity]), over=from_region, by=from_region[region]), ' + 'over=from_commodity, by=from_commodity[commodity]) >= 0' + ), + } + }, + 'objective': {'expression': 'sum(supply)'}, + } + ) + assert spec.program.parameters['ire_flo'].in_ == ('top_ire',), 'the links that exist are the index set' + + def test_a_sum_inside_an_entry_over_the_alias_takes_an_index_of_its_own(): """A sum over `g` inside a sum over its alias printed `w_{g',g'}`: its dummy took the alias's own index. diff --git a/tests/test_dimensions.py b/tests/test_dimensions.py index 60e432cc..f4837f2f 100644 --- a/tests/test_dimensions.py +++ b/tests/test_dimensions.py @@ -969,7 +969,7 @@ def test_every_dims_of_the_program_is_sorted_by_name(path): @pytest.mark.parametrize( 'entry', [ - pytest.param({'where': 'zone_of'}, id='the-rows-of-the-relation'), + pytest.param({'in': ['zone_of']}, id='the-rows-of-the-relation'), pytest.param({'where': "zone_of[zone] == 'north'"}, id='a-column-against-a-label'), pytest.param({'where': 'zone_of[zone] == home_of[zone]'}, id='a-column-against-another-relation'), ], diff --git a/tests/test_lowering.py b/tests/test_lowering.py index eb04785a..21e2b95c 100644 --- a/tests/test_lowering.py +++ b/tests/test_lowering.py @@ -724,7 +724,7 @@ def test_a_relation_lowers_with_the_join_each_call_names(): 'mappings': {'zone_of': {'zone': {'dim': 'zone'}}}, 'parameters': {'price': {'dims': ['snapshot', 'zone']}}, 'variables': { - 'p': {'dims': ['snapshot', 'generator'], 'where': "zone_of AND zone_of[zone] == 'A'"}, + 'p': {'dims': ['snapshot', 'generator'], 'in': ['zone_of'], 'where': "zone_of[zone] == 'A'"}, 'first': { 'dims': ['snapshot', 'generator'], 'where': 'position(generator, within=zone_of[zone]) == 0', @@ -796,7 +796,7 @@ def test_a_relation_lowers_with_the_join_each_call_names(): assert [(type(a).__name__, a.dims) for a in p_where.atoms] == [ ('RelationDefined', ('generator', 'snapshot')), ('RelationComparison', ('generator', 'snapshot')), - ], 'a row and a comparison are both read at the key of a keyed relation' + ], 'the rows `in:` names and a comparison are both read at the key of a keyed relation' first_where = program.variables['first'].where assert first_where is not None assert first_where.dims == ('generator', 'snapshot'), 'a position within a group is read at every key column' @@ -805,8 +805,8 @@ def test_a_relation_lowers_with_the_join_each_call_names(): @pytest.mark.parametrize( ('variable', 'entry', 'atom'), [ - pytest.param('q', {'where': 'link'}, RelationDefined('link', ('g', 'h')), id='a-bare-relation-tests-a-row'), - pytest.param('p', {'where': 'g[h]'}, RelationDefined('g', ('g',), 'h'), id='a-mapping-alone-tests-a-label'), + pytest.param('q', {'in': ['link']}, RelationDefined('link', ('g', 'h')), id='in-tests-a-row'), + pytest.param('p', {'where': 'g[h]'}, RelationDefined('g', ('g',), 'h'), id='a-map-alone-tests-a-label'), ], ) def test_a_condition_tests_a_row_or_a_label_by_how_it_is_written(variable, entry, atom): diff --git a/tests/test_piecewise.py b/tests/test_piecewise.py index e0f37cf3..31bd1bcc 100644 --- a/tests/test_piecewise.py +++ b/tests/test_piecewise.py @@ -12,7 +12,7 @@ from __future__ import annotations import re -from typing import get_args +from typing import Any, get_args import pytest @@ -782,13 +782,21 @@ def test_a_ragged_where_reaches_the_weights_as_written_and_the_frame_rows_as_a_c assert expanded.constraints['cost_curve_p'].where == 'count(has_curve AND bp_x, over=bp) > 0' -def test_a_where_joins_both_gate_rows(): +@pytest.mark.parametrize( + 'gate', + [ + pytest.param({'where': 'committable'}, id='a-gate-with-a-where'), + pytest.param({'in': ['running']}, id='a-gate-in-a-relation'), + ], +) +def test_a_where_joins_both_gate_rows(gate: dict[str, Any]): """`activity:` splits the convexity row across the gate's own mask, and the block's where holds over both halves.""" spec = spec_of( MASKED, **{ - 'variables.u': {'dims': ['snapshot', 'generator'], 'domain': 'binary', 'where': 'committable'}, + 'variables.u': {'dims': ['snapshot', 'generator'], 'domain': 'binary', **gate}, 'parameters.committable': {'dims': ['generator'], 'dtype': 'bool'}, + 'relations.running': {'dims': ['generator', 'snapshot'], 'missing': 'absent'}, 'piecewise.cost_curve.activity': 'u', }, ) diff --git a/tests/test_validation.py b/tests/test_validation.py index 5281abf8..e30627d8 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -2023,7 +2023,7 @@ def test_the_mapping_a_bare_comparison_is_told_to_read_is_one_the_language_takes pytest.param( {'variables.x': {'dims': ['link']}}, "Variable 'x' names the relation 'link', and a relation is not a dimension. Name its dimensions, " - "['g', 'h'], under dims:, and confine the entry with where: link.", + "['g', 'h'], under dims:, and confine the entry with in: [link].", id='dims-of-a-variable', ), pytest.param( @@ -2032,12 +2032,6 @@ def test_the_mapping_a_bare_comparison_is_told_to_read_is_one_the_language_takes "['g', 'h'].", id='dims-of-a-constraint', ), - pytest.param( - {'constraints.c': {'dims': [], 'expression': 'sum(q, over=link) >= 0'}}, - "Constraint 'c': sum(over=link) names the relation 'link', and a relation is not a dimension. Name its " - "dimensions, ['g', 'h'].", - id='over', - ), pytest.param( {'constraints.c': {'dims': ['g'], 'expression': 'p <= shift(p, along=link, offset=1)'}}, "Constraint 'c': shift(along=link) names the relation 'link', and a relation is not a dimension. Name " @@ -2051,6 +2045,42 @@ def test_a_relation_named_where_a_dimension_belongs_is_refused(patch: dict[str, assert message in _refusal(**_LINKED, **patch) +@pytest.mark.parametrize( + ('relation', 'dims'), + [ + pytest.param('sum(q, over=link) >= 0', 'sum(q, over=[g, h]) >= 0', id='plain'), + pytest.param('sum(q, over=[link]) >= 0', 'sum(q, over=[g, h]) >= 0', id='in-a-list'), + pytest.param('sum(q, over=link, by=link[to]) >= 0', 'sum(q, over=[g, h], by=link[to]) >= 0', id='beside-by'), + ], +) +def test_a_sum_over_a_relation_is_a_sum_over_its_dims(relation: str, dims: str): + """`over=` a relation was refused, with its dims as the rewrite; the load can write the rewrite itself.""" + patch = {**_LINKED, 'mappings.link': {'to': {'dim': 'h'}}} + frame = ['h'] if 'by=' in relation else [] + over_relation = _spec(**patch, **{'constraints.c': {'dims': frame, 'expression': relation}}) + over_dims = _spec(**patch, **{'constraints.c': {'dims': frame, 'expression': dims}}) + assert over_relation.program.constraints['c'] == over_dims.program.constraints['c'], ( + 'a sum over a relation lowers to the sum over its dims' + ) + + +def test_a_sum_over_a_relation_with_a_map_sums_over_its_key_alone(): + """The expansion took every dimension a column of the relation is over, the mapping's too. + + So `sum(q, over=link, by=link[to])` with `to` over a third dimension `b` + named `b` in both `over=` and `by=`, and was refused. + """ + patch = {**_LINKED, 'dimensions.b': {}, 'mappings.link': {'to': {'dim': 'b'}}} + for frame, expression in [([], 'sum(q, over=link) >= 0'), (['b'], 'sum(q, over=link, by=link[to]) >= 0')]: + _spec(**patch, **{'constraints.c': {'dims': frame, 'expression': expression}}) + + +def test_a_sum_over_a_relation_and_one_of_its_dims_names_that_dim_twice(): + """The expansion is not a union: `over=[link, g]` names `g` twice, and the refusal says so.""" + message = _refusal(**_LINKED, **{'constraints.c': {'dims': [], 'expression': 'sum(q, over=[link, g]) >= 0'}}) + assert "names 'g' twice" in message, message + + #: `p` over the pairs of `g` and `h`, with two relations that list some of them. ON_A_RELATION: dict[str, Any] = { 'dimensions': {'g': {}, 'h': {}}, @@ -2060,11 +2090,59 @@ def test_a_relation_named_where_a_dimension_belongs_is_refused(patch: dict[str, } +@pytest.mark.parametrize( + ('patch', 'message'), + [ + pytest.param( + {}, + "Variable 'p': `where: pair` tests the rows of 'pair', which `in:` states. Write `in: [pair]` in place " + 'of the `where:`.', + id='a-relation', + ), + pytest.param( + {'variables.p.where': 'pair AND link'}, + "tests the rows of 'link' and 'pair', which `in:` states. Write `in: [link, pair]` in place of the " + '`where:`.', + id='two-relations', + ), + pytest.param( + {'variables.p.where': 'pair AND flag'}, + 'Write `in: [pair]`, and keep the rest in the `where:`.', + id='a-relation-beside-a-test', + ), + pytest.param( + {'variables.p.in': ['link'], 'variables.p.where': 'pair AND flag'}, + 'Write `in: [link, pair]`, and keep the rest in the `where:`.', + id='a-relation-beside-an-index-set', + ), + pytest.param( + {'variables.p': {'dims': ['g', 'h']}, 'parameters.cap': {'dims': ['g', 'h'], 'where': 'pair'}}, + "Parameter 'cap': `where: pair` tests the rows of 'pair', which `in:` states.", + id='a-parameter', + ), + pytest.param( + {'variables.p.where': 'pair OR flag'}, + "tests the rows of 'pair' under OR or NOT, which `in:` cannot state. Declare those rows as a bool " + 'parameter, and test that in its place.', + id='under-an-or', + ), + pytest.param({'variables.p.where': 'NOT pair'}, "rows of 'pair' under OR or NOT", id='under-a-not'), + pytest.param( + {'relations.pair.missing': 'refused'}, + "under missing: refused 'pair' has a row everywhere. Delete it from the `where:`.", + id='a-relation-with-a-row-everywhere', + ), + ], +) +def test_a_where_that_tests_the_rows_of_a_relation_is_refused(patch: dict[str, Any], message: str): + """`where: pair` and `in: [pair]` said one thing twice, and a parameter's `where:` dropped what `in:` refuses.""" + assert message in _refusal(ON_A_RELATION, **patch) + + @pytest.mark.parametrize( 'patch', [ - pytest.param({}, id='the-rows-of-a-relation'), - pytest.param({'variables.p.where': 'pair AND NOT link'}, id='the-rows-of-two-relations'), + pytest.param({'variables.p': {'dims': ['g', 'h'], 'in': ['pair']}}, id='the-rows-under-in'), pytest.param({'variables.p.where': 'flag'}, id='a-test-of-a-value'), pytest.param( {'mappings.g.zone': {'dim': 'h', 'missing': 'absent'}, 'variables.p.where': 'g[zone]'}, @@ -2079,11 +2157,11 @@ def test_a_relation_named_where_a_dimension_belongs_is_refused(patch: dict[str, 'variables.p': {'dims': ['g', 'h']}, 'constraints.c': {'dims': ['g', 'h'], 'where': 'pair', 'expression': 'p >= 0'}, }, - id='on-a-constraint', + id='a-constraint-has-no-in', ), ], ) -def test_a_where_that_reads_a_relation_or_a_mapping_loads(patch: dict[str, Any]): +def test_a_where_that_in_cannot_state_loads(patch: dict[str, Any]): to_spec(varied(ON_A_RELATION, **patch)) @@ -2881,6 +2959,14 @@ def test_a_values_table_under_a_ragged_where_is_not_refused(parameter, declared) pytest.param({'parameters.n': {'dims': ['g'], 'dtype': 'str', 'missing': 'absent'}}, id='a-label-under-absent'), pytest.param({'given.parameters.n': {'dims': ['g'], 'dtype': 'int'}}, id='a-given-int'), pytest.param({'parameters.n': {'dims': ['g'], 'dtype': 'int', 'where': 'flag'}}, id='a-masked-int'), + pytest.param( + { + 'parameters.n': {'dims': ['g', 'h'], 'dtype': 'int', 'in': ['link']}, + 'relations.link': {'dims': ['g', 'h'], 'missing': 'absent'}, + 'variables.p.dims': ['g', 'h'], + }, + id='an-int-on-a-relation', + ), ], ) def test_a_bare_parameter_whose_answer_needs_the_data_loads(patch): @@ -2897,6 +2983,92 @@ def test_a_bare_parameter_whose_answer_needs_the_data_loads(patch): } +#: The same parameter, over the rows of the relation rather than masked by it. +_IN = { + 'relations.pair': {'dims': ['g', 'h'], 'missing': 'absent'}, + 'parameters.w': {'dims': ['g', 'h'], 'in': ['pair'], 'missing': 1}, +} + +_LINK = {'relations.link': {'dims': ['g', 'h'], 'missing': 'absent'}} + +#: Each kind of entry that takes `in:`, as the label its refusals open with and `w` of that kind over `pair`. +_IN_KINDS = { + 'parameters': ('Parameter', _IN), + 'variables': ( + 'Variable', + {'relations.pair': _IN['relations.pair'], 'variables.w': {'dims': ['g', 'h'], 'in': ['pair']}}, + ), +} + + +def _in(kind: str, *relations: str, **entry: Any) -> dict[str, Any]: + """`w` of *kind* over the rows of *relations*, *entry* laid over it.""" + _, patch = _IN_KINDS[kind] + return {**patch, f'{kind}.w': {**patch[f'{kind}.w'], 'in': list(relations), **entry}} + + +@pytest.mark.parametrize('kind', _IN_KINDS) +@pytest.mark.parametrize( + ('patch', 'in_'), + [ + pytest.param( + {'parameters.v': {'dims': ['g', 'h'], 'missing': 'absent'}}, {'w': ('pair',)}, id='only-the-named' + ), + pytest.param( + { + 'dimensions.t': {'dtype': 'int'}, + 'parameters.v': {'dims': ['g', 'h', 't'], 'in': ['pair'], 'missing': 'absent'}, + }, + {'w': ('pair',), 'v': ('pair',)}, + id='a-parameter-over-one-more-dimension', + ), + pytest.param( + {'relations.pair': {'dims': {'a': 'g', 'b': 'h'}, 'missing': 'absent'}}, + {'w': ('pair',)}, + id='columns-named-apart-from-their-dimensions', + ), + pytest.param({**_LINK, 'w.in': ['pair', 'link']}, {'w': ('link', 'pair')}, id='two-relations-sorted-by-name'), + pytest.param( + {'mappings.pair': {'to': {'dim': 'g'}}}, + {'w': ('pair',)}, + id='a-relation-with-a-map-over-one-of-its-dimensions', + ), + ], +) +def test_an_entry_has_the_index_set_it_names(kind: str, patch: dict[str, Any], in_: dict[str, tuple[str, ...]]): + """A tool reads an entry's index set off the program, inside its `where`, and an entry that names none stays free.""" + rest = {key: value for key, value in patch.items() if key != 'w.in'} + spec = _spec(**{**_in(kind, *patch.get('w.in', ['pair'])), **rest}) + entries = {**spec.program.parameters, **spec.program.variables} + assert {name: entry.in_ for name, entry in entries.items() if entry.in_} == in_, ( + 'exactly the entries that name a relation have it as their index set, and it alone' + ) + where = entries['w'].where + assert where is not None and where.names_read == set(in_['w']), 'the index set is part of where the entry exists' + assert to_spec(spec.to_yaml()).program == spec.program, 'in: survives a round trip through YAML' + + +@pytest.mark.parametrize( + ('path', 'entry'), + [ + pytest.param('parameters.efficiency', {'missing': 1}, id='a-parameter'), + pytest.param('variables.delivered', {}, id='a-variable'), + ], +) +def test_a_relation_with_maps_is_an_index_set_by_its_own_columns(path: str, entry: dict[str, Any]): + """`connection` is over unit and port; its mappings bus and role are labels a `where:` still tests beside `in:`.""" + raw = read_yaml(EXAMPLES / 'connection.yaml') + written = {'dims': ['unit', 'port', 'snapshot'], 'in': ['connection'], 'where': "connection[role] == 'output'"} + spec = to_spec(varied(raw, **{path: {**written, **entry}})) + kind, name = path.split('.') + indexed = getattr(spec.program, kind)[name] + assert indexed.in_ == ('connection',), 'the relation with mappings is the index set' + assert indexed.where is not None and indexed.where.dims == ('port', 'unit'), 'by its own columns, not its mappings' + assert indexed.where.within(indexed.in_) == to_spec(raw).program.parameters['efficiency'].where, ( + 'inside the index set, the `where:` on the role is what the file wrote' + ) + + def test_a_parameter_mask_is_part_of_the_program(): """A tool reads where a parameter has coordinates off the program, and the file writes the mask back.""" spec = _spec(**_MASKED) @@ -2935,6 +3107,12 @@ def test_a_parameter_mask_is_part_of_the_program(): "the `where:` of 'flag', or a relation.", id='a-mask-that-reads-a-masked-parameter', ), + pytest.param( + {**_IN, 'parameters.u': {'dims': ['g', 'h'], 'where': 'w > 0.5'}}, + "Parameter 'u': its `where:` contains 'w', which has an `in:` of its own. Test a relation that the " + "`in:` of 'w' names instead.", + id='a-mask-that-reads-a-parameter-with-an-index-set', + ), pytest.param( {'parameters.c.where': 'flag', 'parameters.flag.where': 'c > 0'}, "Parameter 'c': its `where:` contains 'flag', which has a `where:` of its own", @@ -2951,6 +3129,88 @@ def test_a_parameter_mask_that_nothing_can_evaluate_is_refused(patch, fragment): assert fragment in _refusal(**patch) +@pytest.mark.parametrize('kind', _IN_KINDS) +@pytest.mark.parametrize( + ('patch', 'fragment'), + [ + pytest.param( + {'where': 'pair'}, + "{entry} 'w': in: gives 'w' coordinates only where 'pair' has a row, so `where: pair` narrows nothing. " + 'Delete the `where:`, or the relation from it.', + id='a-where-that-repeats-the-relation', + ), + pytest.param( + {'where': 'pair OR flag'}, '`where: pair OR flag` narrows nothing.', id='a-where-the-relation-satisfies' + ), + pytest.param( + {'where': 'NOT pair'}, + "`where: NOT pair` leaves 'w' no coordinate. Write a `where:` that holds somewhere inside the relation.", + id='a-where-the-relation-leaves-empty', + ), + pytest.param( + {'where': 'pair AND flag'}, + "{entry} 'w': in: gives 'w' coordinates only where 'pair' has a row, so `where: pair AND flag` repeats " + 'the relation. Delete the relation from it.', + id='a-where-that-repeats-the-relation-beside-a-test', + ), + pytest.param( + {'in': ['pairs']}, + "{entry} 'w': in: names 'pairs', which is not a relation. Did you mean 'pair'?", + id='a-name-that-is-not-a-relation', + ), + pytest.param( + {'in': ['c']}, + "{entry} 'w': in: names 'c', which is not a relation. Relations: pair.", + id='a-parameter-in-place-of-a-relation', + ), + pytest.param( + {'in': ['pair', 'pair']}, "{entry} 'w': in: names 'pair' twice. Delete one.", id='a-relation-named-twice' + ), + pytest.param( + {'in': ['next'], 'relations.next': {'dims': {'from': 'g', 'to': 'g'}, 'missing': 'absent'}}, + "{entry} 'w': in: names 'next', which has two columns over 'g', and 'w' has 'g' once. Name a " + 'relation over each dimension once.', + id='a-relation-with-two-columns-over-one-dimension', + ), + pytest.param( + {'dims': ['g']}, + "{entry} 'w': in: names 'pair', over ['g', 'h'], and 'w' is not over ['h']. Add ['h'] to its " + "dims:, or delete 'pair' from in:.", + id='a-dimension-the-entry-is-not-over', + ), + pytest.param( + {'in': ['every'], 'relations.every': {'dims': ['g', 'h']}}, + "{entry} 'w': in: names 'every', which under missing: refused has a row at every combination, so it " + "leaves nothing out. Write missing: absent on 'every', or delete it from in:.", + id='a-relation-with-a-row-everywhere', + ), + pytest.param( + {'in': ['g[h]']}, + "{entry} 'w': in: names 'g[h]', a mapping of 'g', and an index set is a relation. To keep the rows the mapping " + 'gives a label, write where: g[h].', + id='a-map-of-a-dimension-in-place-of-a-relation', + ), + pytest.param( + {'in': ['pair[x]'], 'mappings.pair': {'x': {'dim': 'h'}}}, + "{entry} 'w': in: names 'pair[x]', a mapping of 'pair', and an index set is a relation. To keep the rows " + 'the mapping gives a label, write where: pair[x].', + id='a-map-of-a-relation-in-place-of-the-relation', + ), + pytest.param( + {'in': ['g']}, + "{entry} 'w': in: names 'g', which is a dimension, and an index set is a relation. Delete 'g' from in:, since dims: lists it.", + id='a-dimension-in-place-of-a-relation', + ), + ], +) +def test_an_index_set_that_leaves_nothing_out_is_refused(kind: str, patch: dict[str, Any], fragment: str): + """Each refusal names the rewrite, alike on a parameter and on a variable.""" + entry, _ = _IN_KINDS[kind] + declared = {key: value for key, value in patch.items() if key.startswith(('relations.', 'mappings.'))} + fields = {key: value for key, value in patch.items() if key not in declared} + assert fragment.format(entry=entry) in _refusal(**_in(kind, 'pair', **fields), **declared) + + def test_a_masked_parameter_is_not_read_as_its_fill_outside_the_mask(): """Outside its mask a parameter is absent, so a comparison on it is false there, whatever `missing:` says. @@ -2970,6 +3230,14 @@ def test_a_masked_parameter_is_not_read_as_its_fill_outside_the_mask(): assert _spec(**patch).program.expressions['level'] is not None, 'unmasked, a missing row reads 0 and no case holds' masked = {**patch, **_MASKED, 'parameters.cost': {**patch['parameters.cost'], 'where': 'pair[kind]'}} assert "cases 'high' and 'low' both claim the value where cost has no row" in _refusal(**masked) + indexed = { + **patch, + 'relations.pair': _IN['relations.pair'], + 'parameters.cost': {**patch['parameters.cost'], 'in': ['pair']}, + } + assert "cases 'high' and 'low' both claim the value where cost has no row" in _refusal(**indexed), ( + 'outside its index set a parameter is absent too, so it has no fill there' + ) @pytest.mark.parametrize('fmt', FORMATS) @@ -2981,6 +3249,18 @@ def test_the_legend_says_where_a_parameter_has_coordinates(fmt: FormatName): assert row.index('only where') < row.index('where the data has no row'), 'the mask comes before the fill' +@pytest.mark.parametrize('kind', _IN_KINDS) +@pytest.mark.parametrize('fmt', FORMATS) +def test_the_legend_prints_an_index_set_as_the_domain(fmt: FormatName, kind: str): + """`in:` is where the entry has coordinates, so the legend prints the relation's rows after `over`, not as a mask.""" + spec = _spec(**_in(kind, 'pair', where='flag'), **{'variables.q.bounds': {'upper': 'c'}}) + (row,) = [line for line in typeset(spec, fmt).splitlines() if re.search(r'w[`}] over ', line)] + assert 'pair' in row[: row.index(' over ') + 60], 'the domain is the rows of the relation' + if kind == 'parameters': + assert row.index('pair') < row.index('only where') < row.index('flag'), 'the domain, then the mask inside it' + assert 'pair' not in row[row.index('only where') :], 'the relation is not printed again as a mask' + + def test_a_given_parameter_has_no_missing(): """The file that declares the parameter owns what a missing row means, as it owns a variable's bounds.""" message = _refusal(**{'given.parameters.d': {'dims': ['g'], 'missing': 'absent'}}) @@ -3091,6 +3371,11 @@ def test_a_variable_missing_outside_what_the_variable_takes_is_refused(declared, assert fragment in _refusal(**{'variables.p': {'dims': ['g'], 'where': 'c', **declared}}) +def test_a_number_stands_for_a_variable_outside_its_index_set(): + """Outside its `in:` a variable does not exist, so `missing:` has coordinates to stand for, as outside a `where:`.""" + assert _spec(**_in('variables', 'pair', missing=0)).program.variables['w'].missing == 0 + + def test_a_variable_missing_is_not_blamed_for_a_where_that_failed_to_load(): """A `where:` that fails to load is absent from what `missing:` is checked against, not null.""" message = _refusal(**{'variables.p': {'dims': ['g'], 'where': 3, 'missing': 0}}) diff --git a/tests/typesetting/golden/latex.out b/tests/typesetting/golden/latex.out index 40c0466a..12739063 100644 --- a/tests/typesetting/golden/latex.out +++ b/tests/typesetting/golden/latex.out @@ -10,13 +10,13 @@ \paragraph{Sets} \begin{description} \item[{$\mathcal{B}$}] index $b$ --- \texttt{bp} -\item[{$\mathcal{U}$}] index $u$ --- \texttt{bus} with $\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U}$, \texttt{connection} is \texttt{absent} where the data has no row, \texttt{bus[area]} is \texttt{absent} where the data gives no label +\item[{$\mathcal{U}$}] index $u$ --- \texttt{bus} with $\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{bus\_zone} \subseteq \mathcal{U} \times \mathcal{Z},\ \mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U}$, \texttt{bus\_zone} is \texttt{absent} where the data has no row, \texttt{connection} is \texttt{absent} where the data has no row, \texttt{bus[area]} is \texttt{absent} where the data gives no label \item[{$\mathcal{U}$}] index $u'$ --- \texttt{from\_bus}, an alias of \texttt{bus} \item[{$\mathcal{G}$}] index $g$ --- \texttt{generator} with $\mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{gen\_zone} \subseteq \mathcal{G} \times \mathcal{T},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U},\ \mathrm{technology}: \mathcal{G} \to \mathcal{E}$, \texttt{connection} is \texttt{absent} where the data has no row \item[{$\mathcal{S}$}] index $s$ --- \texttt{season} with $\mathrm{season}: \mathcal{T} \to \mathcal{S}$ \item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{gen\_zone} \subseteq \mathcal{G} \times \mathcal{T},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z},\ \mathrm{rep}: \mathcal{T} \to \mathcal{T},\ \mathrm{season}: \mathcal{T} \to \mathcal{S}$ \item[{$\mathcal{E}$}] index $e$ --- \texttt{technology} with $\mathrm{technology}: \mathcal{G} \to \mathcal{E}$ -\item[{$\mathcal{Z}$}] index $z$ --- \texttt{zone} (labels \texttt{north}, \texttt{south}) with $\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z}$, \texttt{bus[area]} is \texttt{absent} where the data gives no label +\item[{$\mathcal{Z}$}] index $z$ --- \texttt{zone} (labels \texttt{north}, \texttt{south}) with $\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{bus\_zone} \subseteq \mathcal{U} \times \mathcal{Z},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z}$, \texttt{bus\_zone} is \texttt{absent} where the data has no row, \texttt{bus[area]} is \texttt{absent} where the data gives no label \end{description} \paragraph{Parameters} @@ -32,7 +32,7 @@ \item[{$\mathrm{growth}$}] \texttt{growth} (scalar) \item[{$\mathrm{is\_flexible}$}] \texttt{is\_flexible} over $\mathcal{G}$, \texttt{false} where the data has no row \item[{$\mathrm{lead}$}] \texttt{lead} over $\mathcal{G}$ -\item[{$\mathrm{link\_eff}$}] \texttt{link\_eff} over $\mathcal{U} \times \mathcal{G}$, only where $\left( u,\ g \right) \in \mathrm{connection}$, $1$ where the data has no row +\item[{$\mathrm{link\_eff}$}] \texttt{link\_eff} over $\left( u,\ g \right) \in \mathrm{connection}$, $1$ where the data has no row \item[{$\mathrm{load}$}] \texttt{load} over $\mathcal{U} \times \mathcal{T}$ \item[{$\mathrm{min\_up}$}] \texttt{min\_up} over $\mathcal{G}$ \item[{$\mathrm{p}^{\mathrm{max}}$}] \texttt{p\_max} over $\mathcal{G}$, $\infty$ where the data has no row @@ -40,6 +40,7 @@ \item[{$\mathrm{susceptance}$}] \texttt{susceptance} over $\mathcal{U} \times \mathcal{U}$, $0$ where the data has no row \item[{$\mathrm{tech\_cap}$}] \texttt{tech\_cap} over $\mathcal{U} \times \mathcal{E}$, \texttt{absent} where the data has no row \item[{$\mathrm{zone\_cap}$}] \texttt{zone\_cap} over $\mathcal{Z}$ +\item[{$\mathrm{zone\_share}$}] \texttt{zone\_share} over $\left( u,\ z \right) \in \mathrm{bus\_zone}$, $1$ where the data has no row \end{description} \paragraph{Conditions} @@ -50,6 +51,7 @@ \paragraph{Variables} \begin{description} +\item[{$\mathit{flow}$}] \texttt{flow} over $\left( u,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T}$ \item[{$\mathit{fuel}$}] \texttt{fuel} over $\mathcal{G} \times \mathcal{T}$ \item[{$\mathit{headroom}$}] \texttt{headroom} (scalar), $0$ where the mask leaves it out \item[{$\mathit{heat}$}] \texttt{heat} over $\mathcal{G} \times \mathcal{T}$ @@ -95,6 +97,7 @@ \paragraph{Variable domains} \begin{align} +\text{flow} && \mathit{flow}_{u,g,t} & \ge 0 && \forall\, \left( u,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T} \,:\, \mathrm{p}^{\mathrm{max}}_{g} > 0 \\ \text{fuel} && \mathit{fuel}_{g,t} & \ge 0 && \forall\, g \in \mathcal{G},\ t \in \mathcal{T} \\ \text{headroom} && \mathit{headroom} & \ge 0 && \text{where } \mathrm{budget} \text{ is defined} \\ \text{heat} && \mathit{heat}_{g,t} & \ge 0 && \forall\, g \in \mathcal{G},\ t \in \mathcal{T} \\ diff --git a/tests/typesetting/golden/markdown.out b/tests/typesetting/golden/markdown.out index 7508d802..4b07c909 100644 --- a/tests/typesetting/golden/markdown.out +++ b/tests/typesetting/golden/markdown.out @@ -7,13 +7,13 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | Symbol | Meaning | |---|---| | $`\mathcal{B}`$ | index $`b`$ — `bp` | -| $`\mathcal{U}`$ | index $`u`$ — `bus` with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U}`$, `connection` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label | +| $`\mathcal{U}`$ | index $`u`$ — `bus` with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{bus\_zone} \subseteq \mathcal{U} \times \mathcal{Z},\ \mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U}`$, `bus_zone` is `absent` where the data has no row, `connection` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label | | $`\mathcal{U}`$ | index $`u'`$ — `from_bus`, an alias of `bus` | | $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{connection} \subseteq \mathcal{U} \times \mathcal{G},\ \mathrm{gen\_zone} \subseteq \mathcal{G} \times \mathcal{T},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z},\ \mathrm{bus}: \mathcal{G} \to \mathcal{U},\ \mathrm{technology}: \mathcal{G} \to \mathcal{E}`$, `connection` is `absent` where the data has no row | | $`\mathcal{S}`$ | index $`s`$ — `season` with $`\mathrm{season}: \mathcal{T} \to \mathcal{S}`$ | | $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{gen\_zone} \subseteq \mathcal{G} \times \mathcal{T},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z},\ \mathrm{rep}: \mathcal{T} \to \mathcal{T},\ \mathrm{season}: \mathcal{T} \to \mathcal{S}`$ | | $`\mathcal{E}`$ | index $`e`$ — `technology` with $`\mathrm{technology}: \mathcal{G} \to \mathcal{E}`$ | -| $`\mathcal{Z}`$ | index $`z`$ — `zone` (labels `north`, `south`) with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z}`$, `bus[area]` is `absent` where the data gives no label | +| $`\mathcal{Z}`$ | index $`z`$ — `zone` (labels `north`, `south`) with $`\mathrm{area}: \mathcal{U} \to \mathcal{Z},\ \mathrm{zone}: \mathcal{U} \to \mathcal{Z},\ \mathrm{bus\_zone} \subseteq \mathcal{U} \times \mathcal{Z},\ \mathrm{zone}: \mathrm{gen\_zone} \to \mathcal{Z}`$, `bus_zone` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label | #### Parameters @@ -30,7 +30,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | $`\mathrm{growth}`$ | `growth` (scalar) | | $`\mathrm{is\_flexible}`$ | `is_flexible` over $`\mathcal{G}`$, `false` where the data has no row | | $`\mathrm{lead}`$ | `lead` over $`\mathcal{G}`$ | -| $`\mathrm{link\_eff}`$ | `link_eff` over $`\mathcal{U} \times \mathcal{G}`$, only where $`\left( u,\ g \right) \in \mathrm{connection}`$, $`1`$ where the data has no row | +| $`\mathrm{link\_eff}`$ | `link_eff` over $`\left( u,\ g \right) \in \mathrm{connection}`$, $`1`$ where the data has no row | | $`\mathrm{load}`$ | `load` over $`\mathcal{U} \times \mathcal{T}`$ | | $`\mathrm{min\_up}`$ | `min_up` over $`\mathcal{G}`$ | | $`\mathrm{p}^{\mathrm{max}}`$ | `p_max` over $`\mathcal{G}`$, $`\infty`$ where the data has no row | @@ -38,6 +38,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | $`\mathrm{susceptance}`$ | `susceptance` over $`\mathcal{U} \times \mathcal{U}`$, $`0`$ where the data has no row | | $`\mathrm{tech\_cap}`$ | `tech_cap` over $`\mathcal{U} \times \mathcal{E}`$, `absent` where the data has no row | | $`\mathrm{zone\_cap}`$ | `zone_cap` over $`\mathcal{Z}`$ | +| $`\mathrm{zone\_share}`$ | `zone_share` over $`\left( u,\ z \right) \in \mathrm{bus\_zone}`$, $`1`$ where the data has no row | #### Conditions @@ -50,6 +51,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | Symbol | Meaning | |---|---| +| $`\mathit{flow}`$ | `flow` over $`\left( u,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T}`$ | | $`\mathit{fuel}`$ | `fuel` over $`\mathcal{G} \times \mathcal{T}`$ | | $`\mathit{headroom}`$ | `headroom` (scalar), $`0`$ where the mask leaves it out | | $`\mathit{heat}`$ | `heat` over $`\mathcal{G} \times \mathcal{T}`$ | @@ -104,6 +106,12 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 #### Variable domains +**`flow`** + +```math +\mathit{flow}_{u,g,t} \ge 0 \qquad \forall\, \left( u,\ g \right) \in \mathrm{connection},\ t \in \mathcal{T} \,:\, \mathrm{p}^{\mathrm{max}}_{g} > 0 +``` + **`fuel`** ```math diff --git a/tests/typesetting/golden/model.yaml b/tests/typesetting/golden/model.yaml index f7673841..4fd517c9 100644 --- a/tests/typesetting/golden/model.yaml +++ b/tests/typesetting/golden/model.yaml @@ -23,8 +23,9 @@ dimensions: bp: { dtype: int, ordered: true } # the breakpoints every curve below runs through relations: - gen_zone: { dims: [generator, snapshot] } # a relation over two dimensions, with a mapping: a call sums one away and joins on the other - connection: { dims: [generator, bus], missing: absent } # a relation with no mapping: many-to-many, read only by sum with both ends named + gen_zone: { dims: [generator, snapshot] } # a relation over two dimensions, with a map: a call sums one away and joins on the other + connection: { dims: [generator, bus], missing: absent } # a relation with no map: many-to-many, read only by sum with both ends named + bus_zone: { dims: [bus, zone], missing: absent } # the pairs that exist mappings: generator: { bus: { dim: bus }, technology: { dim: technology } } # two mappings of one subject, read to both at once @@ -44,7 +45,8 @@ parameters: is_flexible: { dims: [generator], dtype: bool, missing: false } zone_cap: { dims: [zone] } tech_cap: { dims: [bus, technology], missing: absent } - link_eff: { dims: [generator, bus], where: connection, missing: 1 } + link_eff: { dims: [generator, bus], in: [connection], missing: 1 } # over the pairs a relation lists; a `where:` may not test them + zone_share: { dims: [bus, zone], in: [bus_zone], missing: 1 } # over the pairs that exist: data refused outside them, with no `where:` of its own susceptance: { dims: [from_bus, bus], missing: 0 } # over one set twice, through its alias area_weight: { dims: [bus], where: "bus[area]" } # a mapping's label as a condition: the buses that have an area min_up: { dims: [generator], dtype: int } @@ -83,6 +85,11 @@ variables: reserve: # an empty dims: a scalar entry, whose line carries no quantifier dims: [] bounds: { lower: 0 } + flow: # a domain of the pairs `connection` lists, at every snapshot, and a `where:` inside it + dims: [snapshot, generator, bus] + in: [connection] + where: "p_max > 0" + bounds: { lower: 0 } headroom: # scalar too, but masked, so the condition stands with no set beside it; 0, so the legend names the value dims: [] where: "budget" diff --git a/tests/typesetting/golden/typst.out b/tests/typesetting/golden/typst.out index 122a3873..a67bb10d 100644 --- a/tests/typesetting/golden/typst.out +++ b/tests/typesetting/golden/typst.out @@ -5,13 +5,13 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 == Sets / $cal(B)$: index $b$ --- `bp` -/ $cal(U)$: index $u$ --- `bus` with $upright("area"): cal(U) arrow.r cal(Z), upright("zone"): cal(U) arrow.r cal(Z), upright("connection") subset.eq cal(U) times cal(G), upright("bus"): cal(G) arrow.r cal(U)$, `connection` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label +/ $cal(U)$: index $u$ --- `bus` with $upright("area"): cal(U) arrow.r cal(Z), upright("zone"): cal(U) arrow.r cal(Z), upright("bus_zone") subset.eq cal(U) times cal(Z), upright("connection") subset.eq cal(U) times cal(G), upright("bus"): cal(G) arrow.r cal(U)$, `bus_zone` is `absent` where the data has no row, `connection` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label / $cal(U)$: index $u'$ --- `from_bus`, an alias of `bus` / $cal(G)$: index $g$ --- `generator` with $upright("connection") subset.eq cal(U) times cal(G), upright("gen_zone") subset.eq cal(G) times cal(T), upright("zone"): upright("gen_zone") arrow.r cal(Z), upright("bus"): cal(G) arrow.r cal(U), upright("technology"): cal(G) arrow.r cal(E)$, `connection` is `absent` where the data has no row / $cal(S)$: index $s$ --- `season` with $upright("season"): cal(T) arrow.r cal(S)$ / $cal(T)$: index $t$ --- `snapshot` (`int` coordinates) with $upright("gen_zone") subset.eq cal(G) times cal(T), upright("zone"): upright("gen_zone") arrow.r cal(Z), upright("rep"): cal(T) arrow.r cal(T), upright("season"): cal(T) arrow.r cal(S)$ / $cal(E)$: index $e$ --- `technology` with $upright("technology"): cal(G) arrow.r cal(E)$ -/ $cal(Z)$: index $z$ --- `zone` (labels `north`, `south`) with $upright("area"): cal(U) arrow.r cal(Z), upright("zone"): cal(U) arrow.r cal(Z), upright("zone"): upright("gen_zone") arrow.r cal(Z)$, `bus[area]` is `absent` where the data gives no label +/ $cal(Z)$: index $z$ --- `zone` (labels `north`, `south`) with $upright("area"): cal(U) arrow.r cal(Z), upright("zone"): cal(U) arrow.r cal(Z), upright("bus_zone") subset.eq cal(U) times cal(Z), upright("zone"): upright("gen_zone") arrow.r cal(Z)$, `bus_zone` is `absent` where the data has no row, `bus[area]` is `absent` where the data gives no label == Parameters / $upright("area_weight")$: `area_weight` over $cal(U)$, only where $upright("area")(u) upright(" is defined")$ @@ -25,7 +25,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 / $upright("growth")$: `growth` (scalar) / $upright("is_flexible")$: `is_flexible` over $cal(G)$, `false` where the data has no row / $upright("lead")$: `lead` over $cal(G)$ -/ $upright("link_eff")$: `link_eff` over $cal(U) times cal(G)$, only where $(u, g) in upright("connection")$, $1$ where the data has no row +/ $upright("link_eff")$: `link_eff` over $(u, g) in upright("connection")$, $1$ where the data has no row / $upright("load")$: `load` over $cal(U) times cal(T)$ / $upright("min_up")$: `min_up` over $cal(G)$ / $upright("p")^(upright("max"))$: `p_max` over $cal(G)$, $infinity$ where the data has no row @@ -33,12 +33,14 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 / $upright("susceptance")$: `susceptance` over $cal(U) times cal(U)$, $0$ where the data has no row / $upright("tech_cap")$: `tech_cap` over $cal(U) times cal(E)$, `absent` where the data has no row / $upright("zone_cap")$: `zone_cap` over $cal(Z)$ +/ $upright("zone_share")$: `zone_share` over $(u, z) in upright("bus_zone")$, $1$ where the data has no row == Conditions / $upright("dispatchable")$: `dispatchable` over $cal(G)$ --- a unit that dispatches and cannot be turned down / $upright("priced_dispatch")$: `priced_dispatch` over $cal(G)$ == Variables +/ $italic("flow")$: `flow` over $(u, g) in upright("connection"), t in cal(T)$ / $italic("fuel")$: `fuel` over $cal(G) times cal(T)$ / $italic("headroom")$: `headroom` (scalar), $0$ where the mask leaves it out / $italic("heat")$: `heat` over $cal(G) times cal(T)$ @@ -78,7 +80,8 @@ $ upright("dispatchable") & upright("dispatchable")_(g) & arrow.l.r.double.long == Variable domains #set math.equation(numbering: "(1)") -$ upright("fuel") & italic("fuel")_(g,t) & >= 0 & forall g in cal(G), t in cal(T) \ +$ upright("flow") & italic("flow")_(u,g,t) & >= 0 & forall (u, g) in upright("connection"), t in cal(T) colon upright("p")^(upright("max"))_(g) > 0 \ + upright("fuel") & italic("fuel")_(g,t) & >= 0 & forall g in cal(G), t in cal(T) \ upright("headroom") & italic("headroom") & >= 0 & upright("where ") upright("budget") upright(" is defined") \ upright("heat") & italic("heat")_(g,t) & >= 0 & forall g in cal(G), t in cal(T) \ upright("on") & italic("on")_(g,t) & in {0, 1} & forall g in cal(G), t in cal(T) \ diff --git a/tools/notation.py b/tools/notation.py index 8e0740fc..8f6091eb 100644 --- a/tools/notation.py +++ b/tools/notation.py @@ -71,6 +71,7 @@ 'spare': 'Unbounded integer domain', 'reserve': 'Scalar variable', 'headroom': 'Scalar variable with a condition', + 'flow': 'Variable over the rows of a relation', 'weight': 'Variable in a special ordered set', 'fuel': 'Second axis of a curve', 'heat': 'Third axis of a curve',