Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ linopy 1.0 makes the strict arithmetic convention the only behaviour and puts de
- `Model.from_spec` builds no term over a dimension without a coordinate: such a term, and a sum of one, is the constant zero, while the data it reads is still read and checked. An assumption over such a dimension is skipped unread. A lookup through a relation without a row no longer lays out where the relation has one. ([\#1155](https://github.com/PyPSA/linopy/pull/1155))
- `Model.from_spec` builds faster: it reads a shifted axis by position, checks a relation's labels against the hash table of their dimension, decides a `where` early when a condition holds nowhere, and sets a scalar scaling factor without broadcasting it. ([\#1169](https://github.com/PyPSA/linopy/pull/1169))
- `Model.from_spec(..., rename=...)` builds each variable and constraint under the name a mapping or a function gives it, e.g. `Generator-p` for the spec's `Generator_p`. Pass the same `rename` to `linopy.spec.evaluate`. ([\#1160](https://github.com/PyPSA/linopy/pull/1160))
- `Model.from_spec(..., dim_order=...)` sets the axis order of every variable, constraint and named expression it builds: the dims it names come first, the rest keep their name order. E.g. `dim_order=["store"]` builds `(store, snapshot)`, where name order gives `(snapshot, store)`. Pass the same `dim_order` to `linopy.spec.evaluate`. ([\#1179](https://github.com/PyPSA/linopy/pull/1179))
- A missing row takes the value that `missing:` sets, e.g. `missing: 0`, in bounds, expressions and conditions alike. A parameter's `where:` limits its coordinates, and a dimension's `one_of:` its labels. Built objects follow the name order of their dimensions, as mathspec sorts every `dims:` list. ([\#1137](https://github.com/PyPSA/linopy/pull/1137))
- Limits: a constraint row that absence leaves without terms raises, where mathspec skips it ([\#993](https://github.com/PyPSA/linopy/issues/993)). `store=True` stores only relations with one key and one value column. A product of two variables in a constraint is accepted only where the data leaves it no live row.

Expand Down
19 changes: 18 additions & 1 deletion linopy/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,7 @@ def from_spec(
store: bool = False,
expressions: bool = True,
rename: Mapping[str, str] | Callable[[str], str] | None = None,
dim_order: Sequence[str] | None = None,
**model_kwargs: Any,
) -> Model:
"""
Expand Down Expand Up @@ -535,6 +536,18 @@ def from_spec(
name, and the entries a ``piecewise:`` curve expands into are
named too. Expressions and duals in the spec keep reading spec
names. Pass the same *rename* to :func:`linopy.spec.evaluate`.
dim_order : sequence of str, optional
The axis order of each variable, constraint and named expression
the spec builds, and so the order its labels are allocated in.
The dims of an entry named here come first, in this order; the
rest keep the order ``None`` gives them: by name, except that a
dim only the data side of a constraint carries, such as a
piecewise breakpoint dim, comes last. A name an entry lacks is
ignored there. For instance, ``dim_order=["store"]`` builds a
variable over ``snapshot`` and ``store`` as
``(store, snapshot)``, where ``None`` gives
``(snapshot, store)``. Pass the same *dim_order* to
:func:`linopy.spec.evaluate`.
**model_kwargs
Passed to :class:`Model`, such as ``sparse=True``.

Expand All @@ -552,6 +565,10 @@ def from_spec(
If a key of a mapping *rename* names no variable or constraint.
ValueError
If *rename* gives two variables, or two constraints, one name.
TypeError
If *dim_order* is a ``str``.
ValueError
If *dim_order* names a dim twice.

Warns
-----
Expand All @@ -562,7 +579,7 @@ def from_spec(
from linopy.spec.api import build_into

model = cls(**model_kwargs)
build_into(model, spec, sources, store, expressions, rename)
build_into(model, spec, sources, store, expressions, rename, dim_order)
return model

@property
Expand Down
69 changes: 61 additions & 8 deletions linopy/spec/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

import functools
from collections import Counter
from collections.abc import Callable, Mapping
from collections.abc import Callable, Mapping, Sequence
from pathlib import Path
from types import MappingProxyType
from typing import Any, TypeAlias
Expand Down Expand Up @@ -57,13 +57,15 @@ def build_into(
store: bool,
expressions: bool,
rename: Rename | None = None,
dim_order: Sequence[str] | None = None,
) -> None:
"""
Build *spec* with *sources* into the fresh *model*.

With *store* its data goes into ``model.parameters``, with
*expressions* its named expressions into ``model.expressions``, and
each variable and constraint under the name *rename* gives it.
each variable and constraint under the name *rename* gives it, its dims
in the order *dim_order* sets.

Raises
------
Expand All @@ -81,10 +83,13 @@ def build_into(
A constraint multiplies two variable-carrying operands.
KeyError, ValueError
What :func:`_renamer` raises for *rename*.
TypeError, ValueError
What :func:`_dim_order` raises for *dim_order*.
"""
_warn_evolving_api("spec", EVOLVING_MESSAGE, stacklevel=4)
order = _dim_order(dim_order)
program, uses = _normalized(spec)
_build(model, program, uses, sources, store, expressions, rename)
_build(model, program, uses, sources, store, expressions, rename, order)


def _build(
Expand All @@ -95,6 +100,7 @@ def _build(
store: bool,
expressions: bool,
rename: Rename | None,
dim_order: tuple[str, ...],
) -> None:
"""
The body of :func:`build_into`, in a frame of its own.
Expand All @@ -112,10 +118,37 @@ def _build(
if store:
unfilled = {name: raw(name).values for name in program.parameters}
model.parameters = xr.Dataset(unfilled | relations, coords=attached.coords)
ctx = _context(attached, uses, model, raw, solved=False, rename=named)
ctx = _context(
attached, uses, model, raw, solved=False, rename=named, dim_order=dim_order
)
build(ctx, expressions)


def _dim_order(dim_order: Sequence[str] | None) -> tuple[str, ...]:
"""
*dim_order* as the tuple :meth:`Context.frame` reads, empty for ``None``.

Raises
------
TypeError
*dim_order* is a ``str``, which would read as one dim per character.
ValueError
*dim_order* names a dim twice.
"""
if dim_order is None:
return ()
if isinstance(dim_order, str):
raise TypeError(
f"dim_order takes a sequence of dim names, not the str {dim_order!r}; "
f"pass [{dim_order!r}]"
)
order = tuple(dim_order)
twice = sorted(d for d, count in Counter(order).items() if count > 1)
if twice:
raise ValueError(f"dim_order names the dim {twice[0]!r} twice")
return order


def _renamer(program: ms.Program, rename: Rename | None) -> Callable[[str], str]:
"""
The model name of each variable and constraint of *program*: what *rename* gives, its own name where it gives none.
Expand Down Expand Up @@ -192,6 +225,7 @@ def evaluate(
*names: str,
sources: Sources | None = None,
rename: Rename | None = None,
dim_order: Sequence[str] | None = None,
) -> dict[str, xr.DataArray]:
"""
The named expressions *names* of *spec*, all in name order when none is named, folded over *model*.
Expand All @@ -201,7 +235,8 @@ def evaluate(
``Model.from_spec(..., store=True)`` writes it: a body over data alone
needs no solve.
A model built with ``Model.from_spec(..., rename=...)`` is read with the
same *rename*.
same *rename*, and each expression is laid out in the order *dim_order*
sets, as ``Model.from_spec(..., dim_order=...)`` lays out what it builds.

Raises
------
Expand All @@ -215,10 +250,15 @@ def evaluate(
SpecDataError
The data does not fit the spec, or a body reads a missing row of a
``missing: refused`` parameter.
TypeError
*dim_order* is a ``str``.
ValueError
*dim_order* names a dim twice.
"""
_warn_evolving_api("spec", EVOLVING_MESSAGE)
order = _dim_order(dim_order)
program, uses = _normalized(spec)
return _folded(program, uses, model, names, sources, rename)
return _folded(program, uses, model, names, sources, rename, order)


def _folded(
Expand All @@ -228,14 +268,23 @@ def _folded(
names: tuple[str, ...],
sources: Sources | None,
rename: Rename | None,
dim_order: tuple[str, ...],
) -> dict[str, xr.DataArray]:
"""The body of :func:`evaluate`, in a frame of its own for the reason :func:`_build` gives."""
for name in names:
if name not in program.expressions:
raise unknown("expression", name, program.expressions)
named = _renamer(program, rename)
attached = attach(program, _stored(program, model) if sources is None else sources)
ctx = _context(attached, uses, model, attached.read, solved=True, rename=named)
ctx = _context(
attached,
uses,
model,
attached.read,
solved=True,
rename=named,
dim_order=dim_order,
)
return {name: fold(name, ctx) for name in names or program.expressions}


Expand Down Expand Up @@ -273,6 +322,7 @@ def _context(
*,
solved: bool,
rename: Callable[[str], str],
dim_order: tuple[str, ...],
) -> Context:
program = attached.program
tables = (model, program, attached.coords, attached.relations)
Expand All @@ -282,7 +332,9 @@ def unmasked(name: str) -> Parameter:
parameter = raw(name)
return Parameter(fill(program, name, parameter), parameter.present)

conditions = Context(*tables, unmasked, solved=solved, rename=rename)
conditions = Context(
*tables, unmasked, solved=solved, rename=rename, dim_order=dim_order
)

def read(name: str) -> Parameter:
if program.parameters[name].where is None:
Expand All @@ -298,6 +350,7 @@ def read(name: str) -> Parameter:
solved=solved,
memo=Memo(uses=dict(uses)),
rename=rename,
dim_order=dim_order,
)


Expand Down
32 changes: 23 additions & 9 deletions linopy/spec/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
sets, constraints, the objective and the named expressions that carry a
variable term; which linopy call each construct becomes is one branch of
:func:`linopy.spec.evaluator.evaluate`. Everything built is framed in the
name order of its dims, the order mathspec sorts every ``dims:`` list into.
order :meth:`~linopy.spec.context.Context.frame` gives its dims: the dims
``dim_order`` names first, the rest in the name order mathspec sorts every
``dims:`` list into.
"""

from __future__ import annotations
Expand Down Expand Up @@ -104,7 +106,7 @@ def _variables(ctx: Context) -> None:
ctx.model.add_variables(
lower=lower,
upper=upper,
coords={d: ctx.coords[d] for d in declared.dims},
coords={d: ctx.coords[d] for d in ctx.frame(declared.dims)},
name=ctx.rename(name),
mask=mask,
binary=declared.domain == "binary",
Expand Down Expand Up @@ -162,7 +164,11 @@ def _constraints(ctx: Context) -> None:
)
term, other = carried(term, other)
ctx.model.add_constraints(
_framed(term, row.dims), sense, other, name=ctx.rename(name), mask=mask
_framed(term, row.dims, ctx),
sense,
other,
name=ctx.rename(name),
mask=mask,
)


Expand Down Expand Up @@ -306,19 +312,27 @@ def _expressions(ctx: Context) -> None:
continue
reference = ms.ExpressionReference(name, body, dims=declared.dims)
value = cast(Term, evaluate(reference, ctx))
ctx.model.add_expressions(_framed(value, declared.dims), name=name)
ctx.model.add_expressions(_framed(value, declared.dims, ctx), name=name)


def _framed(
term: Term, dims: tuple[str, ...]
term: Term, dims: tuple[str, ...], ctx: Context
) -> LinearExpression | QuadraticExpression:
"""
*term* with its dimensions in the order of the entry's dims, sorted by
name, without stray coordinates. A dimension the entry declares and the term lacks is
what the other side of a comparison brings in.
*term* with its dimensions in the order :meth:`Context.frame` gives the
entry's *dims*, without stray coordinates. A dimension the entry declares
and the term lacks is what the other side of a comparison brings in, and
lands after the term's own, as a piecewise breakpoint dim does; one
``dim_order`` names is spread onto the term to take its place in front.
"""
expression = without_strays(as_expression(term))
order = [d for d in dims if d in expression.coord_dims]
order = [
d for d in ctx.frame(dims) if d in expression.coord_dims or d in ctx.dim_order
]
lacking = {d: ctx.coords[d] for d in order if d not in expression.coord_dims}
if lacking:
shape = [len(index) for index in lacking.values()]
expression = expression + xr.DataArray(np.zeros(shape), coords=lacking)
if isinstance(expression, LinearExpression) and expression.is_sparse:
store = expression._store.transpose(order)
return LinearExpression._from_store(store, expression.model)
Expand Down
9 changes: 8 additions & 1 deletion linopy/spec/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,8 @@ class Context:
narrows the finiteness check to the rows a constraint builds, and
``subject`` names what is being built in its refusals. ``memo`` is
shared by every context replaced from one. ``rename`` gives the model
name of a variable or constraint the program declares.
name of a variable or constraint the program declares. ``dim_order``
names the dims :meth:`frame` puts first.
"""

model: Model
Expand All @@ -150,6 +151,12 @@ class Context:
subject: str = "the spec"
memo: Memo = field(default_factory=Memo, compare=False, repr=False)
rename: Callable[[str], str] = field(default=str, compare=False, repr=False)
dim_order: tuple[str, ...] = ()

def frame(self, dims: tuple[str, ...]) -> tuple[str, ...]:
"""*dims* of an entry, those ``dim_order`` names first, in its order, the rest after in name order, as mathspec states them."""
first = tuple(d for d in self.dim_order if d in dims)
return (*first, *(d for d in dims if d not in first))


def is_empty(dims: Iterable[str], ctx: Context) -> bool:
Expand Down
6 changes: 4 additions & 2 deletions linopy/spec/evaluator.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,9 @@ def fold(name: str, ctx: Context) -> xr.DataArray:
refuse_holes(f"expression '{name}'", [declared.expression], ctx)
value = evaluate(declared.expression, ctx)
if isinstance(value, xr.DataArray):
return without_strays(value).rename(name).transpose(*declared.dims, ...)
return (
without_strays(value).rename(name).transpose(*ctx.frame(declared.dims), ...)
)
return xr.DataArray(float(cast(float, value)), name=name)


Expand Down Expand Up @@ -197,7 +199,7 @@ def _dual(constraint: str, ctx: Context) -> xr.DataArray:
if renamed in built:
reported = built[renamed]._dual
elif any(c._dual is not None for c in built.values()):
dims = ctx.program.constraints[constraint].dims
dims = ctx.frame(ctx.program.constraints[constraint].dims)
shape = [len(ctx.coords[d]) for d in dims]
coords = {d: ctx.coords[d] for d in dims}
reported = xr.DataArray(np.full(shape, np.nan), coords, dims=dims)
Expand Down
Loading
Loading