diff --git a/doc/release-notes.md b/doc/release-notes.md index 67b1bdcb..79689661 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -67,6 +67,7 @@ linopy 1.0 makes the strict arithmetic convention the only behaviour and puts de - A sparse constraint keeps `rhs`, `sign`, `scaling` and `dual` on its full grid like a dense one; masked rows have label `-1`. ([\#1047](https://github.com/PyPSA/linopy/issues/1047)) - `Constraint.update` raises when a new `lhs` or `rhs` adds dimensions, and `update(variables=...)` accepts only a `Variable`. After `update(coeffs=...)` or `update(variables=...)`, `coeffs` and `vars` keep the `_term` dimension. `sanitize_zeros` removes near-zero terms, so `_term` can shrink. `set_blocks` stores the `blocks` field. ([\#1046](https://github.com/PyPSA/linopy/issues/1046), [\#1047](https://github.com/PyPSA/linopy/issues/1047)) - `Constraint.active_labels`, `to_matrix` and `to_polars` raise on a constraint without labels. +- A constraint row without a nonzero term stays in the model and reaches the solver, e.g. a sum whose terms are all absent: `x.where(mask).sum("j") >= 5` is `0 >= 5` where the mask is all `False`, so the model is infeasible where 0.x dropped the row. `ncons`, the matrices and the duals count such rows. Mask them, e.g. `add_constraints(lhs >= 5, mask=lhs.has_terms)`, or fix the data. ([\#1170](https://github.com/PyPSA/linopy/issues/1170), [\#1183](https://github.com/PyPSA/linopy/pull/1183)) *Saved models* @@ -153,7 +154,9 @@ linopy 1.0 makes the strict arithmetic convention the only behaviour and puts de *Solvers* +- A constraint row without a nonzero term, e.g. `0 * x >= 5`, reaches the solver instead of being dropped; `solve` and `to_file` warn when it cannot hold. ([\#1170](https://github.com/PyPSA/linopy/issues/1170), [\#1183](https://github.com/PyPSA/linopy/pull/1183)) - With `add_objective(..., scaling=...)`, the solve log reports objective and dual bound in model units. ([\#1077](https://github.com/PyPSA/linopy/issues/1077)) +- GLPK: an infeasible MIP, or an infeasible LP solved without presolve, is reported as `infeasible` instead of `ok`. ([\#1183](https://github.com/PyPSA/linopy/pull/1183)) - CPLEX: "optimal with unscaled infeasibilities" is reported as `imprecise` and its solution is imported. ([\#368](https://github.com/PyPSA/linopy/issues/368)) - HiGHS: a solve that stops at a limit without a solution no longer returns all-zero values, and a MIP no longer returns all-zero duals. `log_fn` and solver options also apply to a re-solve of a direct-API model. ([\#1130](https://github.com/PyPSA/linopy/issues/1130)) - The `solvers` extra excludes highspy 1.16.0. Its MPS writer drops columns that appear only in the quadratic objective, so Gurobi and COPT reject the file. ([\#1149](https://github.com/PyPSA/linopy/pull/1149)) diff --git a/doc/user-guide/design/convention.md b/doc/user-guide/design/convention.md index a9d5d317..aa28044b 100644 --- a/doc/user-guide/design/convention.md +++ b/doc/user-guide/design/convention.md @@ -300,6 +300,14 @@ Each slot becomes one constraint row. An absent slot yields no row — absence propagated into a comparison drops the constraint there, the same outcome as masking it. +A present slot always yields a row, also one without a nonzero term, e.g. a sum +over absent terms (§13) or `0 * x`. `0 * x >= 5` reaches the solver as `0 >= 5`, +so the model is infeasible; `solve` and `to_file` name such rows in a warning. +Only explicit steps drop a present row: `sanitize_missings()`, +`sanitize_infinities()` (run by `solve`; it drops `<= inf` and `>= -inf` +rows) and [`restrict()`][linopy.model.Model.restrict], which drops a row that keeps no term +in the window if it holds within `tol`. + ### §13. Reductions skip absent slots Reductions collapse a dimension rather than combining two operands, so the diff --git a/doc/user-guide/migrating-to-v1.md b/doc/user-guide/migrating-to-v1.md index 1121c595..078de1f6 100644 --- a/doc/user-guide/migrating-to-v1.md +++ b/doc/user-guide/migrating-to-v1.md @@ -62,6 +62,7 @@ Every row is a legacy guess that is now an explicit rule. | A reindexing `join=` that **creates labels** neither operand had | The join still returns the union of the coordinates; the created rows are filled per side — the linopy operand contributes the zero expression, the constant operand `fill_value=` (0 by default). A missing divisor now zeroes that row instead of leaving the term unscaled, and a missing numerator no longer comes out as `1 / divisor`. | Pass `fill_value=` on `.add` / `.sub` / `.mul` / `.div` to choose the value, e.g. `.div(other, join="outer", fill_value=1)` to keep the term unscaled, or `fill_value=linopy.ABSENT` to keep the created rows absent so the constraint drops there. | | An **unlabeled** operand (numpy array, list, polars `Series`) | Pairs with the linopy operand's dimensions by *size*; an ambiguous match (a square array, or two dimensions of equal length) or no size match raises rather than guessing. | Wrap it in a `DataArray` / `Series` / `DataFrame` with named dimensions so it aligns by label. | | A **masked / absent** variable in arithmetic | Absence propagates (the slot stays absent) instead of counting as `0`. | Decide the intent: `.fillna(0)` to keep the old "treat as zero", or leave it to propagate and drop the term. | +| A constraint row **without a nonzero term**, e.g. a `sum` over terms that are all absent | Stays in the model as `0 >= rhs` and reaches the solver, where 0.x dropped it. A row that cannot hold makes the model infeasible; `solve` and `to_file` name it in a warning. | Mask the row, e.g. `add_constraints(lhs >= rhs, mask=lhs.has_terms)`, or fix the data. | | **Conflicting auxiliary coordinates** on a shared dim — including a leftover *scalar* coord from positional indexing (`x.isel(time=0)` keeps `time` as a scalar coord, so a cyclic/boundary constraint like `x.isel(time=0) == x.isel(time=-1)` sees `time` = first vs last) | Raises instead of silently dropping one. | `.drop_vars(name)` / `.assign_coords(name=...)`; for a leftover index coord, drop it at the source with `.isel(..., drop=True)` / `.sel(..., drop=True)`. | | A first-class `pd.MultiIndex` **dimension** — and a per-*level* input onto it (e.g. per-`period` bounds onto a `(period, timestep)` `snapshot`) | Rejected; linopy uses a flat dimension with the levels as aux coords, and a per-level input must be mapped onto the flat dimension explicitly (no implicit projection). Affects PyPSA multi-investment models. Netcdf files written with such a dimension by linopy \< 1.0 are refused by [`read_netcdf()`][linopy.io.read_netcdf]; rebuild the model and save it again (see [Saved models (netcdf)](#saved-models-netcdf)). | `.reset_index(dim)` to flatten, then project the per-level input by its level aux coord. | | A multi-key `groupby(...).sum()` result | A flat `group` dimension with the keys as aux coords (not a stacked `group` MultiIndex). | Select on the key aux coords; convert an existing result with `.reset_index("group")`. | diff --git a/doc/user-guide/restrict.md b/doc/user-guide/restrict.md index 8a915dfb..abcd2e7e 100644 --- a/doc/user-guide/restrict.md +++ b/doc/user-guide/restrict.md @@ -48,8 +48,9 @@ $$ where $x_W$ are the window's variables, $x_B$ the variables outside it, and $x_B$ on the right-hand side the values you pass in `fix`. In words, each outside term adds `coefficient * fixed value` to the -right-hand side. Rows that lose every term are checked and dropped when they -are satisfied. +right-hand side. Rows that have no term in the window, also those without a +term in the model, are checked against `tol` and dropped when they are +satisfied. Without `fix`, a window that cuts a coupling raises [`RestrictionError`][linopy.restrict.RestrictionError] naming the constraints. A window diff --git a/linopy/constraints.py b/linopy/constraints.py index 87eae77b..501d0c44 100644 --- a/linopy/constraints.py +++ b/linopy/constraints.py @@ -578,7 +578,7 @@ def nterm(self) -> int: @property def ncons(self) -> int: - """Get the number of active constraints (non-masked, with at least one valid variable).""" + """Get the number of active constraints: rows with a label, also those without a term.""" return int(self.active_row_mask().sum()) @property @@ -1004,8 +1004,8 @@ def to_matrix( return csr, self._label_values()[active] def active_row_mask(self) -> np.ndarray: - """Boolean mask over raveled rows: label set and at least one variable present.""" - return (self._label_values() != -1) & self._lhs._store.has_terms() + """Boolean mask over raveled rows: label set, with or without a term.""" + return self._label_values() != -1 def active_labels(self) -> np.ndarray: """Active constraint labels in build order, without building the CSR.""" @@ -1264,9 +1264,22 @@ def to_polars(self) -> pl.DataFrame: ------- df : polars.DataFrame """ + return self._to_polars(None) + + def _to_polars(self, empty_var: int | None) -> pl.DataFrame: + """ + Long frame of :meth:`to_polars`. With ``empty_var``, a row without a + term gets the single term ``0 * empty_var``, so a writer keeps it. + """ labels_flat = self._label_values() mask = labels_flat != -1 counts, vars_, coeffs = self._lhs._store.terms(active=mask) + empty = mask & (counts == 0) + if empty_var is not None and empty.any(): + starts = (np.cumsum(counts) - counts)[empty] + vars_ = np.insert(vars_, starts, empty_var) + coeffs = np.insert(coeffs, starts, 0.0) + counts = np.where(empty, 1, counts) long = pl.DataFrame( {"labels": np.repeat(labels_flat, counts), "coeffs": coeffs, "vars": vars_} ) @@ -1463,8 +1476,8 @@ def ncons(self) -> int: """ Get the number all constraints effectively used by the model. - This excludes constraints with missing labels or where all variables - are masked (vars == -1). + This excludes rows with missing labels (-1). A row with a label but + without a term counts. """ return sum(con.ncons for con in self.data.values() if not con.is_indicator) diff --git a/linopy/io.py b/linopy/io.py index 139683a8..b566bf5b 100644 --- a/linopy/io.py +++ b/linopy/io.py @@ -645,6 +645,13 @@ def constraints_to_file( m, explicit_coordinate_names=explicit_coordinate_names ) scaled = variable_scaling is not None or constraint_scaling is not None + vlabels = m.variables.label_index.vlabels + if not vlabels.size and m.constraints.ncons: + raise ValueError( + "The model has constraint rows without a term, but no variable " + "to write them with." + ) + empty_var = int(vlabels[0]) if vlabels.size else None f.write(b"\n\ns.t.\n\n") names = list(regular) @@ -657,7 +664,8 @@ def constraints_to_file( for name in names: con = regular[name] - for df in map(Constraint.to_polars, con.iterate_slices(slice_size)): + for part in con.iterate_slices(slice_size): + df = part._to_polars(empty_var) if df.height == 0: continue if scaled: diff --git a/linopy/model.py b/linopy/model.py index 11933447..3c574c60 100644 --- a/linopy/model.py +++ b/linopy/model.py @@ -27,7 +27,7 @@ from xarray import DataArray, Dataset from xarray.core.types import T_Chunks -from linopy import solvers +from linopy import io, solvers from linopy.alignment import ( _as_index, _coords_to_dict, @@ -39,13 +39,16 @@ assigned_labels, best_int, coords_reorder_set, + format_coord, maybe_replace_sign, maybe_replace_signs, reassign_coords, replace_by_map, to_path, ) +from linopy.config import options from linopy.constants import ( + EQUAL, FACTOR_DIM, GREATER_EQUAL, LESS_EQUAL, @@ -79,7 +82,6 @@ to_block_files, to_cuopt, to_cupdlpx, - to_file, to_gurobipy, to_highspy, to_mosek, @@ -94,7 +96,7 @@ from linopy.remote import RemoteHandler from linopy.restrict.matrix import DROPPED_ROW_TOL from linopy.scaling import scalar_scaling, validate_scaling -from linopy.semantics import check_user_nan, enforce_no_multiindex +from linopy.semantics import check_user_nan, enforce_no_multiindex, warn_outside_linopy from linopy.solver_capabilities import solver_supports from linopy.solvers import ( IO_APIS, @@ -2222,6 +2224,7 @@ def solve( "`m.add_objective(...)` first (e.g. `m.add_objective(0 * x)` " "for a pure feasibility problem)." ) + self._warn_empty_rows_that_cannot_hold() solved = remote.solve_on_remote( self, solver_name=solver_name, @@ -2288,6 +2291,7 @@ def solve( self.constraints.sanitize_zeros() if sanitize_infinities: self.constraints.sanitize_infinities() + self._warn_empty_rows_that_cannot_hold() try: self.solver = None # closes any previous solver @@ -2636,6 +2640,7 @@ def _mock_solve( if sanitize_infinities: self.constraints.sanitize_infinities() + self._warn_empty_rows_that_cannot_hold() self.objective._value = 0.0 self.status = "ok" @@ -2704,7 +2709,62 @@ def reset_solution(self) -> None: to_netcdf = to_netcdf - to_file = to_file + def to_file( + self, + fn: Path | str | None, + io_api: str | None = None, + integer_label: str = "general", + slice_size: int = 2_000_000, + progress: bool | None = None, + explicit_coordinate_names: bool = False, + ) -> Path: + """ + Write out the model to an LP or MPS file, see :func:`linopy.io.to_file`. + + Warns about constraint rows without a nonzero term that cannot hold. + """ + self._warn_empty_rows_that_cannot_hold() + return io.to_file( + self, + fn, + io_api=io_api, + integer_label=integer_label, + slice_size=slice_size, + progress=progress, + explicit_coordinate_names=explicit_coordinate_names, + ) + + def _warn_empty_rows_that_cannot_hold(self) -> None: + """Warn about rows without a nonzero term whose ``0 sign rhs`` fails.""" + found = [] + for con in self.constraints.data.values(): + if con.is_indicator: + continue + labels = con._label_values() + rhs = con.rhs.values.ravel() + sign = con.sign.values.ravel() + excess = np.where(sign == LESS_EQUAL, -rhs, rhs) + excess = np.where(sign == EQUAL, np.abs(rhs), excess) + rows = (labels != -1) & (excess > DROPPED_ROW_TOL) + if rows.any(): + rows &= con._lhs._store.terms(active=rows)[0] == 0 + found.append(labels[rows]) + labels = np.concatenate([np.array([], dtype=int), *found]) + if not labels.size: + return + shown = options["display_max_rows"] + names = [ + f"{name}{format_coord(coord)}" + for name, coord in self.constraints.get_label_position(labels[:shown]) + ] + more = ", ..." if labels.size > shown else "" + warn_outside_linopy( + f"{labels.size} constraint row(s) have no nonzero term and cannot " + f"hold: {', '.join(names)}{more}. They reach the solver, which will " + "likely report the model infeasible. Mask them, or fix their " + "coefficients or right-hand side.", + UserWarning, + ) to_gurobipy = to_gurobipy diff --git a/linopy/restrict/matrix.py b/linopy/restrict/matrix.py index e8d3c68b..866c8f53 100644 --- a/linopy/restrict/matrix.py +++ b/linopy/restrict/matrix.py @@ -162,9 +162,9 @@ def _check_dropped( ) if violated.any(): raise RestrictionError( - "Cannot window the matrices: folding the dangling terms makes " - f"constraint labels {clabels[violated].tolist()} unsatisfiable " - f"(folded rhs {rhs[violated].tolist()} against sign " + "Cannot window the matrices: constraint labels " + f"{clabels[violated].tolist()} have no term in the window and are " + f"unsatisfiable (folded rhs {rhs[violated].tolist()} against sign " f"{sense[violated].tolist()}). Raise `tol` or exclude the rows " "via `drop`." ) diff --git a/linopy/solvers.py b/linopy/solvers.py index 449ca314..5d8446b2 100644 --- a/linopy/solvers.py +++ b/linopy/solvers.py @@ -1068,7 +1068,8 @@ def _build_file(self, **build_kwargs: Any) -> None: "lp files, disabling it." ) explicit_coordinate_names = False - problem_fn = model.to_file( + problem_fn = linopy.io.to_file( + model, Path(problem_fn) if not isinstance(problem_fn, Path) else problem_fn, io_api=io_api, explicit_coordinate_names=explicit_coordinate_names, @@ -1725,6 +1726,8 @@ def _run_file( "integer optimal": "optimal", "integer undefined": "infeasible_or_unbounded", "undefined": "infeasible_or_unbounded", + "infeasible (final)": "infeasible", + "integer empty": "infeasible", } sense = read_sense_from_problem_file(problem_fn) io_api = read_io_api_from_problem_file(problem_fn) diff --git a/test/test_constraint.py b/test/test_constraint.py index 8219841a..a76a2666 100644 --- a/test/test_constraint.py +++ b/test/test_constraint.py @@ -669,6 +669,15 @@ def test_constraint_to_polars(c: linopy.constraints.Constraint) -> None: assert isinstance(c.to_polars(), pl.DataFrame) +def test_constraint_to_polars_has_no_term_for_an_empty_row( + m: Model, x: linopy.Variable +) -> None: + coeffs = xr.DataArray(np.arange(10) % 2, coords=[x.indexes["first"]]) + df = m.add_constraints(coeffs * x >= 5, name="half").to_polars() + assert df["labels"].n_unique() == 5 + assert (df["coeffs"] == 1).all() + + def test_constraint_to_polars_mixed_signs(m: Model, x: linopy.Variable) -> None: """Test to_polars when a constraint has mixed sign values across dims.""" # Use Constraint so sign data can be patched diff --git a/test/test_dualization.py b/test/test_dualization.py index cac46752..b16ac540 100644 --- a/test/test_dualization.py +++ b/test/test_dualization.py @@ -478,6 +478,23 @@ def test_strong_duality_simple() -> None: assert abs(primal_obj - dual_obj) < 1e-5 +@needs_solver +@pytest.mark.filterwarnings("ignore:.*cannot hold:UserWarning") +def test_dual_of_an_empty_row_that_cannot_hold_is_unbounded() -> None: + m = Model() + i = pd.RangeIndex(2, name="i") + x = m.add_variables(lower=0, upper=10, coords=[i], name="x") + m.add_constraints(xr.DataArray([0, 1], coords=[i]) * x >= 5, name="c") + m.add_objective(x.sum()) + dual = m.dualize() + + m.solve(solver_name=_lp_solver, io_api="lp") + dual.solve(solver_name=_lp_solver, io_api="lp") + + assert m.termination_condition == "infeasible" + assert dual.termination_condition in ("unbounded", "infeasible_or_unbounded") + + @needs_solver def test_strong_duality_array_variable() -> None: """Strong duality with array variables.""" diff --git a/test/test_indicator_constraints.py b/test/test_indicator_constraints.py index 7ab33d89..f906fdc4 100644 --- a/test/test_indicator_constraints.py +++ b/test/test_indicator_constraints.py @@ -371,6 +371,18 @@ def test_gurobi_enforces_at_trigger( assert m.objective.value is not None assert np.isclose(m.objective.value, expected, atol=1e-6) + @requires_gurobi + @pytest.mark.parametrize("io_api", ["lp", "direct"]) + def test_gurobi_enforces_empty_lhs( + self, mbx: tuple[Model, Variable, Variable], io_api: str + ) -> None: + """An active trigger on ``0 * x >= 5`` cannot hold, so b stays 0.""" + m, b, x = mbx + m.add_indicator_constraints(b, 1, 0 * x, ">=", 5, name="ic0") + m.add_objective(-1 * b) + m.solve(solver_name="gurobi", io_api=io_api) + assert b.solution.item() == 0 + @requires_gurobi def test_gurobi_multiple(self) -> None: """Multiple indicators combine: x <= min(10, 5) = 5.""" diff --git a/test/test_infeasibility.py b/test/test_infeasibility.py index 5cafa60d..32849d96 100644 --- a/test/test_infeasibility.py +++ b/test/test_infeasibility.py @@ -7,6 +7,7 @@ import pandas as pd import pytest +import xarray as xr from linopy import Model, licensed_solvers @@ -282,3 +283,22 @@ def test_masked_constraint_infeasibility( for time_coord in grouped_coords["sum_lower"]: assert f"sum_lower[{time_coord}]" in output assert f"x_upper[{time_coord}]" in output + + @pytest.mark.parametrize("solver", ["gurobi", "xpress", "highs"]) + @pytest.mark.parametrize("io_api", ["lp", "direct"]) + def test_rows_after_an_empty_row_keep_their_names( + self, solver: str, io_api: str + ) -> None: + if solver not in licensed_solvers: + pytest.skip(f"{solver} not available") + + m = Model() + i = pd.RangeIndex(2, name="i") + x = m.add_variables(lower=0, upper=10, coords=[i], name="x") + rhs = xr.DataArray([-1, 1], coords=[i]) + m.add_constraints(xr.DataArray([0, 1], coords=[i]) * x >= rhs, name="c") + m.add_constraints(x.sum() >= 30, name="d") + m.add_objective(x.sum()) + m.solve(solver_name=solver, io_api=io_api, sanitize_zeros=False) + + assert m.compute_infeasibilities() == [2] diff --git a/test/test_io.py b/test/test_io.py index 6550a990..c91be985 100644 --- a/test/test_io.py +++ b/test/test_io.py @@ -1188,6 +1188,113 @@ def test_to_file_lp_constraint_section( assert _lp_constraint_section(m, fn, slice_size) == expected +@pytest.mark.parametrize("slice_size", [2_000_000, 1], ids=["default", "slice-1"]) +def test_to_file_lp_writes_empty_row_with_a_zero_term( + tmp_path: Path, backend: Model, slice_size: int +) -> None: + m = backend + i = pd.RangeIndex(3, name="i") + x = m.add_variables(coords=[i], name="x") + coeffs = xr.DataArray([0, 1, 0], coords=[i]) + m.add_constraints(coeffs * x >= -5, name="a") + m.add_constraints(LinearExpression(12, m) == 12, name="b") + m.add_objective(x.sum()) + + fn = tmp_path / "empty.lp" + assert _lp_constraint_section(m, fn, slice_size) == ( + "c0:\n+0.0 x0\n>= -5.0\n" + "c1:\n+1.0 x1\n>= -5.0\n" + "c2:\n+0.0 x0\n>= -5.0\n" + "c3:\n+0.0 x0\n= 0.0\n" + ) + + +@pytest.mark.skipif("highs" not in available_solvers, reason="highspy not installed") +@pytest.mark.parametrize( + "io_api,fn", + [("lp", "empty.lp"), ("lp-polars", "empty.lp"), ("mps", "empty.mps")], +) +def test_to_file_keeps_empty_row(tmp_path: Path, io_api: str, fn: str) -> None: + import highspy + + m = Model() + x = m.add_variables(lower=0, coords=[pd.RangeIndex(2, name="i")], name="x") + m.add_constraints(x >= 1, name="a") + m.add_constraints(LinearExpression(12, m) >= 1, name="b") + m.add_objective(x.sum()) + m.to_file(tmp_path / fn, io_api=io_api) + + h = highspy.Highs() + h.setOptionValue("output_flag", False) + h.readModel(str(tmp_path / fn)) + assert h.getNumRow() == 3 + + +def test_to_file_lp_raises_for_empty_row_without_any_variable(tmp_path: Path) -> None: + m = Model() + m.add_constraints(LinearExpression(12, m) >= 1, name="con") + with pytest.raises(ValueError, match="no variable to write them with"): + m.to_file(tmp_path / "empty.lp") + + +def empty_rows_model() -> Model: + m = Model() + x = m.add_variables(lower=0, coords=[pd.RangeIndex(20, name="i")], name="x") + m.add_constraints(LinearExpression(12, m) == 13, name="k") + m.add_constraints(0 * x >= 5, name="con") + m.add_constraints(0 * x <= 5, name="holds") + m.add_constraints(x >= 5, name="live") + m.add_objective(x.sum()) + return m + + +WRITERS: dict[str, Callable[[Model, Path], Any]] = { + "to_file-lp": lambda m, p: m.to_file(p / "m.lp"), + "to_file-mps": lambda m, p: m.to_file(p / "m.mps"), + "solve-mock": lambda m, p: m.solve(mock_solve=True), + "solve-lp": lambda m, p: m.solve("highs", io_api="lp"), + "solve-direct": lambda m, p: m.solve("highs", io_api="direct"), +} + + +@pytest.mark.skipif("highs" not in available_solvers, reason="highspy not installed") +@pytest.mark.parametrize("write", WRITERS) +def test_empty_rows_that_cannot_hold_are_named_once(tmp_path: Path, write: str) -> None: + with pytest.warns(UserWarning, match="cannot hold") as record: + WRITERS[write](empty_rows_model(), tmp_path) + + hints = [str(w.message) for w in record if "cannot hold" in str(w.message)] + assert len(hints) == 1 + assert hints[0].startswith("21 constraint row(s) have no nonzero term") + assert ": k, con[0], con[1], " in hints[0] + assert "con[12], ..." in hints[0] + assert "con[13]" not in hints[0] + assert "holds" not in hints[0] + + +@pytest.mark.skipif("highs" not in available_solvers, reason="highspy not installed") +@pytest.mark.parametrize("write", ["solve-mock", "solve-lp", "solve-direct"]) +def test_row_emptied_by_sanitize_zeros_is_named(tmp_path: Path, write: str) -> None: + m = Model() + x = m.add_variables(lower=0, coords=[pd.RangeIndex(2, name="i")], name="x") + m.add_constraints(1e-11 * x >= 5, name="tiny") + m.add_objective(x.sum()) + with pytest.warns(UserWarning, match=r"2 constraint row\(s\).*: tiny\[0\], tiny"): + WRITERS[write](m, tmp_path) + + +def test_empty_rows_that_hold_raise_no_warning(tmp_path: Path) -> None: + m = Model() + x = m.add_variables(lower=0, coords=[pd.RangeIndex(2, name="i")], name="x") + m.add_constraints(0 * x >= [-5, 0], name="a") + m.add_constraints(0 * x <= [5, 0], name="b") + m.add_constraints(LinearExpression(12, m) == 12, name="c") + m.add_objective(x.sum()) + with warnings.catch_warnings(): + warnings.simplefilter("error") + m.to_file(tmp_path / "m.lp") + + def test_to_file_lp_unsorted_constraint_labels(tmp_path: Path) -> None: m = Model() i = pd.RangeIndex(3, name="i") diff --git a/test/test_optimization.py b/test/test_optimization.py index bb4b9fcc..01d0c3d4 100644 --- a/test/test_optimization.py +++ b/test/test_optimization.py @@ -9,6 +9,7 @@ import itertools import logging +from collections.abc import Callable from typing import Any import numpy as np @@ -17,7 +18,7 @@ import xarray as xr from xarray.testing import assert_allclose, assert_equal -from linopy import GREATER_EQUAL, LESS_EQUAL, Model, solvers +from linopy import GREATER_EQUAL, LESS_EQUAL, Model, Variable, solvers from linopy.common import to_path from linopy.expressions import LinearExpression from linopy.solver_capabilities import ( @@ -683,6 +684,68 @@ def test_infeasible_model( model.compute_infeasibilities() +EMPTY_ROWS: dict[str, Callable[[Variable], Any]] = { + "zero-times-x": lambda x: 0 * x >= 5, + "zero-coeff": lambda x: xr.DataArray([0, 1], coords=[x.indexes["i"]]) * x >= 5, + "constant": lambda x: LinearExpression(12, x.model) == 13, + "all-absent-sum": lambda x: ( + x.where(xr.DataArray([False, True], coords=[x.indexes["i"]])).sum("j") >= 5 + ), +} + + +def add_empty_row(m: Model, case: str) -> Variable: + coords = [pd.RangeIndex(2, name="i"), pd.RangeIndex(2, name="j")] + x = m.add_variables(lower=0, upper=10, coords=coords, name="x") + m.add_constraints(EMPTY_ROWS[case](x), name="con") + m.add_objective(x.sum()) + return x + + +@pytest.mark.filterwarnings("ignore:.*cannot hold:UserWarning") +@pytest.mark.parametrize("case", EMPTY_ROWS) +@pytest.mark.parametrize("solver,io_api,explicit_coordinate_names", params) +def test_empty_row_that_cannot_hold_makes_the_model_infeasible( + case: str, solver: str, io_api: str, explicit_coordinate_names: bool +) -> None: + m = Model() + add_empty_row(m, case) + status, condition = m.solve( + solver, io_api=io_api, explicit_coordinate_names=explicit_coordinate_names + ) + assert status == "warning" + assert "infeasible" in condition + + +@pytest.mark.skipif("highs" not in direct_solvers, reason="highspy not installed") +@pytest.mark.filterwarnings("ignore:.*cannot hold:UserWarning") +def test_empty_row_that_cannot_hold_makes_a_sparse_model_infeasible() -> None: + m = Model(sparse=True) + add_empty_row(m, "all-absent-sum") + assert m.solve("highs", io_api="direct") == ("warning", "infeasible") + + +@pytest.mark.parametrize("solver,io_api,explicit_coordinate_names", params) +def test_empty_row_that_holds_reaches_the_solver( + solver: str, io_api: str, explicit_coordinate_names: bool +) -> None: + m = Model() + i = pd.RangeIndex(2, name="i") + x = m.add_variables(lower=0, upper=10, coords=[i], name="x") + rhs = xr.DataArray([-5, 5], coords=[i]) + m.add_constraints(xr.DataArray([0, 1], coords=[i]) * x >= rhs, name="con") + m.add_objective(x.sum()) + + status, _ = m.solve( + solver, io_api=io_api, explicit_coordinate_names=explicit_coordinate_names + ) + + assert status == "ok" + assert m.ncons == 2 + np.testing.assert_allclose(x.solution, [0, 5]) + assert m.constraints["con"].dual[0] == pytest.approx(0, abs=1e-6) + + @pytest.mark.parametrize("solver,io_api,explicit_coordinate_names", params) def test_model_with_inf( model_with_inf: Model, solver: str, io_api: str, explicit_coordinate_names: bool @@ -807,6 +870,25 @@ def test_milp_model( assert report.dual_bound is not None +@pytest.mark.parametrize( + "solver,io_api,explicit_coordinate_names", + [p for p in params if p[0] not in ["mindopt"]], +) +def test_infeasible_milp_model( + milp_model: Model, solver: str, io_api: str, explicit_coordinate_names: bool +) -> None: + if solver not in feasible_mip_solvers: + pytest.skip(f"{solver} does not support MIP") + + milp_model.add_constraints(milp_model.variables["x"] <= -1) + + status, condition = milp_model.solve( + solver, io_api=io_api, explicit_coordinate_names=explicit_coordinate_names + ) + assert status == "warning" + assert "infeasible" in condition + + @pytest.mark.parametrize( "solver,io_api,explicit_coordinate_names", [p for p in params if p[0] not in ["mindopt"]], diff --git a/test/test_persistent_apply_update.py b/test/test_persistent_apply_update.py index b481cbfb..d268ebcb 100644 --- a/test/test_persistent_apply_update.py +++ b/test/test_persistent_apply_update.py @@ -4,6 +4,7 @@ import numpy as np import pytest +import xarray as xr from linopy import Model from linopy.persistent import RebuildReason @@ -206,6 +207,43 @@ def test_cross_model_in_place(solver_name: str) -> None: assert np.isclose(cross_obj, _obj(m3)) +@pytest.mark.parametrize("solver_name", SOLVER_PARAMS) +@pytest.mark.parametrize( + "sign,rhs,update", + [(">=", -5.0, {"rhs": 5.0}), ("<=", 5.0, {"sign": ">="})], + ids=["rhs", "sign"], +) +def test_update_makes_empty_row_infeasible( + solver_name: str, sign: str, rhs: float, update: dict[str, Any] +) -> None: + m = _base_model() + x = m.variables["x"] + m.add_constraints(xr.DataArray([0, 1, 1]) * x, sign, rhs, name="c3") + m.constraints.sanitize_zeros() + s = _built(solver_name, m) + s.solve(assign=True) + + m.constraints["c3"].update(**update) + result = s.solve(m) + + assert result.status.termination_condition.value == "infeasible" + + +@pytest.mark.parametrize("solver_name", SOLVER_PARAMS) +def test_update_fills_row_without_stored_term(solver_name: str) -> None: + m = _base_model() + x = m.variables["x"] + lhs = (1 * x).where(xr.DataArray([False, True, True])).fillna(0) + m.add_constraints(lhs >= xr.DataArray([-5, 0, 0]), name="c3") + s = _built(solver_name, m) + s.solve(assign=True) + + m.constraints["c3"].update(lhs=1 * x, rhs=xr.DataArray([7.0, 0, 0])) + s.solve(m, assign=True) + + assert np.isclose(x.solution[0], 7) + + @pytest.mark.parametrize("solver_name", SOLVER_PARAMS) def test_sign_flip(solver_name: str) -> None: m = _base_model() diff --git a/test/test_restrict_matrix.py b/test/test_restrict_matrix.py index c9eb5ce6..ae2492f3 100644 --- a/test/test_restrict_matrix.py +++ b/test/test_restrict_matrix.py @@ -157,6 +157,32 @@ def test_folded_residual_passes_within_tol_or_when_dropped( assert window.matrices.clabels.size == 2 +@pytest.mark.parametrize( + ("kwargs", "raises"), + [ + ({"time": TIME[:3]}, True), + ({"time": TIME[1:3]}, False), + ({"time": TIME[:3], "drop": {"pin": TIME == 0}}, False), + ], + ids=["selected", "outside", "dropped"], +) +def test_selected_row_without_terms_is_checked_against_its_sign( + kwargs: dict[str, Any], raises: bool +) -> None: + m = Model() + x = m.add_variables(lower=0, upper=10, coords=[TIME], name="x") + m.add_constraints(xr.DataArray(TIME != 0, coords=[TIME]) * x >= 2, name="pin") + m.add_objective(x.sum()) + m.constraints.sanitize_zeros() + if raises: + with pytest.raises(RestrictionError, match=r"labels \[0\] have no term"): + m.restrict(**kwargs) + return + window = m.restrict(**kwargs).solve(solver_name="highs", output_flag=False) + + assert window.termination_condition == "optimal" + + def test_advance_keeps_the_tolerance() -> None: window = pinned_model("==", 1e-7).restrict(time=TIME[1:3], fix=ZERO_X, tol=1e-6) window.advance(time=TIME[2:4])