diff --git a/docs/testing/TEST_COVERAGE.md b/docs/testing/TEST_COVERAGE.md index d5928079d..1f5ffd44c 100644 --- a/docs/testing/TEST_COVERAGE.md +++ b/docs/testing/TEST_COVERAGE.md @@ -575,7 +575,7 @@ Same shape as §11, applied to a different axis: §11 covers expression operator **Per-operator tests** (under `tests/core/operator/window/$operator/`): - **Documents-mode frame computation**: operator's computed result under whole-partition ["unbounded", "unbounded"], cumulative ["unbounded", "current"], reverse-cumulative ["current","unbounded"], and sliding frame shapes [-1,1] — verifies the operator produces correct values *given* a set of documents in the frame. This is distinct from stage-level frame boundary tests which verify that the correct documents are *selected into* the frame. - **Field paths**: operator handles nested fields (`"a.b.c"`), missing fields, field exists with `null` value (distinct from missing), array fields, arrays at intermediate path level (`"arr.nested.value"` where `arr` is an array of objects), and numeric path components (`"arr.0.field"`) -- **Numeric precision**: type mixing (Int32/Int64/Double/Decimal128), overflow, catastrophic cancellation +- **Numeric precision**: type mixing (Int32/Int64/Double/Decimal128), overflow, catastrophic cancellation. For operators computed by an incremental/online update over the frame (`$avg`, `$stdDevPop`, `$stdDevSamp`, `$covariancePop`, `$covarianceSamp`), overflow of an *intermediate* value is distinct from overflow of the result and must be tested separately — see the multi-expression operator notes under **Category applicability** for the required cases and why a single "large values" test is insufficient. - **Special floats**: NaN/Infinity propagation in removable vs non-removable windows - **Non-numeric handling**: null, missing, string, boolean, date, object, array values in the expression field - **Argument validation**: operator rejects invalid input shapes — unknown keys in the operator spec, wrong argument type (e.g. string where object expected), missing required parameters, and extra parameters. Each operator defines its accepted shape (empty object for rank operators, expression for accumulators, named params for $shift/$expMovingAvg/etc.); anything outside that shape must error with the correct code. @@ -615,6 +615,20 @@ Same shape as §11, applied to a different axis: §11 covers expression operator - **Multi-expression operators** ($covariancePop, $covarianceSamp): take two expressions `["$x", "$y"]`. Test null/missing/type-mixing per expression position independently. $covarianceSamp with single element → null (N-1 divisor). + **Overflow requirements.** Covariance is computed by an online (Welford-style) update whose *intermediate* values — the per-row deviation `xᵢ - x̄`, the running mean, and the deviation product — can overflow independently of whether the final result is representable. Overflow tests must therefore distinguish four cases, because a single "large values" test cannot tell them apart and passing one says nothing about the others. Required for both `$covariancePop` and `$covarianceSamp`, on both the double and Decimal128 paths (`DOUBLE_NEAR_MAX` / `1e308`; `DECIMAL128_MAX` / `DECIMAL128_MIN`): + + 1. **Identical large values — no overflow in the formula.** All `xᵢ` equal at maximum magnitude (e.g. `x=[DECIMAL128_MAX]×3`, `y=[1,2,3]`). Every deviation `xᵢ - x̄` is exactly `0` regardless of magnitude, so the formula never requires `Σxᵢ`. The server nonetheless derives the mean from a running sum and reports a signed Infinity once that sum overflows, which makes its result **count-dependent**: `n=2` returns `0` while `n≥3` returns an Infinity, for input whose mathematical value is the same either way. Cover both counts and mirror on the `y` side, and record the observed value per count rather than assuming a single expectation covers the shape. + + 2. **Intermediate deviation overflow, representable result.** Opposing maximum magnitudes in the *same* column (e.g. `x=[DECIMAL128_MAX, DECIMAL128_MIN]`), so that `xᵢ - x̄` overflows while the mathematical result is within range. This is the only shape that exercises deviation overflow, and it must assert the **sign** of the result, not merely that it is non-finite — an implementation whose overflowing deviation corrupts the running mean can invert the sign of the deviation product, and `+Infinity` where `-Infinity` is expected would satisfy a non-finiteness-only assertion. Cover both a positively-correlated pair (`x=y=[MAX, MIN]`, limit `+∞`) and an anti-correlated pair (`x=[MAX, MIN]`, `y=[MIN, MAX]`, limit `-∞`). + + 3. **Product overflow with finite deviations.** Opposing magnitudes large enough that the deviation *product* overflows but each deviation stays finite (e.g. `x=y=[1e200, -1e200]`). Keeping this separate from case 2 is what distinguishes an overflow arising in the deviation step from one arising in the product step; collapsed together, a failure cannot be attributed to either. + + 4. **Result exceeds type range.** Mathematical result beyond the type's maximum. + + Cases 1 and 2 must not be collapsed into one test: case 1's expected value is finite and case 2's is infinite, and the two have opposite failure modes. Note also that a literal `Infinity` *input* does not exercise any of these — non-finite inputs are typically short-circuited into counters before the online update runs, so Infinity-input tests belong under **Special floats**, not here. + + As everywhere in this document, expected values are the reference server's observed behavior (§ *Target Spec Version*), including where that behavior is an artifact of its accumulation strategy rather than the mathematically exact answer. Verify each expectation against a live reference instance rather than deriving it from the formula. + - **Gap-filler operators** ($locf, $linearFill): core dimension is **gap definition and boundary behavior** — null/missing as gaps, first value in partition is gap (no prior anchor → null), last value is gap, consecutive gaps, partition boundary isolation. $linearFill requires at least two non-null numeric anchors to interpolate. - **Calculus operators** ($derivative, $integral): require `unit` parameter (time unit string) when sortBy is date, omit `unit` when sortBy is numeric. Test zero delta in sortBy (division by zero for $derivative), single document in frame → null. @@ -653,7 +667,7 @@ For any DocumentDB feature, ensure coverage of: - [ ] **Field lookup**: simple, nested, array, non-existent, composite, composite array - [ ] **Sign handling**: positive, negative, zero - [ ] **Type conversion**: all numeric type combinations -- [ ] **Overflow handling**: `INT32_MAX`, `INT64_MAX` boundaries +- [ ] **Overflow handling**: `INT32_MAX`, `INT64_MAX` boundaries; for operators with incremental intermediate state, also intermediate-value overflow with a **signed** expectation (§22) - [ ] **Underflow handling**: `INT32_MIN`, `INT64_MIN` boundaries - [ ] **Decimal128 precision**: high precision, boundaries (if applicable) - [ ] **Error codes**: correct error codes for invalid operations diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_argument_validation.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_argument_validation.py new file mode 100644 index 000000000..04480c29e --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_argument_validation.py @@ -0,0 +1,422 @@ +""" +Tests for $covariancePop argument validation in window context. + +Covers: valid expression forms (array of two field paths, operator expressions), +invalid argument shapes that produce null results (not an array, wrong length, +single expression, three expressions, empty array, object expression), +and structural errors (unknown keys in output field spec, multiple accumulators). + +Server behavior (verified): $covariancePop does NOT reject bad argument +shapes at parse time. Invalid forms (single expression, wrong-length arrays, +objects, empty arrays) all succeed and return null for every document. +Only structural $setWindowFields errors (unknown keys, multiple accumulators, +no accumulator, bad field paths, unrecognized operators) produce parse errors. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertFailureCode, assertSuccess +from documentdb_tests.framework.error_codes import ( + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + FAILED_TO_PARSE_ERROR, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + UNRECOGNIZED_EXPRESSION_ERROR, +) +from documentdb_tests.framework.executor import execute_command + +TWO_DOCS = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, +] + +SINGLE_DOC = [{"_id": 1, "partition": "A", "x": 1, "y": 2}] + +# Property [Valid Expression Forms]: accepted expression inputs + + +def test_covariancePop_two_field_paths(collection): + """$covariancePop accepts array of two field path expressions.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=TWO_DOCS, + expression=["$x", "$y"], + window={"documents": ["unbounded", "unbounded"]}, + ) + # x=[1,2], y=[2,4]: mean_x=1.5, mean_y=3 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 1/2 = 0.5 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 0.5}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 0.5}, + ] + assertSuccess(result, expected, msg="two field path expressions accepted") + + +def test_covariancePop_operator_expressions(collection): + """$covariancePop accepts operator expressions within the array.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs=docs, + expression=[{"$multiply": ["$x", 2]}, {"$multiply": ["$y", 2]}], + window={"documents": ["unbounded", "unbounded"]}, + ) + # x*2=[2,4], y*2=[4,8]: mean_x=3, mean_y=6 + # covPop = ((-1)(-2)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 2.0}, + ] + assertSuccess(result, expected, msg="operator expressions within array accepted") + + +def test_covariancePop_same_field_both_positions(collection): + """$covariancePop with same field in both positions equals varPop.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10}, + {"_id": 2, "partition": "A", "x": 20}, + {"_id": 3, "partition": "A", "x": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs=docs, + expression=["$x", "$x"], + window={"documents": ["unbounded", "unbounded"]}, + ) + # covPop(x,x) = varPop(x) = 200/3 = 66.6667 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "result": 66.66666666666667}, + {"_id": 2, "partition": "A", "x": 20, "result": 66.66666666666667}, + {"_id": 3, "partition": "A", "x": 30, "result": 66.66666666666667}, + ] + assertSuccess(result, expected, msg="same field both positions equals varPop") + + +def test_covariancePop_literal_numeric_expressions(collection): + """$covariancePop with literal numeric values — constant values produce covPop=0.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10}, + {"_id": 2, "partition": "A", "x": 20}, + {"_id": 3, "partition": "A", "x": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs=docs, + expression=[{"$literal": 5}, {"$literal": 10}], + window={"documents": ["unbounded", "unbounded"]}, + ) + # All rows have same pair (5,10) -> covPop = 0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="literal numeric expressions produce 0 covPop") + + +# Property [Invalid Argument Shapes Return Null]: Server does NOT reject these at parse time + + +def test_covariancePop_single_expression_not_array_returns_null(collection): + """$covariancePop with single field path (not array) returns null for all docs.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=TWO_DOCS, + expression="$x", + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="single field path (not array) returns null") + + +def test_covariancePop_single_element_array_returns_null(collection): + """$covariancePop with array of only 1 expression returns null for all docs.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=TWO_DOCS, + expression=["$x"], + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="array of 1 expression returns null") + + +def test_covariancePop_three_element_array_returns_null(collection): + """$covariancePop with array of 3 expressions returns null for all docs.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=TWO_DOCS, + expression=["$x", "$y", "$x"], + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="array of 3 expressions returns null") + + +def test_covariancePop_empty_array_returns_null(collection): + """$covariancePop with empty array [] returns null for all docs.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=TWO_DOCS, + expression=[], + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="empty array returns null") + + +def test_covariancePop_object_expression_returns_null(collection): + """$covariancePop with object (not array) as argument returns null for all docs.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=TWO_DOCS, + expression={"$add": ["$x", 1]}, + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="object (not array) as argument returns null") + + +# Property [Structural Parse Errors]: errors in $setWindowFields output spec structure + + +def test_covariancePop_unknown_key_in_output_field_errors(collection): + """Unknown key alongside $covariancePop in output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, FAILED_TO_PARSE_ERROR, msg="unknown key alongside $covariancePop rejected" + ) + + +def test_covariancePop_unknown_key_errors_on_empty_collection(collection): + """Parse-time error fires on empty collection — no documents needed.""" + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="parse-time error fires on empty collection", + ) + + +def test_covariancePop_multiple_accumulators_in_output_field_errors(collection): + """Multiple accumulators in same output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": ["$x", "$y"], + "$sum": "$x", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="multiple accumulators in output field rejected", + ) + + +def test_covariancePop_no_accumulator_in_output_field_errors(collection): + """Output field with no accumulator (only window key) produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="no accumulator in output field rejected") + + +def test_covariancePop_unrecognized_expression_operator_in_array_errors(collection): + """$covariancePop with unrecognized expression operator in array element produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": [{"$unknownOp": "$x"}, "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + UNRECOGNIZED_EXPRESSION_ERROR, + msg="unrecognized expression operator in array rejected", + ) + + +def test_covariancePop_field_path_empty_component_errors(collection): + """$covariancePop with field path containing empty component produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": ["$a..b", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + msg="field path with empty component rejected", + ) + + +def test_covariancePop_multi_field_expression_object_errors(collection): + """$covariancePop with multi-field expression object in array element produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": [ + {"$add": ["$x", 1], "$multiply": ["$x", 2]}, + "$y", + ], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + msg="multi-field expression object in array element rejected", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_field_paths.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_field_paths.py new file mode 100644 index 000000000..fa6382d91 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_field_paths.py @@ -0,0 +1,369 @@ +""" +Tests for $covariancePop with nested field paths, array field traversal, +expressions that return different types per document, and $project +removing fields before $setWindowFields. + +Covers: dotted field paths, missing intermediate paths, array index access, +array-of-objects traversal, top-level array fields, expressions returning +mixed types per row, and pipeline stages removing expression fields. + +$covariancePop takes two expressions: ["$x", "$y"]. These tests exercise +various field path forms for both expressions. +""" + +from datetime import datetime, timezone + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +# Property [Dotted Field Path]: +# Tests that $covariancePop correctly accesses nested document values via dotted paths. + + +def test_covariancePop_dotted_field_path(collection): + """$covariancePop with dotted field path accesses nested document value.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"x": 1, "y": 2}}}, + {"_id": 2, "partition": "A", "data": {"metrics": {"x": 2, "y": 4}}}, + {"_id": 3, "partition": "A", "data": {"metrics": {"x": 3, "y": 6}}}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.metrics.x", "$data.metrics.y"], + ) + # x=[1,2,3], y=[2,4,6] -> covPop = 4/3 = 1.3333... + expected = [ + { + "_id": 1, + "partition": "A", + "data": {"metrics": {"x": 1, "y": 2}}, + "result": 1.3333333333333333, + }, + { + "_id": 2, + "partition": "A", + "data": {"metrics": {"x": 2, "y": 4}}, + "result": 1.3333333333333333, + }, + { + "_id": 3, + "partition": "A", + "data": {"metrics": {"x": 3, "y": 6}}, + "result": 1.3333333333333333, + }, + ] + assertSuccess( + result, expected, msg="dotted field path accesses nested value for both expressions" + ) + + +# Property [Missing Intermediate Path]: +# Tests that missing intermediate paths are treated as missing (row ignored). + + +def test_covariancePop_missing_intermediate_path_x(collection): + """$covariancePop with missing intermediate path in first expression (x) — row ignored.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2}, + {"_id": 2, "partition": "A", "y": 4}, # missing data.x entirely + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.x", "$y"], + ) + # Doc 2 has missing x -> ignored. Only pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covPop = ((-1)(-2)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="missing intermediate path in x -> row ignored") + + +def test_covariancePop_missing_intermediate_path_y(collection): + """$covariancePop with missing intermediate path in second expression (y) — row ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "data": {"y": 2}}, + {"_id": 2, "partition": "A", "x": 2, "data": {"other": 99}}, # missing data.y + {"_id": 3, "partition": "A", "x": 3, "data": {"y": 6}}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$data.y"], + ) + # Doc 2 has missing y -> ignored. Only pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covPop = ((-1)(-2)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "data": {"y": 2}, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2, "data": {"other": 99}, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "data": {"y": 6}, "result": 2.0}, + ] + assertSuccess(result, expected, msg="missing intermediate path in y -> row ignored") + + +def test_covariancePop_top_level_missing_object(collection): + """$covariancePop where the top-level field of a dotted path is missing.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"x": 1, "y": 2}}, + {"_id": 2, "partition": "A"}, # missing 'data' entirely + {"_id": 3, "partition": "A", "data": {"x": 3, "y": 6}}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.x", "$data.y"], + ) + # Doc 2 missing both x and y -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covPop = 2.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"x": 1, "y": 2}, "result": 2.0}, + {"_id": 2, "partition": "A", "result": 2.0}, + {"_id": 3, "partition": "A", "data": {"x": 3, "y": 6}, "result": 2.0}, + ] + assertSuccess(result, expected, msg="top-level field missing in dotted path = row ignored") + + +# Property [Null Value in Nested Path]: +# Tests that a field existing with null value through a dotted path is ignored. + + +def test_covariancePop_nested_field_explicit_null(collection): + """$covariancePop with nested field that exists but is null — row ignored.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2}, + {"_id": 2, "partition": "A", "data": {"x": None}, "y": 4}, # x exists but null + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.x", "$y"], + ) + # Doc 2 has explicit null in data.x -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covPop = ((-1)(-2)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "data": {"x": None}, "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="nested field with explicit null = row ignored") + + +# Property [Array Field Non-Numeric]: +# Tests that top-level array values are treated as non-numeric and ignored. + + +def test_covariancePop_array_field_is_non_numeric(collection): + """$covariancePop on a top-level array field — arrays are non-numeric, should be ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": [1, 2, 3], "y": 10}, + {"_id": 2, "partition": "A", "x": 5, "y": 20}, + {"_id": 3, "partition": "A", "x": [4, 5], "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Docs 1 and 3 have array x -> non-numeric -> ignored + # Only doc 2 has numeric x. Single numeric pair -> covPop = 0 + expected = [ + {"_id": 1, "partition": "A", "x": [1, 2, 3], "y": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 5, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": [4, 5], "y": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="array field values are non-numeric — ignored") + + +# Property [Array at Intermediate Path Level]: +# Tests that a dotted path traversing through an array-of-objects resolves to an +# array (non-numeric) and is ignored. + + +def test_covariancePop_array_of_objects_traversal(collection): + """$covariancePop with path traversing array-of-objects — resolves to array, ignored.""" + docs = [ + {"_id": 1, "partition": "A", "items": [{"value": 10}, {"value": 20}], "y": 100}, + {"_id": 2, "partition": "A", "items": {"value": 5}, "y": 200}, + {"_id": 3, "partition": "A", "items": [{"value": 30}, {"value": 40}], "y": 300}, + {"_id": 4, "partition": "A", "items": {"value": 15}, "y": 400}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$items.value", "$y"], + ) + # Docs 1,3: items is array-of-objects -> $items.value resolves to + # [10,20]/[30,40] (array) -> ignored + # Docs 2,4: items is plain object -> $items.value resolves to 5/15 (scalar) -> participates + # Valid pairs: (5, 200) and (15, 400) + # mean_x=10, mean_y=300, covPop = ((-5)(-100)+(5)(100))/2 = 1000/2 = 500.0 + expected = [ + { + "_id": 1, + "partition": "A", + "items": [{"value": 10}, {"value": 20}], + "y": 100, + "result": 500.0, + }, + {"_id": 2, "partition": "A", "items": {"value": 5}, "y": 200, "result": 500.0}, + { + "_id": 3, + "partition": "A", + "items": [{"value": 30}, {"value": 40}], + "y": 300, + "result": 500.0, + }, + {"_id": 4, "partition": "A", "items": {"value": 15}, "y": 400, "result": 500.0}, + ] + assertSuccess(result, expected, msg="path through array-of-objects resolves to array — ignored") + + +# Property [Expression Returns Mixed Types]: +# Tests that non-numeric expression results are ignored in the computation. + + +def test_covariancePop_expression_returns_different_types(collection): + """$covariancePop expression returning different types per row — non-numeric ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20}, + {"_id": 2, "partition": "A", "x": -5, "y": 40}, + {"_id": 3, "partition": "A", "x": 30, "y": 60}, + {"_id": 4, "partition": "A", "x": -1, "y": 80}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=[{"$cond": [{"$gt": ["$x", 0]}, "$x", "not_a_number"]}, "$y"], + ) + # $cond on x returns: 10, "not_a_number", 30, "not_a_number" + # Only rows 1 and 3 have numeric first expr: pairs (10,20) and (30,60) + # mean_x=20, mean_y=40, covPop = ((-10)(-20)+(10)(20))/2 = (200+200)/2 = 200.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "result": 200.0}, + {"_id": 2, "partition": "A", "x": -5, "y": 40, "result": 200.0}, + {"_id": 3, "partition": "A", "x": 30, "y": 60, "result": 200.0}, + {"_id": 4, "partition": "A", "x": -1, "y": 80, "result": 200.0}, + ] + assertSuccess(result, expected, msg="expression returning mixed types — non-numeric ignored") + + +# Property [Expression Returns Null]: +# Tests that null expression results are ignored in the computation. + + +def test_covariancePop_expression_returns_null_for_some(collection): + """$covariancePop expression returning null for some docs, number for others.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "factor": 2}, + {"_id": 2, "partition": "A", "x": 20, "y": 40, "factor": None}, + {"_id": 3, "partition": "A", "x": 30, "y": 60, "factor": 2}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=[{"$multiply": ["$x", "$factor"]}, "$y"], + ) + # $multiply on x: [10*2=20, 20*null=null, 30*2=60] + # Row 2 produces null x -> ignored. Pairs: (20,20) and (60,60) + # mean_x=40, mean_y=40, covPop = ((-20)(-20)+(20)(20))/2 = (400+400)/2 = 400.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "factor": 2, "result": 400.0}, + {"_id": 2, "partition": "A", "x": 20, "y": 40, "factor": None, "result": 400.0}, + {"_id": 3, "partition": "A", "x": 30, "y": 60, "factor": 2, "result": 400.0}, + ] + assertSuccess(result, expected, msg="expression returning null for some — null results ignored") + + +# Property [Date Value as Expression]: +# Tests that Date values in expression field are non-numeric and ignored. + + +def test_covariancePop_date_value_as_expression_ignored(collection): + """$covariancePop with Date value in one expression field — non-numeric, ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": datetime(2023, 1, 1, tzinfo=timezone.utc), "y": 10}, + {"_id": 2, "partition": "A", "x": 20, "y": 40}, + {"_id": 3, "partition": "A", "x": datetime(2023, 6, 1, tzinfo=timezone.utc), "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Docs 1 and 3 have Date in x -> non-numeric -> ignored + # Only doc 2 is valid. Single pair -> covPop = 0 + expected = [ + { + "_id": 1, + "partition": "A", + "x": datetime(2023, 1, 1, tzinfo=timezone.utc), + "y": 10, + "result": 0.0, + }, + {"_id": 2, "partition": "A", "x": 20, "y": 40, "result": 0.0}, + { + "_id": 3, + "partition": "A", + "x": datetime(2023, 6, 1, tzinfo=timezone.utc), + "y": 30, + "result": 0.0, + }, + ] + assertSuccess(result, expected, msg="Date values in expression field are non-numeric — ignored") + + +# Property [Numeric Path Component]: +# Tests that numeric path components access array elements or object keys. + + +def test_covariancePop_numeric_path_component(collection): + """$covariancePop with numeric path component accesses array element or object key.""" + docs = [ + {"_id": 1, "partition": "A", "arr": [{"x": 10, "y": 20}]}, + {"_id": 2, "partition": "A", "arr": [{"x": 30, "y": 40}]}, + {"_id": 3, "partition": "A", "arr": [{"x": 50, "y": 60}]}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$arr.0.x", "$arr.0.y"], + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # In $setWindowFields context, $arr.0.x does not resolve to array element — + # the path returns non-numeric (array) values which are ignored, resulting in null. + expected = [ + {"_id": 1, "result": None}, + {"_id": 2, "result": None}, + {"_id": 3, "result": None}, + ] + assertSuccess(result, expected, msg="numeric path component in window context returns null") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_frame_computation.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_frame_computation.py new file mode 100644 index 000000000..4a24201cb --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_frame_computation.py @@ -0,0 +1,261 @@ +""" +Tests for $covariancePop computation under documents-mode window frame shapes. + +Verifies the operator computes correct results given the 4 defined frame shapes: +whole-partition, cumulative, reverse-cumulative, and sliding. + +$covariancePop semantics: +- Takes array of exactly 2 expressions: ["$x", "$y"] +- Population covariance = sum((xi - mean_x)(yi - mean_y)) / N +- Single value (N=1) -> covariancePop = 0 (divides by N, not N-1) +- Empty window -> null + +Note: Stage-level frame boundary tests (under stages/setWindowFields/) verify +that the correct documents are selected into the frame (centered, trailing, +leading, non-overlapping, edge cases). These per-operator tests verify the +operator produces correct values given those documents. +""" + +import pytest + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + COVAR_DOCS, + WindowTestCase, + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.parametrize import pytest_params + +# COVAR_DOCS: x = [1,2,3,4,5], y = [2,4,6,8,10] (y = 2x) +# mean_x = 3, mean_y = 6 +# covPop = sum((xi-3)(yi-6))/5 = (8+2+0+2+8)/5 = 20/5 = 4.0 + + +COVARIANCEPOP_DOCUMENTS_FRAME_TESTS: list[WindowTestCase] = [ + # Property [Whole Partition]: unbounded-unbounded frame covers entire partition + # covPop(x, y) for all 5 docs = 4.0 (calculated above) + WindowTestCase( + "whole_partition", + docs=COVAR_DOCS, + window={"documents": ["unbounded", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 4.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 4.0}, + ], + msg="whole partition covariancePop should be 4.0", + ), + # Property [Cumulative Frame]: expanding frame from start to current + # Row 1 (n=1): [(1,2)] -> covPop = 0 (single value, divides by N=1) + # Row 2 (n=2): [(1,2),(2,4)] -> mean_x=1.5, mean_y=3 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = (0.5+0.5)/2 = 0.5 + # Row 3 (n=3): [(1,2),(2,4),(3,6)] -> mean_x=2, mean_y=4 + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = (2+0+2)/3 = 4/3 = 1.3333... + # Row 4 (n=4): [(1,2),(2,4),(3,6),(4,8)] -> mean_x=2.5, mean_y=5 + # covPop = ((-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3))/4 = (4.5+0.5+0.5+4.5)/4 = 10/4 = 2.5 + # Row 5 (n=5): all docs -> covPop = 4.0 + WindowTestCase( + "cumulative", + docs=COVAR_DOCS, + window={"documents": ["unbounded", "current"]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 0.5}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 1.3333333333333333}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 2.5}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 4.0}, + ], + msg="cumulative covariancePop should grow", + ), + # Property [Reverse Cumulative Frame]: shrinking frame from current to end + # Row 1 (n=5): all docs -> covPop = 4.0 + # Row 2 (n=4): [(2,4),(3,6),(4,8),(5,10)] -> mean_x=3.5, mean_y=7 + # covPop = ((-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3))/4 = (4.5+0.5+0.5+4.5)/4 = 2.5 + # Row 3 (n=3): [(3,6),(4,8),(5,10)] -> mean_x=4, mean_y=8 + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = 4/3 = 1.3333... + # Row 4 (n=2): [(4,8),(5,10)] -> mean_x=4.5, mean_y=9 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 1/2 = 0.5 + # Row 5 (n=1): [(5,10)] -> covPop = 0 + WindowTestCase( + "reverse_cumulative", + docs=COVAR_DOCS, + window={"documents": ["current", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 2.5}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 1.3333333333333333}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 0.5}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 0.0}, + ], + msg="reverse-cumulative covariancePop should shrink", + ), + # Property [Sliding Frame]: fixed-size window that moves with current row + # Window [-1, 1] = 3-doc centered (clamped at edges) + # Row 1: [(1,2),(2,4)] (edge clamp) -> mean_x=1.5, mean_y=3 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 0.5 + # Row 2: [(1,2),(2,4),(3,6)] -> mean_x=2, mean_y=4 + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = 4/3 = 1.3333... + # Row 3: [(2,4),(3,6),(4,8)] -> mean_x=3, mean_y=6 + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = 4/3 = 1.3333... + # Row 4: [(3,6),(4,8),(5,10)] -> mean_x=4, mean_y=8 + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = 4/3 = 1.3333... + # Row 5: [(4,8),(5,10)] (edge clamp) -> mean_x=4.5, mean_y=9 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 0.5 + WindowTestCase( + "sliding_centered", + docs=COVAR_DOCS, + window={"documents": [-1, 1]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 0.5}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 1.3333333333333333}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 1.3333333333333333}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 1.3333333333333333}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 0.5}, + ], + msg="centered sliding window [-1,1]", + ), +] + + +@pytest.mark.parametrize("test", pytest_params(COVARIANCEPOP_DOCUMENTS_FRAME_TESTS)) +def test_covariancePop_documents_frames(collection, test): + """$covariancePop with various documents-mode window frames.""" + result = run_window_operator( + collection, + "$covariancePop", + test.docs, + test.window, + sort_by=test.sort_by, + expression=["$x", "$y"], + ) + assertSuccess(result, test.expected, msg=test.msg) + + +def test_covariancePop_negative_correlation(collection): + """$covariancePop with negative correlation (y decreases as x increases).""" + # x = [1, 2, 3], y = [6, 4, 2] -> y = -2x + 8 + # mean_x=2, mean_y=4 + # covPop = ((-1)(2)+(0)(0)+(1)(-2))/3 = (-2+0-2)/3 = -4/3 = -1.3333... + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 6}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 2}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 6, "result": -1.3333333333333333}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": -1.3333333333333333}, + {"_id": 3, "partition": "A", "x": 3, "y": 2, "result": -1.3333333333333333}, + ] + assertSuccess(result, expected, msg="negative correlation produces negative covariancePop") + + +def test_covariancePop_zero_covariance(collection): + """$covariancePop returns 0 when variables are uncorrelated.""" + # x = [1, 2, 3], y = [5, 5, 5] -> y is constant + # mean_x=2, mean_y=5 + # covPop = ((-1)(0)+(0)(0)+(1)(0))/3 = 0 + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 5}, + {"_id": 2, "partition": "A", "x": 2, "y": 5}, + {"_id": 3, "partition": "A", "x": 3, "y": 5}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 5, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 5, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 5, "result": 0.0}, + ] + assertSuccess(result, expected, msg="constant y produces zero covariance") + + +def test_covariancePop_identical_x_and_y(collection): + """$covariancePop where x == y reduces to population variance.""" + # When x==y: covPop(x,x) = varPop(x) + # x = [10, 20, 30] -> mean=20, varPop = (100+0+100)/3 = 200/3 = 66.6667 + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 10}, + {"_id": 2, "partition": "A", "x": 20, "y": 20}, + {"_id": 3, "partition": "A", "x": 30, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # covPop(x,x) = varPop(x) = 200/3 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 10, "result": 66.66666666666667}, + {"_id": 2, "partition": "A", "x": 20, "y": 20, "result": 66.66666666666667}, + {"_id": 3, "partition": "A", "x": 30, "y": 30, "result": 66.66666666666667}, + ] + assertSuccess(result, expected, msg="covPop(x,x) equals varPop(x)") + + +def test_covariancePop_trailing_sliding_window(collection): + """$covariancePop with trailing sliding window [-1, 0].""" + # Window [-1, 0] = look-back 1 row + current + # Row 1: [(1,2)] (only current, edge) -> n=1 -> covPop = 0 + # Row 2: [(1,2),(2,4)] -> covPop = 0.5 (same as cumulative row 2) + # Row 3: [(2,4),(3,6)] -> mean_x=2.5, mean_y=5 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 0.5 + # Row 4: [(3,6),(4,8)] -> mean_x=3.5, mean_y=7 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 0.5 + # Row 5: [(4,8),(5,10)] -> mean_x=4.5, mean_y=9 + # covPop = ((-0.5)(-1)+(0.5)(1))/2 = 0.5 + docs = COVAR_DOCS + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 0.5}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 0.5}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 0.5}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 0.5}, + ] + assertSuccess(result, expected, msg="trailing sliding window [-1, 0]") + + +def test_covariancePop_empty_window_returns_null(collection): + """$covariancePop returns null when the window frame contains zero documents.""" + # Window [5, 10] on a 3-doc partition: for all rows, no documents fall in the frame + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [5, 10]}, + expression=["$x", "$y"], + ) + # Frame [5, 10] means offset +5 to +10 from current row — no such rows exist + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": None}, + ] + assertSuccess(result, expected, msg="empty window frame returns null") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_non_numeric_handling.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_non_numeric_handling.py new file mode 100644 index 000000000..b68c2bb57 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_non_numeric_handling.py @@ -0,0 +1,452 @@ +""" +Tests for $covariancePop null, missing, and non-numeric value handling. + +Covers: null values, missing fields, strings, booleans, arrays, objects, +ObjectId, Regex, Binary, Timestamp, MinKey, MaxKey, mixed numeric and +non-numeric in same frame, and all non-numeric returns null. + +$covariancePop semantics: when either expression in ["$x", "$y"] evaluates to +a non-numeric, null, or missing value for a document, that entire row (pair) +is ignored in the covariance computation. +""" + +from datetime import datetime, timezone + +from bson import Binary, MaxKey, MinKey, ObjectId, Regex, Timestamp + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +# Property [Null and Missing]: null and missing field values cause the row to be ignored + + +def test_covariancePop_null_in_x_ignored(collection): + """$covariancePop ignores rows where x expression is null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": None, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has null x -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covPop = ((-1)(-2)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": None, "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="null x values ignored, covPop of (1,2),(3,6) = 2.0") + + +def test_covariancePop_null_in_y_ignored(collection): + """$covariancePop ignores rows where y expression is null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has null y -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covPop = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2, "y": None, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="null y values ignored") + + +def test_covariancePop_missing_x_field_ignored(collection): + """$covariancePop ignores documents where the x field is missing.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "y": 4}, # x missing + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has missing x -> ignored. Pairs: (1,2) and (3,6) -> covPop = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="missing x field ignored") + + +def test_covariancePop_missing_y_field_ignored(collection): + """$covariancePop ignores documents where the y field is missing.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2}, # y missing + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has missing y -> ignored. Pairs: (1,2) and (3,6) -> covPop = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="missing y field ignored") + + +def test_covariancePop_both_null_ignored(collection): + """$covariancePop ignores rows where both x and y are null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": None, "y": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": None, "y": None, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="both null values -> row ignored") + + +# Property [Non-Numeric Types Ignored]: string, boolean, array, object, date, +# ObjectId, Regex, Binary values are ignored + + +def test_covariancePop_string_in_x_ignored(collection): + """$covariancePop ignores rows where x is a string.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": "hello", "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": "hello", "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="string x values ignored") + + +def test_covariancePop_string_in_y_ignored(collection): + """$covariancePop ignores rows where y is a string.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": "world"}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2, "y": "world", "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="string y values ignored") + + +def test_covariancePop_boolean_values_ignored(collection): + """$covariancePop ignores rows where x or y is boolean.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": True, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": False}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Rows 2 and 3 ignored. Valid pairs: (1,2) and (4,8) + # mean_x=2.5, mean_y=5, covPop = ((-1.5)(-3)+(1.5)(3))/2 = (4.5+4.5)/2 = 4.5 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.5}, + {"_id": 2, "partition": "A", "x": True, "y": 4, "result": 4.5}, + {"_id": 3, "partition": "A", "x": 3, "y": False, "result": 4.5}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 4.5}, + ] + assertSuccess(result, expected, msg="boolean values ignored in both positions") + + +def test_covariancePop_array_values_ignored(collection): + """$covariancePop ignores rows where x or y is an array.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": [1, 2, 3], "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": [1, 2, 3], "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="array values ignored") + + +def test_covariancePop_object_values_ignored(collection): + """$covariancePop ignores rows where x or y is an object/document.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": {"nested": 99}, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": {"nested": 99}, "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="object values ignored") + + +def test_covariancePop_objectid_and_regex_and_binary_ignored(collection): + """$covariancePop ignores ObjectId, Regex, and Binary values.""" + oid = ObjectId("507f1f77bcf86cd799439011") + docs = [ + {"_id": 1, "partition": "A", "x": oid, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": Regex("^test", "i")}, + {"_id": 3, "partition": "A", "x": Binary(b"\x01\x02\x03"), "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + {"_id": 5, "partition": "A", "x": 6, "y": 12}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only rows 4 and 5 valid: (4,8) and (6,12) + # mean_x=5, mean_y=10, covPop = ((-1)(-2)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "result": 2.0}, + {"_id": 2, "result": 2.0}, + {"_id": 3, "result": 2.0}, + {"_id": 4, "result": 2.0}, + {"_id": 5, "result": 2.0}, + ] + assertSuccess(result, expected, msg="ObjectId/Regex/Binary values ignored") + + +def test_covariancePop_timestamp_minkey_maxkey_ignored(collection): + """$covariancePop ignores Timestamp, MinKey, and MaxKey values.""" + docs = [ + {"_id": 1, "partition": "A", "x": Timestamp(1234567890, 1), "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": MinKey()}, + {"_id": 3, "partition": "A", "x": MaxKey(), "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + {"_id": 5, "partition": "A", "x": 6, "y": 12}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only rows 4 and 5 valid: (4,8) and (6,12) -> covPop = 2.0 + expected = [ + {"_id": 1, "result": 2.0}, + {"_id": 2, "result": 2.0}, + {"_id": 3, "result": 2.0}, + {"_id": 4, "result": 2.0}, + {"_id": 5, "result": 2.0}, + ] + assertSuccess(result, expected, msg="Timestamp/MinKey/MaxKey values ignored") + + +# Property [All Non-Numeric Returns Null]: when all values are non-numeric, result is null + + +def test_covariancePop_all_non_numeric_returns_null(collection): + """$covariancePop returns null when all values in frame are non-numeric.""" + docs = [ + {"_id": 1, "partition": "A", "x": "a", "y": 2}, + {"_id": 2, "partition": "A", "x": None, "y": 4}, + {"_id": 3, "partition": "A", "y": 6}, # x missing + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": "a", "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": None, "y": 4, "result": None}, + {"_id": 3, "partition": "A", "y": 6, "result": None}, + ] + assertSuccess(result, expected, msg="all non-numeric x values in frame returns null") + + +def test_covariancePop_all_non_numeric_diverse_types(collection): + """$covariancePop returns null when all values are diverse non-numeric types.""" + docs = [ + {"_id": 1, "partition": "A", "x": "text", "y": 10}, + {"_id": 2, "partition": "A", "x": True, "y": 20}, + {"_id": 3, "partition": "A", "x": datetime(2023, 1, 1, tzinfo=timezone.utc), "y": 30}, + {"_id": 4, "partition": "A", "x": [1, 2], "y": 40}, + {"_id": 5, "partition": "A", "x": {"a": 1}, "y": 50}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # No valid numeric x values -> null + expected = [ + {"_id": 1, "partition": "A", "x": "text", "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": True, "y": 20, "result": None}, + { + "_id": 3, + "partition": "A", + "x": datetime(2023, 1, 1, tzinfo=timezone.utc), + "y": 30, + "result": None, + }, + {"_id": 4, "partition": "A", "x": [1, 2], "y": 40, "result": None}, + {"_id": 5, "partition": "A", "x": {"a": 1}, "y": 50, "result": None}, + ] + assertSuccess(result, expected, msg="all diverse non-numeric types return null") + + +# Property [Mixed Types in Frame]: non-numeric values filtered per-frame, numerics participate + + +def test_covariancePop_mixed_numeric_non_numeric_sliding(collection): + """$covariancePop in sliding window with mix of numeric and non-numeric values.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": "skip", "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": None, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-2, 2]}, + expression=["$x", "$y"], + ) + # Window [-2, 2] (5-doc centered): + # Row 1: frame docs 1-3, valid pairs: (1,10),(3,30) -> mean_x=2,mean_y=20 + # covPop = ((-1)(-10)+(1)(10))/2 = 20/2 = 10.0 + # Row 2: frame docs 1-4, valid pairs: (1,10),(3,30) -> covPop = 10.0 + # Row 3: frame docs 1-5, valid pairs: (1,10),(3,30),(5,50) -> mean_x=3,mean_y=30 + # covPop = ((-2)(-20)+(0)(0)+(2)(20))/3 = (40+0+40)/3 = 80/3 = 26.6667 + # Row 4: frame docs 2-5, valid pairs: (3,30),(5,50) -> mean_x=4,mean_y=40 + # covPop = ((-1)(-10)+(1)(10))/2 = 10.0 + # Row 5: frame docs 3-5, valid pairs: (3,30),(5,50) -> covPop = 10.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "x": "skip", "y": 20, "result": 10.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 26.666666666666668}, + {"_id": 4, "partition": "A", "x": None, "y": 40, "result": 10.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 10.0}, + ] + assertSuccess(result, expected, msg="mixed types in sliding window — non-numeric ignored") + + +def test_covariancePop_numeric_among_diverse_types_cumulative(collection): + """$covariancePop cumulative window with numerics scattered among diverse types.""" + docs = [ + {"_id": 1, "partition": "A", "x": "text", "y": 2}, + {"_id": 2, "partition": "A", "x": 1, "y": 2}, + {"_id": 3, "partition": "A", "x": datetime(2023, 6, 1, tzinfo=timezone.utc), "y": 4}, + {"_id": 4, "partition": "A", "x": 3, "y": 6}, + {"_id": 5, "partition": "A", "x": True, "y": 8}, + {"_id": 6, "partition": "A", "x": 5, "y": 10}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "current"]}, + expression=["$x", "$y"], + ) + # Cumulative, only numeric x and numeric y pairs count: + # Row 1: no valid pair -> null + # Row 2: [(1,2)] -> single pair -> 0 + # Row 3: [(1,2)] -> datetime x ignored, still single pair -> 0 + # Row 4: [(1,2),(3,6)] -> mean_x=2, mean_y=4, covPop = ((-1)(-2)+(1)(2))/2 = 2.0 + # Row 5: [(1,2),(3,6)] -> True ignored, still 2 pairs -> 2.0 + # Row 6: [(1,2),(3,6),(5,10)] -> mean_x=3, mean_y=6 + # covPop = ((-2)(-4)+(0)(0)+(2)(4))/3 = 16/3 = 5.3333... + expected = [ + {"_id": 1, "partition": "A", "x": "text", "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 1, "y": 2, "result": 0.0}, + { + "_id": 3, + "partition": "A", + "x": datetime(2023, 6, 1, tzinfo=timezone.utc), + "y": 4, + "result": 0.0, + }, + {"_id": 4, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + {"_id": 5, "partition": "A", "x": True, "y": 8, "result": 2.0}, + {"_id": 6, "partition": "A", "x": 5, "y": 10, "result": 5.333333333333333}, + ] + assertSuccess(result, expected, msg="cumulative window with numerics among diverse types") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_numeric_precision.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_numeric_precision.py new file mode 100644 index 000000000..e0a3d34c3 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_numeric_precision.py @@ -0,0 +1,1464 @@ +""" +Tests for $covariancePop numeric type mixing, overflow edge cases, +algorithmic precision validation, and Decimal128 type handling. + +Covers: Int32/Int64/Double mixing, Int64 near MAX_LONG (overflow risk when +squaring), catastrophic cancellation in variance calculation, known exact +results, very small differences, consistency between window modes, +Decimal128 (NumberDecimal) values, high-precision Decimal128, mixed Decimal128 +with other numeric types, and Decimal128 special values (NaN, Infinity). + +Server behavior (verified): +- When ANY input value is Decimal128, the server returns Decimal128 type results +- Pure Decimal128 returns Decimal128("1.333333333333333333333333333333333") +- Decimal128 identical values returns Decimal128("0E+12"), not float 0.0 +- Decimal128 sliding window: Row 1 returns float 0.0, subsequent rows Decimal128("0.5") +- 1e308 identical values: returns NaN (overflow in intermediate computation) +""" + +from bson import Decimal128, Int64 + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertResult, assertSuccess, assertSuccessNaN +from documentdb_tests.framework.executor import execute_command +from documentdb_tests.framework.property_checks import Gt, Lte, PerDoc +from documentdb_tests.framework.test_constants import ( + DECIMAL128_INFINITY, + DECIMAL128_LARGE_EXPONENT, + DECIMAL128_MAX, + DECIMAL128_MIN, + DECIMAL128_NEGATIVE_INFINITY, + DECIMAL128_NEGATIVE_ZERO, + DECIMAL128_SMALL_EXPONENT, + DOUBLE_NEAR_MAX, + DOUBLE_NEGATIVE_ZERO, + FLOAT_INFINITY, + FLOAT_NAN, + FLOAT_NEGATIVE_INFINITY, +) + +# Property [Numeric Type Mixing]: Int32, Int64, Double coexist correctly + + +def test_covariancePop_all_int32_values(collection): + """$covariancePop with all Int32 values produces Double result.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1,2,3], y=[2,4,6]: covPop = 4/3 = 1.3333... + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 1.3333333333333333}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 1.3333333333333333}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 1.3333333333333333}, + ] + assertSuccess(result, expected, msg="all Int32 values produce correct Double result") + + +def test_covariancePop_all_int64_values(collection): + """$covariancePop with all Int64 values produces correct result.""" + docs = [ + {"_id": 1, "partition": "A", "x": Int64(1), "y": Int64(2)}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": Int64(4)}, + {"_id": 3, "partition": "A", "x": Int64(3), "y": Int64(6)}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": Int64(1), "y": Int64(2), "result": 1.3333333333333333}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": Int64(4), "result": 1.3333333333333333}, + {"_id": 3, "partition": "A", "x": Int64(3), "y": Int64(6), "result": 1.3333333333333333}, + ] + assertSuccess(result, expected, msg="all Int64 values compute correctly") + + +def test_covariancePop_mixed_int32_int64_double(collection): + """$covariancePop with mixed Int32 + Int64 + Double in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2.0}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": 4}, + {"_id": 3, "partition": "A", "x": 3.0, "y": Int64(6)}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2.0, "result": 1.3333333333333333}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": 4, "result": 1.3333333333333333}, + {"_id": 3, "partition": "A", "x": 3.0, "y": Int64(6), "result": 1.3333333333333333}, + ] + assertSuccess(result, expected, msg="mixed Int32 + Int64 + Double type promotion works") + + +# Property [Large Value Handling]: near-overflow and large-spread values compute without overflow + + +def test_covariancePop_large_int64_near_max(collection): + """$covariancePop with Int64 values near MAX_LONG — squaring would overflow 64-bit.""" + docs = [ + { + "_id": 1, + "partition": "A", + "x": Int64(9223372036854775806), + "y": Int64(9223372036854775806), + }, + { + "_id": 2, + "partition": "A", + "x": Int64(9223372036854775807), + "y": Int64(9223372036854775807), + }, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Both values round to the same float64 at this scale -> covPop = 0.0 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Int64(9223372036854775806), + "y": Int64(9223372036854775806), + "result": 0.0, + }, + { + "_id": 2, + "partition": "A", + "x": Int64(9223372036854775807), + "y": Int64(9223372036854775807), + "result": 0.0, + }, + ] + assertSuccess(result, expected, msg="Int64 near MAX_LONG does not overflow") + + +def test_covariancePop_large_int64_spread(collection): + """$covariancePop with widely spread Int64 values — tests numeric stability.""" + docs = [ + {"_id": 1, "partition": "A", "x": Int64(0), "y": Int64(0)}, + { + "_id": 2, + "partition": "A", + "x": Int64(4611686018427387903), + "y": Int64(4611686018427387903), + }, + { + "_id": 3, + "partition": "A", + "x": Int64(9223372036854775807), + "y": Int64(9223372036854775807), + }, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # When x==y, covPop(x,y) = varPop(x) > 0 + checks = PerDoc( + {"result": Gt(0)}, + {"result": Gt(0)}, + {"result": Gt(0)}, + ) + assertResult(result, expected=checks, msg="Large Int64 spread produces positive result") + + +def test_covariancePop_very_large_value(collection): + """$covariancePop with very large numeric value (1e308) — all identical returns NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1e308, "y": 1e308}, + {"_id": 2, "partition": "A", "x": 1e308, "y": 1e308}, + {"_id": 3, "partition": "A", "x": 1e308, "y": 1e308}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns NaN due to overflow in intermediate computation + expected = [ + {"_id": 1, "partition": "A", "x": 1e308, "y": 1e308, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": 1e308, "y": 1e308, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 1e308, "y": 1e308, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="very large identical values overflow to NaN") + + +def test_covariancePop_alternating_large_values(collection): + """$covariancePop with alternating sign large values — stress accumulator.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1e15, "y": 1e15}, + {"_id": 2, "partition": "A", "x": -1e15, "y": -1e15}, + {"_id": 3, "partition": "A", "x": 1e15, "y": 1e15}, + {"_id": 4, "partition": "A", "x": -1e15, "y": -1e15}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # When x==y: covPop = varPop. Mean=0, var = (4*1e30)/4 = 1e30 + expected = [ + {"_id": 1, "partition": "A", "x": 1e15, "y": 1e15, "result": 1e30}, + {"_id": 2, "partition": "A", "x": -1e15, "y": -1e15, "result": 1e30}, + {"_id": 3, "partition": "A", "x": 1e15, "y": 1e15, "result": 1e30}, + {"_id": 4, "partition": "A", "x": -1e15, "y": -1e15, "result": 1e30}, + ] + assertSuccess(result, expected, msg="alternating large values produce correct covariancePop") + + +# Property [Algorithmic Precision]: known exact results and catastrophic cancellation handling + + +def test_covariancePop_known_exact_result(collection): + """$covariancePop with known exact result: covPop([1,2,3,4],[2,4,6,8]) = 2.5.""" + docs = [ + {"_id": i, "partition": "A", "x": x, "y": y} + for i, (x, y) in enumerate([(1, 2), (2, 4), (3, 6), (4, 8)], 1) + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1,2,3,4], y=[2,4,6,8]: mean_x=2.5, mean_y=5 + # covPop = ((-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3))/4 = (4.5+0.5+0.5+4.5)/4 = 2.5 + expected = [ + {"_id": i, "partition": "A", "x": x, "y": y, "result": 2.5} + for i, (x, y) in enumerate([(1, 2), (2, 4), (3, 6), (4, 8)], 1) + ] + assertSuccess(result, expected, msg="covariancePop of [1,2,3,4],[2,4,6,8] must be exactly 2.5") + + +def test_covariancePop_identical_values_exactly_zero(collection): + """$covariancePop of identical (x,y) pairs where y is constant must be exactly 0.0.""" + docs = [ + {"_id": 1, "partition": "A", "x": 3.0, "y": 7.0}, + {"_id": 2, "partition": "A", "x": 3.0, "y": 7.0}, + {"_id": 3, "partition": "A", "x": 3.0, "y": 7.0}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 3.0, "y": 7.0, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 3.0, "y": 7.0, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3.0, "y": 7.0, "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical pairs produce exactly 0.0") + + +def test_covariancePop_catastrophic_cancellation(collection): + """$covariancePop with large offset values — naive algorithm fails.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1000000001, "y": 1000000002}, + {"_id": 2, "partition": "A", "x": 1000000002, "y": 1000000004}, + {"_id": 3, "partition": "A", "x": 1000000003, "y": 1000000006}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x = [N, N+1, N+2], y = [M, M+2, M+4] where N,M are large + # After centering: x-offsets = [-1, 0, 1], y-offsets = [-2, 0, 2] + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = 4/3 = 1.3333... + expected = [ + { + "_id": 1, + "partition": "A", + "x": 1000000001, + "y": 1000000002, + "result": 1.3333333333333333, + }, + { + "_id": 2, + "partition": "A", + "x": 1000000002, + "y": 1000000004, + "result": 1.3333333333333333, + }, + { + "_id": 3, + "partition": "A", + "x": 1000000003, + "y": 1000000006, + "result": 1.3333333333333333, + }, + ] + assertSuccess( + result, + expected, + msg="catastrophic cancellation handled — correct covPop for large offset values", + ) + + +def test_covariancePop_very_small_differences(collection): + """$covariancePop with values that differ by very small amounts.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1.0000001, "y": 2.0000002}, + {"_id": 2, "partition": "A", "x": 1.0000002, "y": 2.0000004}, + {"_id": 3, "partition": "A", "x": 1.0000003, "y": 2.0000006}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # The covariancePop should be positive and tiny + checks = PerDoc( + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + ) + assertResult( + result, expected=checks, msg="Small differences produce very small positive covPop" + ) + + +# Property [Single Element Frame]: single value produces 0 for population covariance + + +def test_covariancePop_single_element_sliding_window(collection): + """$covariancePop returns 0 when sliding window frame has exactly one value.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [0, 0]}, + expression=["$x", "$y"], + ) + # Window [0, 0] — each frame has exactly one value -> covPop = 0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 0.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 0.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 0.0}, + ] + assertSuccess(result, expected, msg="single element in sliding frame returns 0") + + +# Property [Decimal128 Support]: Decimal128 values return Decimal128 type results. + + +def test_covariancePop_pure_decimal128_values(collection): + """$covariancePop with pure Decimal128 values returns Decimal128 type result.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type with high precision + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": Decimal128("2"), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2"), + "y": Decimal128("4"), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Decimal128("6"), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + ] + assertSuccess(result, expected, msg="pure Decimal128 values return Decimal128 type result") + + +def test_covariancePop_decimal128_with_double(collection): + """$covariancePop with mixed Decimal128 and Double returns Decimal128 type.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2.0}, + {"_id": 2, "partition": "A", "x": 2.0, "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": 6.0}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": 2.0, + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 2, + "partition": "A", + "x": 2.0, + "y": Decimal128("4"), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": 6.0, + "result": Decimal128("1.333333333333333333333333333333333"), + }, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Double returns Decimal128 type") + + +def test_covariancePop_decimal128_with_int32(collection): + """$covariancePop with mixed Decimal128 and Int32 returns Decimal128 type.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": 2, + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 2, + "partition": "A", + "x": 2, + "y": Decimal128("4"), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": 6, + "result": Decimal128("1.333333333333333333333333333333333"), + }, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int32 returns Decimal128 type") + + +def test_covariancePop_decimal128_with_int64(collection): + """$covariancePop with mixed Decimal128 and Int64 returns Decimal128 type.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Int64(2)}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Int64(6)}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": Int64(2), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 2, + "partition": "A", + "x": Int64(2), + "y": Decimal128("4"), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Int64(6), + "result": Decimal128("1.333333333333333333333333333333333"), + }, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int64 returns Decimal128 type") + + +def test_covariancePop_decimal128_all_types_mixed(collection): + """$covariancePop with Decimal128 + Double + Int32 + Int64 all in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2.0}, + {"_id": 2, "partition": "A", "x": 2.0, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": Int64(6)}, + {"_id": 4, "partition": "A", "x": Int64(4), "y": Decimal128("8")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1,2,3,4], y=[2,4,6,8] -> covPop = 2.5 + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": 2.0, + "result": Decimal128("2.50000000000000"), + }, + {"_id": 2, "partition": "A", "x": 2.0, "y": 4, "result": Decimal128("2.50000000000000")}, + { + "_id": 3, + "partition": "A", + "x": 3, + "y": Int64(6), + "result": Decimal128("2.50000000000000"), + }, + { + "_id": 4, + "partition": "A", + "x": Int64(4), + "y": Decimal128("8"), + "result": Decimal128("2.50000000000000"), + }, + ] + assertSuccess(result, expected, msg="all four numeric types mixed returns Decimal128 type") + + +def test_covariancePop_decimal128_sliding_window(collection): + """$covariancePop with Decimal128 values in a sliding window.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + {"_id": 4, "partition": "A", "x": Decimal128("4"), "y": Decimal128("8")}, + {"_id": 5, "partition": "A", "x": Decimal128("5"), "y": Decimal128("10")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Window [-1, 0]: + # Row 1: [(1,2)] -> single pair -> 0.0 (float, before Decimal128 pair contributes) + # Row 2: [(1,2),(2,4)] -> Decimal128("0.5") + # Row 3: [(2,4),(3,6)] -> Decimal128("0.5") + # Row 4: [(3,6),(4,8)] -> Decimal128("0.5") + # Row 5: [(4,8),(5,10)] -> Decimal128("0.5") + expected = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2"), "result": 0.0}, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2"), + "y": Decimal128("4"), + "result": Decimal128("0.5"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Decimal128("6"), + "result": Decimal128("0.5"), + }, + { + "_id": 4, + "partition": "A", + "x": Decimal128("4"), + "y": Decimal128("8"), + "result": Decimal128("0.5"), + }, + { + "_id": 5, + "partition": "A", + "x": Decimal128("5"), + "y": Decimal128("10"), + "result": Decimal128("0.5"), + }, + ] + assertSuccess(result, expected, msg="Decimal128 sliding window returns Decimal128 type") + + +def test_covariancePop_decimal128_identical_values(collection): + """$covariancePop with identical Decimal128 value pairs returns Decimal128('0E+12').""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("42.5"), "y": Decimal128("99.9")}, + {"_id": 2, "partition": "A", "x": Decimal128("42.5"), "y": Decimal128("99.9")}, + {"_id": 3, "partition": "A", "x": Decimal128("42.5"), "y": Decimal128("99.9")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128("0E+12") for identical values, not float 0.0 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("42.5"), + "y": Decimal128("99.9"), + "result": Decimal128("0E+12"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("42.5"), + "y": Decimal128("99.9"), + "result": Decimal128("0E+12"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("42.5"), + "y": Decimal128("99.9"), + "result": Decimal128("0E+12"), + }, + ] + assertSuccess( + result, expected, msg="identical Decimal128 value pairs return Decimal128('0E+12')" + ) + + +# Property [Decimal128 Special Values]: Decimal128 NaN and Infinity handling + + +def test_covariancePop_decimal128_nan_special(collection): + """$covariancePop with Decimal128 NaN — NaN is numeric and poisons the calculation.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("NaN"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": Decimal128("2"), + "result": FLOAT_NAN, + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("NaN"), + "y": Decimal128("4"), + "result": FLOAT_NAN, + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Decimal128("6"), + "result": FLOAT_NAN, + }, + ] + assertSuccessNaN( + result, expected, msg="Decimal128 NaN is numeric and poisons covariancePop to NaN" + ) + + +def test_covariancePop_decimal128_infinity_special(collection): + """$covariancePop with Decimal128 Infinity special value.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("Infinity"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + ] + collection.insert_many(docs) + extra_stages = [ + { + "$addFields": { + "has_result": {"$ne": ["$result", None]}, + } + }, + {"$project": {"_id": 1, "has_result": 1}}, + ] + pipeline = [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covariancePop": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ] + extra_stages + result = execute_command( + collection, + {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}}, + ) + expected = [ + {"_id": 1, "has_result": True}, + {"_id": 2, "has_result": True}, + {"_id": 3, "has_result": True}, + ] + assertSuccess(result, expected, msg="Decimal128 Infinity produces a non-null result") + + +# Property [Decimal128 Precision Boundaries]: boundary values from test_constants + + +def test_covariancePop_decimal128_min_values(collection): + """$covariancePop with DECIMAL128_MIN values — overflow in intermediate computation.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MIN, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MIN, "y": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MIN, "y": Decimal128("3")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # All x values identical, but intermediate computation overflows -> Infinity + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_MIN, + "y": Decimal128("1"), + "result": Decimal128("Infinity"), + }, + { + "_id": 2, + "partition": "A", + "x": DECIMAL128_MIN, + "y": Decimal128("2"), + "result": Decimal128("Infinity"), + }, + { + "_id": 3, + "partition": "A", + "x": DECIMAL128_MIN, + "y": Decimal128("3"), + "result": Decimal128("Infinity"), + }, + ] + assertSuccess( + result, expected, msg="DECIMAL128_MIN overflows to Infinity in intermediate computation" + ) + + +def test_covariancePop_decimal128_large_exponent(collection): + """$covariancePop with DECIMAL128_LARGE_EXPONENT values — high exponent Decimal128.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_LARGE_EXPONENT, "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2E+6144"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3E+6144"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1E+6144, 2E+6144, 3E+6144], y=[2,4,6]: covPop = 4/3 * 1E+6144 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_LARGE_EXPONENT, + "y": Decimal128("2"), + "result": Decimal128("1.333333333333333333333333333333333E+6144"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2E+6144"), + "y": Decimal128("4"), + "result": Decimal128("1.333333333333333333333333333333333E+6144"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3E+6144"), + "y": Decimal128("6"), + "result": Decimal128("1.333333333333333333333333333333333E+6144"), + }, + ] + assertSuccess(result, expected, msg="DECIMAL128_LARGE_EXPONENT produces positive covPop") + + +def test_covariancePop_decimal128_small_exponent(collection): + """$covariancePop with DECIMAL128_SMALL_EXPONENT values — very small Decimal128.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_SMALL_EXPONENT, "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2E-6143"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3E-6143"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1E-6143, 2E-6143, 3E-6143], y=[2,4,6]: covPop = 4/3 * 1E-6143 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_SMALL_EXPONENT, + "y": Decimal128("2"), + "result": Decimal128("1.333333333333333333333333333333333E-6143"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2E-6143"), + "y": Decimal128("4"), + "result": Decimal128("1.333333333333333333333333333333333E-6143"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3E-6143"), + "y": Decimal128("6"), + "result": Decimal128("1.333333333333333333333333333333333E-6143"), + }, + ] + assertSuccess(result, expected, msg="DECIMAL128_SMALL_EXPONENT produces positive covPop") + + +def test_covariancePop_decimal128_negative_zero(collection): + """$covariancePop with DECIMAL128_NEGATIVE_ZERO — treated as numeric zero.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_NEGATIVE_ZERO, "y": Decimal128("10")}, + {"_id": 2, "partition": "A", "x": Decimal128("10"), "y": Decimal128("20")}, + {"_id": 3, "partition": "A", "x": Decimal128("20"), "y": Decimal128("30")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[-0, 10, 20] -> mean_x=10, y=[10, 20, 30] -> mean_y=20 + # covPop = ((-10)(-10)+(0)(0)+(10)(10))/3 = (100+0+100)/3 = 200/3 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_NEGATIVE_ZERO, + "y": Decimal128("10"), + "result": Decimal128("66.66666666666666666666666666666667"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("10"), + "y": Decimal128("20"), + "result": Decimal128("66.66666666666666666666666666666667"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("20"), + "y": Decimal128("30"), + "result": Decimal128("66.66666666666666666666666666666667"), + }, + ] + assertSuccess(result, expected, msg="DECIMAL128_NEGATIVE_ZERO treated as numeric zero") + + +# Property [Negative Zero]: -0.0 treated as numeric zero + + +def test_covariancePop_negative_zero(collection): + """$covariancePop treats -0.0 as numeric zero — participates in computation.""" + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEGATIVE_ZERO, "y": 10}, + {"_id": 2, "partition": "A", "x": 10, "y": 20}, + {"_id": 3, "partition": "A", "x": 20, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[-0, 10, 20] -> mean_x=10, y=[10, 20, 30] -> mean_y=20 + # covPop = ((-10)(-10)+(0)(0)+(10)(10))/3 = (100+0+100)/3 = 200/3 = 66.6667 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DOUBLE_NEGATIVE_ZERO, + "y": 10, + "result": 66.66666666666667, + }, + {"_id": 2, "partition": "A", "x": 10, "y": 20, "result": 66.66666666666667}, + {"_id": 3, "partition": "A", "x": 20, "y": 30, "result": 66.66666666666667}, + ] + assertSuccess(result, expected, msg="-0.0 treated as numeric zero in covariancePop") + + +# Property [Basic Numeric]: standard numeric inputs handled correctly + + +def test_covariancePop_negative_numbers(collection): + """$covariancePop handles negative numbers correctly.""" + docs = [ + {"_id": 1, "partition": "A", "x": -10, "y": -20}, + {"_id": 2, "partition": "A", "x": 0, "y": 0}, + {"_id": 3, "partition": "A", "x": 10, "y": 20}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[-10,0,10] mean=0, y=[-20,0,20] mean=0 + # covPop = ((-10)(-20)+(0)(0)+(10)(20))/3 = (200+0+200)/3 = 400/3 = 133.3333 + expected = [ + {"_id": 1, "partition": "A", "x": -10, "y": -20, "result": 133.33333333333334}, + {"_id": 2, "partition": "A", "x": 0, "y": 0, "result": 133.33333333333334}, + {"_id": 3, "partition": "A", "x": 10, "y": 20, "result": 133.33333333333334}, + ] + assertSuccess(result, expected, msg="negative numbers handled correctly") + + +def test_covariancePop_decimals(collection): + """$covariancePop handles floating-point (double) values.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1.5, "y": 3.0}, + {"_id": 2, "partition": "A", "x": 2.5, "y": 5.0}, + {"_id": 3, "partition": "A", "x": 3.5, "y": 7.0}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1.5,2.5,3.5] mean=2.5, y=[3,5,7] mean=5 + # covPop = ((-1)(-2)+(0)(0)+(1)(2))/3 = 4/3 = 1.3333... + expected = [ + {"_id": 1, "partition": "A", "x": 1.5, "y": 3.0, "result": 1.3333333333333333}, + {"_id": 2, "partition": "A", "x": 2.5, "y": 5.0, "result": 1.3333333333333333}, + {"_id": 3, "partition": "A", "x": 3.5, "y": 7.0, "result": 1.3333333333333333}, + ] + assertSuccess(result, expected, msg="floating-point values handled correctly") + + +# --------------------------------------------------------------------------- +# Property [Intermediate Overflow]: TEST_COVERAGE.md §22 overflow requirements +# +# Covariance is computed by an online (Welford-style) update, so an +# *intermediate* value can overflow independently of whether the final result +# is representable. The four cases below are deliberately kept separate -- +# a single "large values" test cannot distinguish them, and each has a +# different failure mode: +# +# 1. No overflow in the formula -- identical large x, so every deviation is +# exactly 0. +# 2. Deviation overflow -- opposing magnitudes in one column, so +# x_i - mean_x overflows. The expected sign +# is asserted, not merely non-finiteness. +# 3. Product overflow only -- deviations stay finite, their product +# overflows. +# 4. Result exceeds range -- true result beyond the type maximum. +# +# A literal Infinity *input* exercises none of these: non-finite inputs are +# short-circuited before the online update runs. Those live in the +# special_floats file. +# +# All expectations below were verified against the reference server. +# --------------------------------------------------------------------------- + + +# --- Case 1: no overflow -- identical large values, deviations exactly 0 --- + + +def test_covariancePop_identical_decimal128_max_three_docs(collection): + """$covariancePop with 3 identical DECIMAL128_MAX x values -- deviations are exactly 0. + + Nothing in (1/n)*sum((x_i - mean_x)(y_i - mean_y)) overflows: every x_i is the + same value, so every x_i - mean_x is exactly 0 at any magnitude and the formula + never requires sum(x_i). The server reports -Infinity because it derives the mean + from a running sum that overflows; its sign follows the y ordering rather than + the data (see the y-reversed test below). + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("3")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="identical DECIMAL128_MAX x values: The server reports -Infinity from sum-derived mean", + ) + + +def test_covariancePop_identical_decimal128_max_y_reversed(collection): + """$covariancePop identical DECIMAL128_MAX x with descending y -- sign flips. + + Same x column as the previous test, same true answer (0), but reversing y + flips The server's reported infinity from -Infinity to +Infinity. The sign + tracks the y ordering, not the covariance, which is what identifies it as an + overflow artifact rather than a semantic. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("3")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="reversing y flips the reported infinity sign for identical x", + ) + + +def test_covariancePop_identical_decimal128_max_two_docs(collection): + """$covariancePop with 2 identical DECIMAL128_MAX x values -- no overflow yet. + + Count-dependence check. The true answer is 0 for any number of identical x + values, but The server's running sum only overflows once a third value is added, + so n=2 returns 0 while n=3 returns -Infinity. This is the baseline that makes + the n=3 result meaningful. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("2")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": Decimal128("0E+14")} for doc in docs] + assertSuccess( + result, + expected, + msg="two identical DECIMAL128_MAX x values do not overflow the running sum", + ) + + +def test_covariancePop_identical_decimal128_max_in_y(collection): + """$covariancePop with identical DECIMAL128_MAX in the y position. + + Mirror of the x-side test: the same artifact must be probed per expression + position, since the two arguments are accumulated separately. + """ + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": DECIMAL128_MAX}, + {"_id": 2, "partition": "A", "x": Decimal128("2"), "y": DECIMAL128_MAX}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": DECIMAL128_MAX}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="identical DECIMAL128_MAX y values: same artifact on the y side", + ) + + +def test_covariancePop_identical_double_near_max_in_y(collection): + """$covariancePop with identical DOUBLE_NEAR_MAX in y -- double path, y side. + + The double path shows the same count dependence as decimal: two identical + 1e308 x values return 0 (below), while the y-side sum here overflows. + """ + docs = [ + {"_id": 1, "partition": "A", "x": 1.0, "y": DOUBLE_NEAR_MAX}, + {"_id": 2, "partition": "A", "x": 2.0, "y": DOUBLE_NEAR_MAX}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="identical DOUBLE_NEAR_MAX y values overflow the running sum", + ) + + +def test_covariancePop_identical_double_near_max_two_docs(collection): + """$covariancePop with 2 identical DOUBLE_NEAR_MAX x values -- returns 0. + + Double-path baseline for case 1: x=[1e308, 1e308] does not overflow the + running sum, so the result is exactly 0. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": 1.0}, + {"_id": 2, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": 2.0}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": 0.0} for doc in docs] + assertSuccess( + result, + expected, + msg="two identical DOUBLE_NEAR_MAX x values give exactly 0", + ) + + +# --- Case 2: deviation overflow -- SIGN-critical --- + + +def test_covariancePop_deviation_overflow_positive_decimal128(collection): + """$covariancePop with x=y=[DECIMAL128_MAX, DECIMAL128_MIN] -- expects +Infinity. + + Opposing maximum magnitudes in the same column make x_i - mean_x overflow. + x and y move together, so the limit is +Infinity. The sign is asserted + rather than just non-finiteness, since -Infinity is also non-finite and + would satisfy a weaker assertion. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MIN, "y": DECIMAL128_MIN}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="positively-correlated deviation overflow gives +Infinity", + ) + + +def test_covariancePop_deviation_overflow_negative_decimal128(collection): + """$covariancePop anti-correlated DECIMAL128 extremes -- expects -Infinity. + + Same magnitudes as the previous test with y inverted, so the limit is + -Infinity. Paired with that test, this pins the sign to the direction of the + data rather than to the magnitudes. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MIN}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MIN, "y": DECIMAL128_MAX}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="anti-correlated deviation overflow gives -Infinity", + ) + + +def test_covariancePop_deviation_overflow_positive_double(collection): + """$covariancePop with x=y=[1e308, -1e308] -- double path, expects +Infinity. + + The deviation -1e308 - 1e308 overflows to -inf in the double path exactly as + it does in decimal, so the case is covered for both numeric types. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": DOUBLE_NEAR_MAX}, + {"_id": 2, "partition": "A", "x": -DOUBLE_NEAR_MAX, "y": -DOUBLE_NEAR_MAX}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="double-path positively-correlated deviation overflow gives +Infinity", + ) + + +def test_covariancePop_deviation_overflow_negative_double(collection): + """$covariancePop anti-correlated 1e308 extremes -- double path, expects -Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": -DOUBLE_NEAR_MAX}, + {"_id": 2, "partition": "A", "x": -DOUBLE_NEAR_MAX, "y": DOUBLE_NEAR_MAX}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="double-path anti-correlated deviation overflow gives -Infinity", + ) + + +def test_covariancePop_deviation_overflow_representable_result(collection): + """$covariancePop with x=[DECIMAL128_MAX, 0] -- large magnitude, exact result. + + Deviations reach half of DECIMAL128_MAX without overflowing, and the result + -2.5E+6144 is representable, so it is returned exactly. This establishes that + full-range magnitudes alone do not trigger the overflow cases above. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": Decimal128("0"), "y": Decimal128("2")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {**doc, "result": Decimal128("-2.500000000000000000000000000000000E+6144")} for doc in docs + ] + assertSuccess( + result, + expected, + msg="DECIMAL128_MAX-scale deviations with a representable result are exact", + ) + + +# --- Case 3: product overflow with finite deviations (control) --- + + +def test_covariancePop_product_overflow_finite_deviations_double(collection): + """$covariancePop with x=y=[1e200, -1e200] -- product overflows, deviations do not. + + Control for the case-2 sign tests. Here 1e200 - (-1e200) = 2e200 stays finite + and only the deviation *product* overflows, giving +Infinity. Keeping this + separate from case 2 distinguishes an overflow in the deviation step from one + in the product step. + """ + docs = [ + {"_id": 1, "partition": "A", "x": 1e200, "y": 1e200}, + {"_id": 2, "partition": "A", "x": -1e200, "y": -1e200}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="product overflow with finite deviations preserves the sign", + ) + + +def test_covariancePop_product_overflow_finite_deviations_negative(collection): + """$covariancePop anti-correlated 1e200 -- product overflows to -Infinity. + + Negative-direction half of the control pair. + """ + docs = [ + {"_id": 1, "partition": "A", "x": 1e200, "y": -1e200}, + {"_id": 2, "partition": "A", "x": -1e200, "y": 1e200}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="anti-correlated product overflow gives -Infinity", + ) + + +def test_covariancePop_product_overflow_finite_deviations_decimal128(collection): + """$covariancePop with x=y=[1E+3100, -1E+3100] -- decimal product overflow. + + Decimal128 half of the control: deviations reach 2E+3100 (well inside range) + while their product exceeds E+6144. + """ + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1E+3100"), "y": Decimal128("1E+3100")}, + {"_id": 2, "partition": "A", "x": Decimal128("-1E+3100"), "y": Decimal128("-1E+3100")}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="decimal product overflow with finite deviations preserves the sign", + ) + + +# --- Case 4: true result exceeds the type range --- + + +def test_covariancePop_result_exceeds_decimal128_range(collection): + """$covariancePop with x=y=DECIMAL128_MAX repeated -- server returns NaN. + + Identical values in both columns, so the formula gives 0, but The server's + sum-derived mean overflows in both accumulators and the indeterminate + Infinity - Infinity yields NaN. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": Decimal128("NaN")} for doc in docs] + assertSuccessNaN( + result, + expected, + msg="DECIMAL128_MAX in both columns overflows both means to NaN", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_order_independence.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_order_independence.py new file mode 100644 index 000000000..f77ec03db --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_order_independence.py @@ -0,0 +1,139 @@ +""" +Tests for $covariancePop order independence in window context. + +Verifies that $covariancePop produces the same result regardless of sortBy direction, +confirming it is an order-independent operator. Population covariance is a +symmetric statistic over the frame — it depends only on which documents are in +the frame, not on their processing order. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + COVAR_DOCS, + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +UNBOUNDED_WINDOW = {"documents": ["unbounded", "unbounded"]} + +# Property [Order Independence]: $covariancePop produces same result regardless of sort direction + + +def test_covariancePop_whole_partition_ascending_sort(collection): + """$covariancePop whole partition with ascending sort.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": 1}, + ) + # covPop of (x,y) where y=2x: covPop = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 4.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 4.0}, + ] + assertSuccess(result, expected, msg="ascending sort produces correct covariancePop") + + +def test_covariancePop_whole_partition_descending_sort(collection): + """$covariancePop whole partition with descending sort produces same result as ascending.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": -1}, + extra_stages=[{"$sort": {"_id": 1}}], + ) + # Same result regardless of sort direction — order-independent operator + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 4.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 4.0}, + ] + assertSuccess( + result, expected, msg="descending sort produces same covariancePop — order independent" + ) + + +def test_covariancePop_sort_by_value_vs_sort_by_id(collection): + """$covariancePop whole partition: sort by value field vs sort by _id gives same result.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"x": -1}, + extra_stages=[{"$sort": {"_id": 1}}], + ) + # Sorting by x descending should not affect whole-partition covariancePop + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 4.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 4.0}, + ] + assertSuccess( + result, + expected, + msg="sort by value field produces same result as sort by _id — order independent", + ) + + +NEGATIVE_COVAR_DOCS = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 8}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + {"_id": 4, "partition": "A", "x": 4, "y": 4}, + {"_id": 5, "partition": "A", "x": 5, "y": 2}, +] + +# x=[1,2,3,4,5] mean=3, y=[10,8,6,4,2] mean=6 +# covPop = ((-2)(4)+(-1)(2)+(0)(0)+(1)(-2)+(2)(-4))/5 = (-8-2+0-2-8)/5 = -20/5 = -4.0 +NEGATIVE_COVAR_EXPECTED = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": -4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 8, "result": -4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": -4.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 4, "result": -4.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 2, "result": -4.0}, +] + + +def test_covariancePop_negative_correlation_ascending_sort(collection): + """$covariancePop with negative correlation gives correct result with ascending sort.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=NEGATIVE_COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": 1}, + ) + assertSuccess( + result, NEGATIVE_COVAR_EXPECTED, msg="negative correlation ascending sort gives -4.0" + ) + + +def test_covariancePop_negative_correlation_descending_sort(collection): + """$covariancePop with negative correlation gives same result with descending sort.""" + result = run_window_operator( + collection, + "$covariancePop", + docs=NEGATIVE_COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": -1}, + extra_stages=[{"$sort": {"_id": 1}}], + ) + assertSuccess( + result, NEGATIVE_COVAR_EXPECTED, msg="negative correlation descending sort gives same -4.0" + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_special_floats.py b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_special_floats.py new file mode 100644 index 000000000..34229c7fc --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covariancePop/test_window_covariancePop_special_floats.py @@ -0,0 +1,532 @@ +""" +Tests for $covariancePop with special float values (NaN, Infinity, -Infinity). + +Covers: Infinity as numeric participant, -Infinity, NaN values, +sliding window behavior with special floats, and cumulative window behavior. + +$covariancePop semantics for special floats (verified against server 8.2.4): +- NaN and Infinity are numeric values (not ignored like null/missing) +- In non-removable windows (whole partition): Inf produces Infinity, -Inf produces + -Infinity, NaN produces NaN, Inf+(-Inf) cancels to 0.0, all-Inf produces null +- In cumulative windows (unbounded, current): single Inf = null, then propagates +- In sliding/removable windows: Inf/NaN in frame yields specific behavior + (not simply null) — Row with only special value returns null, pairs with Inf + can return 0.0, clean frames compute normally +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess, assertSuccessNaN +from documentdb_tests.framework.test_constants import ( + FLOAT_INFINITY, + FLOAT_NAN, + FLOAT_NEGATIVE_INFINITY, +) + +# Property [Infinity Non-Removable Window]: Infinity in non-removable whole partition windows + + +def test_covariancePop_positive_infinity_whole_partition(collection): + """$covariancePop with Infinity in x, whole partition returns Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 4, "result": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_INFINITY}, + ] + assertSuccess(result, expected, msg="Infinity in x produces Infinity for whole partition") + + +def test_covariancePop_positive_infinity_in_y(collection): + """$covariancePop with Infinity in second expression (y) produces Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_INFINITY, "result": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_INFINITY}, + ] + assertSuccess(result, expected, msg="Infinity in y produces Infinity in whole partition") + + +def test_covariancePop_negative_infinity_in_y(collection): + """$covariancePop with -Infinity in second expression (y) produces -Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_NEGATIVE_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + { + "_id": 1, + "partition": "A", + "x": 1, + "y": FLOAT_NEGATIVE_INFINITY, + "result": FLOAT_NEGATIVE_INFINITY, + }, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NEGATIVE_INFINITY}, + ] + assertSuccess(result, expected, msg="-Infinity in y produces -Infinity in whole partition") + + +def test_covariancePop_opposing_infinity_same_pair(collection): + """$covariancePop with +Inf and -Inf in the same (x, y) pair produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": FLOAT_NEGATIVE_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + { + "_id": 1, + "partition": "A", + "x": FLOAT_INFINITY, + "y": FLOAT_NEGATIVE_INFINITY, + "result": FLOAT_NAN, + }, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="opposing infinities in same pair produce NaN") + + +def test_covariancePop_both_inf_signs_separate_rows(collection): + """$covariancePop with +Inf and -Inf in separate rows, count_finite>=2, produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": FLOAT_NAN}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="both inf signs in separate rows with count_finite>=2 produce NaN" + ) + + +def test_covariancePop_mixed_inf_types_across_rows(collection): + """$covariancePop with opposing-sign pair + same-sign pair + finite: count_finite=1 -> 0.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": 5, "y": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": FLOAT_INFINITY, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 5, "y": FLOAT_NEGATIVE_INFINITY, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="mixed inf types with count_finite=1 returns 0.0 for Pop") + + +def test_covariancePop_opposing_inf_signs_across_columns(collection): + """$covariancePop with -Inf in x (one row) and +Inf in y (another row) produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": FLOAT_INFINITY}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="opposing inf signs across x and y columns produce NaN") + + +def test_covariancePop_negative_infinity_whole_partition(collection): + """$covariancePop with -Infinity in x, whole partition returns -Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_NEGATIVE_INFINITY}, + { + "_id": 2, + "partition": "A", + "x": FLOAT_NEGATIVE_INFINITY, + "y": 4, + "result": FLOAT_NEGATIVE_INFINITY, + }, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NEGATIVE_INFINITY}, + ] + assertSuccess(result, expected, msg="-Infinity in x produces -Infinity for whole partition") + + +def test_covariancePop_inf_and_neg_inf_in_same_frame(collection): + """$covariancePop with both Infinity and -Infinity in x cancels to 0.0.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": 10, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 10, "y": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="Inf + -Inf in same frame cancels to 0.0") + + +def test_covariancePop_all_infinity_values(collection): + """$covariancePop where all x values are Infinity returns null (Inf-Inf=NaN internally).""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30, "result": None}, + ] + assertSuccess( + result, expected, msg="All Inf x values: Inf-Inf=NaN internally, server returns null" + ) + + +def test_covariancePop_infinity_cumulative_window(collection): + """$covariancePop cumulative [unbounded, current] with Infinity in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "current"]}, + expression=["$x", "$y"], + ) + # Row 1: single Inf value -> null + # Row 2: frame=[(Inf,10),(2,20)] -> 0.0 + # Row 3: frame=[(Inf,10),(2,20),(3,30)] -> Infinity + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": FLOAT_INFINITY}, + ] + assertSuccess( + result, expected, msg="Cumulative: single Inf=null, 2 values=0.0, 3 values=Infinity" + ) + + +def test_covariancePop_single_infinity_value(collection): + """$covariancePop with single Infinity value in whole partition returns null.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + ] + assertSuccess(result, expected, msg="Single Inf value: covariancePop returns null") + + +# Property [NaN Non-Removable Window]: NaN in non-removable windows produces NaN + + +def test_covariancePop_nan_value_whole_partition(collection): + """$covariancePop with NaN in non-removable window produces NaN (NaN is numeric, poisons).""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": FLOAT_NAN, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": FLOAT_NAN, "y": 4, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="NaN is numeric; non-removable window produces NaN") + + +def test_covariancePop_nan_in_y_whole_partition(collection): + """$covariancePop with NaN in second expression (y) produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": 2, "y": FLOAT_NAN, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="NaN in y expression poisons non-removable window to NaN" + ) + + +# Property [Special Floats Sliding Window]: special floats in removable/sliding windows + + +def test_covariancePop_infinity_sliding(collection): + """$covariancePop sliding window [-1,0] with Infinity in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(Inf,10)] -> null (single Inf pair) + # Row 2: frame=[(Inf,10),(2,20)] -> 0.0 + # Row 3: frame=[(2,20),(3,30)] -> covPop = 2.5 + # Row 4: frame=[(3,30),(4,40)] -> covPop = 2.5 + # Row 5: frame=[(4,40),(5,50)] -> covPop = 2.5 + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 2.5}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 2.5}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 2.5}, + ] + assertSuccess( + result, expected, msg="Sliding window: null for single Inf, 0.0 for Inf pair, then recovers" + ) + + +def test_covariancePop_neg_infinity_sliding(collection): + """$covariancePop sliding window [-1,0] with -Infinity in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(-Inf,10)] -> null (single -Inf pair) + # Row 2: frame=[(-Inf,10),(2,20)] -> 0.0 + # Row 3: frame=[(2,20),(3,30)] -> covPop = 2.5 + # Row 4: frame=[(3,30),(4,40)] -> covPop = 2.5 + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 2.5}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 2.5}, + ] + assertSuccess( + result, + expected, + msg="Sliding window: null for single -Inf, 0.0 for -Inf pair, then recovers", + ) + + +def test_covariancePop_nan_sliding(collection): + """$covariancePop sliding window [-1,0] with NaN in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_NAN, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(NaN,10)] -> null (single NaN pair) + # Row 2: frame=[(NaN,10),(2,20)] -> 0.0 + # Row 3: frame=[(2,20),(3,30)] -> covPop = 2.5 + # Row 4: frame=[(3,30),(4,40)] -> covPop = 2.5 + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_NAN, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 2.5}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 2.5}, + ] + assertSuccessNaN( + result, expected, msg="Sliding window: null for single NaN, 0.0 for NaN pair, then recovers" + ) + + +def test_covariancePop_infinity_centered_sliding(collection): + """$covariancePop centered sliding window [-1, 1] with Infinity in middle.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 1]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(1,10),(2,20)] -> covPop = 2.5 + # Row 2: frame=[(1,10),(2,20),(Inf,30)] -> Infinity (Inf propagates in 3-elem frame) + # Row 3: frame=[(2,20),(Inf,30),(4,40)] -> Infinity + # Row 4: frame=[(Inf,30),(4,40),(5,50)] -> Infinity + # Row 5: frame=[(4,40),(5,50)] -> covPop = 2.5 (with possible FP rounding) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": 2.5}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30, "result": FLOAT_INFINITY}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": FLOAT_INFINITY}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 2.5000000000000107}, + ] + assertSuccess( + result, + expected, + msg="Centered sliding: Inf propagates in 3-elem frames, clean 2-elem frames = 2.5", + ) + + +def test_covariancePop_nan_in_y_sliding(collection): + """$covariancePop sliding window [-1,0] with NaN in y of first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covariancePop", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(1,NaN)] -> null (single pair with NaN) + # Row 2: frame=[(1,NaN),(2,20)] -> 0.0 + # Row 3: frame=[(2,20),(3,30)] -> covPop = 2.5 + # Row 4: frame=[(3,30),(4,40)] -> covPop = 2.5 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_NAN, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 2.5}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 2.5}, + ] + assertSuccessNaN( + result, + expected, + msg="Sliding window: null for single NaN-y, 0.0 for NaN-y pair, then recovers", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_argument_validation.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_argument_validation.py new file mode 100644 index 000000000..10e5e8328 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_argument_validation.py @@ -0,0 +1,422 @@ +""" +Tests for $covarianceSamp argument validation in window context. + +Covers: valid expression forms (array of two field paths, operator expressions), +invalid argument shapes that produce null results (not an array, wrong length, +single expression, three expressions, empty array, object expression), +and structural errors (unknown keys in output field spec, multiple accumulators). + +Server behavior (verified): $covarianceSamp does NOT reject bad argument +shapes at parse time. Invalid forms (single expression, wrong-length arrays, +objects, empty arrays) all succeed and return null for every document. +Only structural $setWindowFields errors (unknown keys, multiple accumulators, +no accumulator, bad field paths, unrecognized operators) produce parse errors. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertFailureCode, assertSuccess +from documentdb_tests.framework.error_codes import ( + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + FAILED_TO_PARSE_ERROR, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + UNRECOGNIZED_EXPRESSION_ERROR, +) +from documentdb_tests.framework.executor import execute_command + +TWO_DOCS = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, +] + +SINGLE_DOC = [{"_id": 1, "partition": "A", "x": 1, "y": 2}] + +# Property [Valid Expression Forms]: accepted expression inputs + + +def test_covarianceSamp_two_field_paths(collection): + """$covarianceSamp accepts array of two field path expressions.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=TWO_DOCS, + expression=["$x", "$y"], + window={"documents": ["unbounded", "unbounded"]}, + ) + # x=[1,2], y=[2,4]: mean_x=1.5, mean_y=3 + # covSamp = ((-0.5)(-1)+(0.5)(1))/1 = 1/1 = 1.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 1.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 1.0}, + ] + assertSuccess(result, expected, msg="two field path expressions accepted") + + +def test_covarianceSamp_operator_expressions(collection): + """$covarianceSamp accepts operator expressions within the array.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs=docs, + expression=[{"$multiply": ["$x", 2]}, {"$multiply": ["$y", 2]}], + window={"documents": ["unbounded", "unbounded"]}, + ) + # x*2=[2,4], y*2=[4,8]: mean_x=3, mean_y=6 + # covSamp = ((-1)(-2)+(1)(2))/1 = 4/1 = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 4.0}, + ] + assertSuccess(result, expected, msg="operator expressions within array accepted") + + +def test_covarianceSamp_same_field_both_positions(collection): + """$covarianceSamp with same field in both positions equals varSamp.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10}, + {"_id": 2, "partition": "A", "x": 20}, + {"_id": 3, "partition": "A", "x": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs=docs, + expression=["$x", "$x"], + window={"documents": ["unbounded", "unbounded"]}, + ) + # covSamp(x,x) = varSamp(x) = 200/2 = 100.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "result": 100.0}, + {"_id": 2, "partition": "A", "x": 20, "result": 100.0}, + {"_id": 3, "partition": "A", "x": 30, "result": 100.0}, + ] + assertSuccess(result, expected, msg="same field both positions equals varSamp") + + +def test_covarianceSamp_literal_numeric_expressions(collection): + """$covarianceSamp with literal numeric values — constant values produce covSamp=0.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10}, + {"_id": 2, "partition": "A", "x": 20}, + {"_id": 3, "partition": "A", "x": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs=docs, + expression=[{"$literal": 5}, {"$literal": 10}], + window={"documents": ["unbounded", "unbounded"]}, + ) + # All rows have same pair (5,10) -> covSamp = 0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="literal numeric expressions produce 0 covSamp") + + +# Property [Invalid Argument Shapes Return Null]: Server does NOT reject these at parse time + + +def test_covarianceSamp_single_expression_not_array_returns_null(collection): + """$covarianceSamp with single field path (not array) returns null for all docs.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=TWO_DOCS, + expression="$x", + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="single field path (not array) returns null") + + +def test_covarianceSamp_single_element_array_returns_null(collection): + """$covarianceSamp with array of only 1 expression returns null for all docs.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=TWO_DOCS, + expression=["$x"], + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="array of 1 expression returns null") + + +def test_covarianceSamp_three_element_array_returns_null(collection): + """$covarianceSamp with array of 3 expressions returns null for all docs.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=TWO_DOCS, + expression=["$x", "$y", "$x"], + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="array of 3 expressions returns null") + + +def test_covarianceSamp_empty_array_returns_null(collection): + """$covarianceSamp with empty array [] returns null for all docs.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=TWO_DOCS, + expression=[], + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="empty array returns null") + + +def test_covarianceSamp_object_expression_returns_null(collection): + """$covarianceSamp with object (not array) as argument returns null for all docs.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=TWO_DOCS, + expression={"$add": ["$x", 1]}, + window={"documents": ["unbounded", "unbounded"]}, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + ] + assertSuccess(result, expected, msg="object (not array) as argument returns null") + + +# Property [Structural Parse Errors]: errors in $setWindowFields output spec structure + + +def test_covarianceSamp_unknown_key_in_output_field_errors(collection): + """Unknown key alongside $covarianceSamp in output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, FAILED_TO_PARSE_ERROR, msg="unknown key alongside $covarianceSamp rejected" + ) + + +def test_covarianceSamp_unknown_key_errors_on_empty_collection(collection): + """Parse-time error fires on empty collection — no documents needed.""" + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="parse-time error fires on empty collection", + ) + + +def test_covarianceSamp_multiple_accumulators_in_output_field_errors(collection): + """Multiple accumulators in same output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": ["$x", "$y"], + "$sum": "$x", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="multiple accumulators in output field rejected", + ) + + +def test_covarianceSamp_no_accumulator_in_output_field_errors(collection): + """Output field with no accumulator (only window key) produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="no accumulator in output field rejected") + + +def test_covarianceSamp_unrecognized_expression_operator_in_array_errors(collection): + """$covarianceSamp with unrecognized expression operator in array element produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": [{"$unknownOp": "$x"}, "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + UNRECOGNIZED_EXPRESSION_ERROR, + msg="unrecognized expression operator in array rejected", + ) + + +def test_covarianceSamp_field_path_empty_component_errors(collection): + """$covarianceSamp with field path containing empty component produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": ["$a..b", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + msg="field path with empty component rejected", + ) + + +def test_covarianceSamp_multi_field_expression_object_errors(collection): + """$covarianceSamp with multi-field expression object in array element produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": [ + {"$add": ["$x", 1], "$multiply": ["$x", 2]}, + "$y", + ], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + msg="multi-field expression object in array element rejected", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_field_paths.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_field_paths.py new file mode 100644 index 000000000..ed3bd6763 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_field_paths.py @@ -0,0 +1,380 @@ +""" +Tests for $covarianceSamp with nested field paths, array field traversal, +expressions that return different types per document, and $project +removing fields before $setWindowFields. + +Covers: dotted field paths, missing intermediate paths, array index access, +array-of-objects traversal, top-level array fields, expressions returning +mixed types per row, and pipeline stages removing expression fields. + +$covarianceSamp takes two expressions: ["$x", "$y"]. These tests exercise +various field path forms for both expressions. + +Key difference from $covariancePop: single valid pair returns null (N-1=0), +and computed values use N-1 divisor. +""" + +from datetime import datetime, timezone + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +# Property [Dotted Field Path]: +# Tests that $covarianceSamp correctly accesses nested document values via dotted paths. + + +def test_covarianceSamp_dotted_field_path(collection): + """$covarianceSamp with dotted field path accesses nested document value.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"x": 1, "y": 2}}}, + {"_id": 2, "partition": "A", "data": {"metrics": {"x": 2, "y": 4}}}, + {"_id": 3, "partition": "A", "data": {"metrics": {"x": 3, "y": 6}}}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.metrics.x", "$data.metrics.y"], + ) + # x=[1,2,3], y=[2,4,6] -> covSamp = 4/2 = 2.0 + expected = [ + { + "_id": 1, + "partition": "A", + "data": {"metrics": {"x": 1, "y": 2}}, + "result": 2.0, + }, + { + "_id": 2, + "partition": "A", + "data": {"metrics": {"x": 2, "y": 4}}, + "result": 2.0, + }, + { + "_id": 3, + "partition": "A", + "data": {"metrics": {"x": 3, "y": 6}}, + "result": 2.0, + }, + ] + assertSuccess( + result, expected, msg="dotted field path accesses nested value for both expressions" + ) + + +# Property [Missing Intermediate Path]: +# Tests that missing intermediate paths are treated as missing (row ignored). + + +def test_covarianceSamp_missing_intermediate_path_x(collection): + """$covarianceSamp with missing intermediate path in first expression (x) — row ignored.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2}, + {"_id": 2, "partition": "A", "y": 4}, # missing data.x entirely + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.x", "$y"], + ) + # Doc 2 has missing x -> ignored. Only pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covSamp = ((-1)(-2)+(1)(2))/1 = 4/1 = 4.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="missing intermediate path in x -> row ignored") + + +def test_covarianceSamp_missing_intermediate_path_y(collection): + """$covarianceSamp with missing intermediate path in second expression (y) — row ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "data": {"y": 2}}, + {"_id": 2, "partition": "A", "x": 2, "data": {"other": 99}}, # missing data.y + {"_id": 3, "partition": "A", "x": 3, "data": {"y": 6}}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$data.y"], + ) + # Doc 2 has missing y -> ignored. Only pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covSamp = ((-1)(-2)+(1)(2))/1 = 4/1 = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "data": {"y": 2}, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "data": {"other": 99}, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "data": {"y": 6}, "result": 4.0}, + ] + assertSuccess(result, expected, msg="missing intermediate path in y -> row ignored") + + +def test_covarianceSamp_top_level_missing_object(collection): + """$covarianceSamp where the top-level field of a dotted path is missing.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"x": 1, "y": 2}}, + {"_id": 2, "partition": "A"}, # missing 'data' entirely + {"_id": 3, "partition": "A", "data": {"x": 3, "y": 6}}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.x", "$data.y"], + ) + # Doc 2 missing both x and y -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covSamp = 4.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"x": 1, "y": 2}, "result": 4.0}, + {"_id": 2, "partition": "A", "result": 4.0}, + {"_id": 3, "partition": "A", "data": {"x": 3, "y": 6}, "result": 4.0}, + ] + assertSuccess(result, expected, msg="top-level field missing in dotted path = row ignored") + + +# Property [Null Value in Nested Path]: +# Tests that a field existing with null value through a dotted path is ignored. + + +def test_covarianceSamp_nested_field_explicit_null(collection): + """$covarianceSamp with nested field that exists but is null — row ignored.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2}, + {"_id": 2, "partition": "A", "data": {"x": None}, "y": 4}, # x exists but null + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$data.x", "$y"], + ) + # Doc 2 has explicit null in data.x -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covSamp = ((-1)(-2)+(1)(2))/1 = 4/1 = 4.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"x": 1}, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "data": {"x": None}, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "data": {"x": 3}, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="nested field with explicit null = row ignored") + + +# Property [Array Field Non-Numeric]: +# Tests that top-level array values are treated as non-numeric and ignored. + + +def test_covarianceSamp_array_field_is_non_numeric(collection): + """$covarianceSamp on a top-level array field — arrays are non-numeric, should be ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": [1, 2, 3], "y": 10}, + {"_id": 2, "partition": "A", "x": 5, "y": 20}, + {"_id": 3, "partition": "A", "x": [4, 5], "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Docs 1 and 3 have array x -> non-numeric -> ignored + # Only doc 2 has numeric x. Single numeric pair -> covSamp = null (N-1=0) + expected = [ + {"_id": 1, "partition": "A", "x": [1, 2, 3], "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 5, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": [4, 5], "y": 30, "result": None}, + ] + assertSuccess( + result, + expected, + msg="array field values are non-numeric — ignored, single pair returns null", + ) + + +# Property [Array at Intermediate Path Level]: +# Tests that a dotted path traversing through an array-of-objects resolves to an +# array (non-numeric) and is ignored. + + +def test_covarianceSamp_array_of_objects_traversal(collection): + """$covarianceSamp with path traversing array-of-objects — resolves to array, ignored.""" + docs = [ + {"_id": 1, "partition": "A", "items": [{"value": 10}, {"value": 20}], "y": 100}, + {"_id": 2, "partition": "A", "items": {"value": 5}, "y": 200}, + {"_id": 3, "partition": "A", "items": [{"value": 30}, {"value": 40}], "y": 300}, + {"_id": 4, "partition": "A", "items": {"value": 15}, "y": 400}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$items.value", "$y"], + ) + # Docs 1,3: items is array-of-objects -> $items.value resolves to + # [10,20]/[30,40] (array) -> ignored + # Docs 2,4: items is plain object -> $items.value resolves to 5/15 (scalar) -> participates + # Valid pairs: (5, 200) and (15, 400) + # mean_x=10, mean_y=300, covSamp = ((-5)(-100)+(5)(100))/1 = 1000/1 = 1000.0 + expected = [ + { + "_id": 1, + "partition": "A", + "items": [{"value": 10}, {"value": 20}], + "y": 100, + "result": 1000.0, + }, + {"_id": 2, "partition": "A", "items": {"value": 5}, "y": 200, "result": 1000.0}, + { + "_id": 3, + "partition": "A", + "items": [{"value": 30}, {"value": 40}], + "y": 300, + "result": 1000.0, + }, + {"_id": 4, "partition": "A", "items": {"value": 15}, "y": 400, "result": 1000.0}, + ] + assertSuccess(result, expected, msg="path through array-of-objects resolves to array — ignored") + + +# Property [Expression Returns Mixed Types]: +# Tests that non-numeric expression results are ignored in the computation. + + +def test_covarianceSamp_expression_returns_different_types(collection): + """$covarianceSamp expression returning different types per row — non-numeric ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20}, + {"_id": 2, "partition": "A", "x": -5, "y": 40}, + {"_id": 3, "partition": "A", "x": 30, "y": 60}, + {"_id": 4, "partition": "A", "x": -1, "y": 80}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=[{"$cond": [{"$gt": ["$x", 0]}, "$x", "not_a_number"]}, "$y"], + ) + # $cond on x returns: 10, "not_a_number", 30, "not_a_number" + # Only rows 1 and 3 have numeric first expr: pairs (10,20) and (30,60) + # mean_x=20, mean_y=40, covSamp = ((-10)(-20)+(10)(20))/1 = (200+200)/1 = 400.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "result": 400.0}, + {"_id": 2, "partition": "A", "x": -5, "y": 40, "result": 400.0}, + {"_id": 3, "partition": "A", "x": 30, "y": 60, "result": 400.0}, + {"_id": 4, "partition": "A", "x": -1, "y": 80, "result": 400.0}, + ] + assertSuccess(result, expected, msg="expression returning mixed types — non-numeric ignored") + + +# Property [Expression Returns Null]: +# Tests that null expression results are ignored in the computation. + + +def test_covarianceSamp_expression_returns_null_for_some(collection): + """$covarianceSamp expression returning null for some docs, number for others.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "factor": 2}, + {"_id": 2, "partition": "A", "x": 20, "y": 40, "factor": None}, + {"_id": 3, "partition": "A", "x": 30, "y": 60, "factor": 2}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=[{"$multiply": ["$x", "$factor"]}, "$y"], + ) + # $multiply on x: [10*2=20, 20*null=null, 30*2=60] + # Row 2 produces null x -> ignored. Pairs: (20,20) and (60,60) + # mean_x=40, mean_y=40, covSamp = ((-20)(-20)+(20)(20))/1 = (400+400)/1 = 800.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "factor": 2, "result": 800.0}, + {"_id": 2, "partition": "A", "x": 20, "y": 40, "factor": None, "result": 800.0}, + {"_id": 3, "partition": "A", "x": 30, "y": 60, "factor": 2, "result": 800.0}, + ] + assertSuccess(result, expected, msg="expression returning null for some — null results ignored") + + +# Property [Date Value as Expression]: +# Tests that Date values in expression field are non-numeric and ignored. + + +def test_covarianceSamp_date_value_as_expression_ignored(collection): + """$covarianceSamp with Date value in one expression field — non-numeric, ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": datetime(2023, 1, 1, tzinfo=timezone.utc), "y": 10}, + {"_id": 2, "partition": "A", "x": 20, "y": 40}, + {"_id": 3, "partition": "A", "x": datetime(2023, 6, 1, tzinfo=timezone.utc), "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Docs 1 and 3 have Date in x -> non-numeric -> ignored + # Only doc 2 is valid. Single pair -> covSamp = null (N-1=0) + expected = [ + { + "_id": 1, + "partition": "A", + "x": datetime(2023, 1, 1, tzinfo=timezone.utc), + "y": 10, + "result": None, + }, + {"_id": 2, "partition": "A", "x": 20, "y": 40, "result": None}, + { + "_id": 3, + "partition": "A", + "x": datetime(2023, 6, 1, tzinfo=timezone.utc), + "y": 30, + "result": None, + }, + ] + assertSuccess( + result, + expected, + msg="Date values in expression field are non-numeric — ignored, single pair returns null", + ) + + +# Property [Numeric Path Component]: +# Tests that numeric path components access array elements or object keys. + + +def test_covarianceSamp_numeric_path_component(collection): + """$covarianceSamp with numeric path component accesses array element or object key.""" + docs = [ + {"_id": 1, "partition": "A", "arr": [{"x": 10, "y": 20}]}, + {"_id": 2, "partition": "A", "arr": [{"x": 30, "y": 40}]}, + {"_id": 3, "partition": "A", "arr": [{"x": 50, "y": 60}]}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$arr.0.x", "$arr.0.y"], + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # In $setWindowFields context, $arr.0.x does not resolve to array element — + # the path returns non-numeric (array) values which are ignored, resulting in null. + expected = [ + {"_id": 1, "result": None}, + {"_id": 2, "result": None}, + {"_id": 3, "result": None}, + ] + assertSuccess(result, expected, msg="numeric path component in window context returns null") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_frame_computation.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_frame_computation.py new file mode 100644 index 000000000..91b60ccf6 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_frame_computation.py @@ -0,0 +1,260 @@ +""" +Tests for $covarianceSamp computation under documents-mode window frame shapes. + +Verifies the operator computes correct results given the 4 defined frame shapes: +whole-partition, cumulative, reverse-cumulative, and sliding. + +$covarianceSamp semantics: +- Takes array of exactly 2 expressions: ["$x", "$y"] +- Sample covariance = sum((xi - mean_x)(yi - mean_y)) / (N - 1) +- Single value (N=1) -> covarianceSamp = null (N-1=0, undefined) +- Empty window -> null + +Note: Stage-level frame boundary tests (under stages/setWindowFields/) verify +that the correct documents are selected into the frame (centered, trailing, +leading, non-overlapping, edge cases). These per-operator tests verify the +operator produces correct values given those documents. +""" + +import pytest + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + COVAR_DOCS, + WindowTestCase, + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.parametrize import pytest_params + +# COVAR_DOCS: x = [1,2,3,4,5], y = [2,4,6,8,10] (y = 2x) +# mean_x = 3, mean_y = 6 +# covSamp = sum((xi-3)(yi-6))/4 = (8+2+0+2+8)/4 = 20/4 = 5.0 + + +COVARIANCESAMP_DOCUMENTS_FRAME_TESTS: list[WindowTestCase] = [ + # Property [Whole Partition]: unbounded-unbounded frame covers entire partition + # covSamp(x, y) for all 5 docs = 5.0 (calculated above) + WindowTestCase( + "whole_partition", + docs=COVAR_DOCS, + window={"documents": ["unbounded", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 5.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 5.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 5.0}, + ], + msg="whole partition covarianceSamp should be 5.0", + ), + # Property [Cumulative Frame]: expanding frame from start to current + # Row 1 (n=1): [(1,2)] -> covSamp = null (N-1=0, undefined) + # Row 2 (n=2): [(1,2),(2,4)] -> mean_x=1.5, mean_y=3 + # covSamp = ((-0.5)(-1)+(0.5)(1))/1 = (0.5+0.5)/1 = 1.0 + # Row 3 (n=3): [(1,2),(2,4),(3,6)] -> mean_x=2, mean_y=4 + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = (2+0+2)/2 = 4/2 = 2.0 + # Row 4 (n=4): [(1,2),(2,4),(3,6),(4,8)] -> mean_x=2.5, mean_y=5 + # covSamp = ((-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3))/3 + # = (4.5+0.5+0.5+4.5)/3 = 10/3 = 3.3333... + # Row 5 (n=5): all docs -> covSamp = 5.0 + WindowTestCase( + "cumulative", + docs=COVAR_DOCS, + window={"documents": ["unbounded", "current"]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 1.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 3.3333333333333335}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 5.0}, + ], + msg="cumulative covarianceSamp should grow", + ), + # Property [Reverse Cumulative Frame]: shrinking frame from current to end + # Row 1 (n=5): all docs -> covSamp = 5.0 + # Row 2 (n=4): [(2,4),(3,6),(4,8),(5,10)] -> mean_x=3.5, mean_y=7 + # covSamp = ((-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3))/3 = 10/3 = 3.3333... + # Row 3 (n=3): [(3,6),(4,8),(5,10)] -> mean_x=4, mean_y=8 + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = 4/2 = 2.0 + # Row 4 (n=2): [(4,8),(5,10)] -> mean_x=4.5, mean_y=9 + # covSamp = ((-0.5)(-1)+(0.5)(1))/1 = 1/1 = 1.0 + # Row 5 (n=1): [(5,10)] -> covSamp = null + WindowTestCase( + "reverse_cumulative", + docs=COVAR_DOCS, + window={"documents": ["current", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 3.3333333333333335}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 1.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": None}, + ], + msg="reverse-cumulative covarianceSamp should shrink", + ), + # Property [Sliding Frame]: fixed-size window that moves with current row + # Window [-1, 1] = 3-doc centered (clamped at edges) + # Row 1: [(1,2),(2,4)] (edge clamp, n=2) -> mean_x=1.5, mean_y=3 + # covSamp = ((-0.5)(-1)+(0.5)(1))/1 = 1.0 + # Row 2: [(1,2),(2,4),(3,6)] (n=3) -> mean_x=2, mean_y=4 + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = 4/2 = 2.0 + # Row 3: [(2,4),(3,6),(4,8)] (n=3) -> mean_x=3, mean_y=6 + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = 4/2 = 2.0 + # Row 4: [(3,6),(4,8),(5,10)] (n=3) -> mean_x=4, mean_y=8 + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = 4/2 = 2.0 + # Row 5: [(4,8),(5,10)] (edge clamp, n=2) -> mean_x=4.5, mean_y=9 + # covSamp = ((-0.5)(-1)+(0.5)(1))/1 = 1.0 + WindowTestCase( + "sliding_centered", + docs=COVAR_DOCS, + window={"documents": [-1, 1]}, + expected=[ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 1.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 2.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 1.0}, + ], + msg="centered sliding window [-1,1]", + ), +] + + +@pytest.mark.parametrize("test", pytest_params(COVARIANCESAMP_DOCUMENTS_FRAME_TESTS)) +def test_covarianceSamp_documents_frames(collection, test): + """$covarianceSamp with various documents-mode window frames.""" + result = run_window_operator( + collection, + "$covarianceSamp", + test.docs, + test.window, + sort_by=test.sort_by, + expression=["$x", "$y"], + ) + assertSuccess(result, test.expected, msg=test.msg) + + +def test_covarianceSamp_negative_correlation(collection): + """$covarianceSamp with negative correlation (y decreases as x increases).""" + # x = [1, 2, 3], y = [6, 4, 2] -> y = -2x + 8 + # mean_x=2, mean_y=4 + # covSamp = ((-1)(2)+(0)(0)+(1)(-2))/2 = (-2+0-2)/2 = -4/2 = -2.0 + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 6}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 2}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 6, "result": -2.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": -2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 2, "result": -2.0}, + ] + assertSuccess(result, expected, msg="negative correlation produces negative covarianceSamp") + + +def test_covarianceSamp_zero_covariance(collection): + """$covarianceSamp returns 0 when variables are uncorrelated.""" + # x = [1, 2, 3], y = [5, 5, 5] -> y is constant + # mean_x=2, mean_y=5 + # covSamp = ((-1)(0)+(0)(0)+(1)(0))/2 = 0 + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 5}, + {"_id": 2, "partition": "A", "x": 2, "y": 5}, + {"_id": 3, "partition": "A", "x": 3, "y": 5}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 5, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 5, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 5, "result": 0.0}, + ] + assertSuccess(result, expected, msg="constant y produces zero covariance") + + +def test_covarianceSamp_identical_x_and_y(collection): + """$covarianceSamp where x == y reduces to sample variance.""" + # When x==y: covSamp(x,x) = varSamp(x) + # x = [10, 20, 30] -> mean=20, varSamp = (100+0+100)/2 = 200/2 = 100.0 + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 10}, + {"_id": 2, "partition": "A", "x": 20, "y": 20}, + {"_id": 3, "partition": "A", "x": 30, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # covSamp(x,x) = varSamp(x) = 200/2 = 100.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 10, "result": 100.0}, + {"_id": 2, "partition": "A", "x": 20, "y": 20, "result": 100.0}, + {"_id": 3, "partition": "A", "x": 30, "y": 30, "result": 100.0}, + ] + assertSuccess(result, expected, msg="covSamp(x,x) equals varSamp(x)") + + +def test_covarianceSamp_trailing_sliding_window(collection): + """$covarianceSamp with trailing sliding window [-1, 0].""" + # Window [-1, 0] = look-back 1 row + current + # Row 1: [(1,2)] (only current, edge) -> n=1 -> covSamp = null + # Row 2: [(1,2),(2,4)] (n=2) -> covSamp = 1/1 = 1.0 + # Row 3: [(2,4),(3,6)] (n=2) -> mean_x=2.5, mean_y=5 + # covSamp = ((-0.5)(-1)+(0.5)(1))/1 = 1.0 + # Row 4: [(3,6),(4,8)] (n=2) -> covSamp = 1.0 + # Row 5: [(4,8),(5,10)] (n=2) -> covSamp = 1.0 + docs = COVAR_DOCS + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 1.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 1.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 1.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 1.0}, + ] + assertSuccess(result, expected, msg="trailing sliding window [-1, 0]") + + +def test_covarianceSamp_empty_window_returns_null(collection): + """$covarianceSamp returns null when the window frame contains zero documents.""" + # Window [5, 10] on a 3-doc partition: for all rows, no documents fall in the frame + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [5, 10]}, + expression=["$x", "$y"], + ) + # Frame [5, 10] means offset +5 to +10 from current row — no such rows exist + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": None}, + ] + assertSuccess(result, expected, msg="empty window frame returns null") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_non_numeric_handling.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_non_numeric_handling.py new file mode 100644 index 000000000..fe397c47e --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_non_numeric_handling.py @@ -0,0 +1,479 @@ +""" +Tests for $covarianceSamp null, missing, and non-numeric value handling. + +Covers: null values, missing fields, strings, booleans, arrays, objects, +ObjectId, Regex, Binary, Timestamp, MinKey, MaxKey, mixed numeric and +non-numeric in same frame, and all non-numeric returns null. + +$covarianceSamp semantics: when either expression in ["$x", "$y"] evaluates to +a non-numeric, null, or missing value for a document, that entire row (pair) +is ignored in the covariance computation. Additionally, when only one valid +numeric pair exists in the frame, $covarianceSamp returns null (N-1=0). +""" + +from datetime import datetime, timezone + +from bson import Binary, MaxKey, MinKey, ObjectId, Regex, Timestamp + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +# Property [Null and Missing]: null and missing field values cause the row to be ignored + + +def test_covarianceSamp_null_in_x_ignored(collection): + """$covarianceSamp ignores rows where x expression is null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": None, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has null x -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covSamp = ((-1)(-2)+(1)(2))/1 = 4/1 = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": None, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="null x values ignored, covSamp of (1,2),(3,6) = 4.0") + + +def test_covarianceSamp_null_in_y_ignored(collection): + """$covarianceSamp ignores rows where y expression is null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has null y -> ignored. Pairs: (1,2) and (3,6) + # mean_x=2, mean_y=4, covSamp = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": None, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="null y values ignored") + + +def test_covarianceSamp_missing_x_field_ignored(collection): + """$covarianceSamp ignores documents where the x field is missing.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "y": 4}, # x missing + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has missing x -> ignored. Pairs: (1,2) and (3,6) -> covSamp = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="missing x field ignored") + + +def test_covarianceSamp_missing_y_field_ignored(collection): + """$covarianceSamp ignores documents where the y field is missing.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2}, # y missing + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Row 2 has missing y -> ignored. Pairs: (1,2) and (3,6) -> covSamp = 4.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="missing y field ignored") + + +def test_covarianceSamp_both_null_ignored(collection): + """$covarianceSamp ignores rows where both x and y are null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": None, "y": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": None, "y": None, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="both null values -> row ignored") + + +# Property [Non-Numeric Types Ignored]: string, boolean, array, object, date, +# ObjectId, Regex, Binary values are ignored + + +def test_covarianceSamp_string_in_x_ignored(collection): + """$covarianceSamp ignores rows where x is a string.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": "hello", "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": "hello", "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="string x values ignored") + + +def test_covarianceSamp_string_in_y_ignored(collection): + """$covarianceSamp ignores rows where y is a string.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": "world"}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": 2, "y": "world", "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="string y values ignored") + + +def test_covarianceSamp_boolean_values_ignored(collection): + """$covarianceSamp ignores rows where x or y is boolean.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": True, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": False}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Rows 2 and 3 ignored. Valid pairs: (1,2) and (4,8) + # mean_x=2.5, mean_y=5, covSamp = ((-1.5)(-3)+(1.5)(3))/1 = (4.5+4.5)/1 = 9.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 9.0}, + {"_id": 2, "partition": "A", "x": True, "y": 4, "result": 9.0}, + {"_id": 3, "partition": "A", "x": 3, "y": False, "result": 9.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 9.0}, + ] + assertSuccess(result, expected, msg="boolean values ignored in both positions") + + +def test_covarianceSamp_array_values_ignored(collection): + """$covarianceSamp ignores rows where x or y is an array.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": [1, 2, 3], "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": [1, 2, 3], "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="array values ignored") + + +def test_covarianceSamp_object_values_ignored(collection): + """$covarianceSamp ignores rows where x or y is an object/document.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": {"nested": 99}, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 4.0}, + {"_id": 2, "partition": "A", "x": {"nested": 99}, "y": 4, "result": 4.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + ] + assertSuccess(result, expected, msg="object values ignored") + + +def test_covarianceSamp_objectid_and_regex_and_binary_ignored(collection): + """$covarianceSamp ignores ObjectId, Regex, and Binary values.""" + oid = ObjectId("507f1f77bcf86cd799439011") + docs = [ + {"_id": 1, "partition": "A", "x": oid, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": Regex("^test", "i")}, + {"_id": 3, "partition": "A", "x": Binary(b"\x01\x02\x03"), "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + {"_id": 5, "partition": "A", "x": 6, "y": 12}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only rows 4 and 5 valid: (4,8) and (6,12) + # mean_x=5, mean_y=10, covSamp = ((-1)(-2)+(1)(2))/1 = 4/1 = 4.0 + expected = [ + {"_id": 1, "result": 4.0}, + {"_id": 2, "result": 4.0}, + {"_id": 3, "result": 4.0}, + {"_id": 4, "result": 4.0}, + {"_id": 5, "result": 4.0}, + ] + assertSuccess(result, expected, msg="ObjectId/Regex/Binary values ignored") + + +def test_covarianceSamp_timestamp_minkey_maxkey_ignored(collection): + """$covarianceSamp ignores Timestamp, MinKey, and MaxKey values.""" + docs = [ + {"_id": 1, "partition": "A", "x": Timestamp(1234567890, 1), "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": MinKey()}, + {"_id": 3, "partition": "A", "x": MaxKey(), "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + {"_id": 5, "partition": "A", "x": 6, "y": 12}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only rows 4 and 5 valid: (4,8) and (6,12) -> covSamp = 4.0 + expected = [ + {"_id": 1, "result": 4.0}, + {"_id": 2, "result": 4.0}, + {"_id": 3, "result": 4.0}, + {"_id": 4, "result": 4.0}, + {"_id": 5, "result": 4.0}, + ] + assertSuccess(result, expected, msg="Timestamp/MinKey/MaxKey values ignored") + + +# Property [All Non-Numeric Returns Null]: when all values are non-numeric, result is null + + +def test_covarianceSamp_all_non_numeric_returns_null(collection): + """$covarianceSamp returns null when all values in frame are non-numeric.""" + docs = [ + {"_id": 1, "partition": "A", "x": "a", "y": 2}, + {"_id": 2, "partition": "A", "x": None, "y": 4}, + {"_id": 3, "partition": "A", "y": 6}, # x missing + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": "a", "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": None, "y": 4, "result": None}, + {"_id": 3, "partition": "A", "y": 6, "result": None}, + ] + assertSuccess(result, expected, msg="all non-numeric x values in frame returns null") + + +def test_covarianceSamp_all_non_numeric_diverse_types(collection): + """$covarianceSamp returns null when all values are diverse non-numeric types.""" + docs = [ + {"_id": 1, "partition": "A", "x": "text", "y": 10}, + {"_id": 2, "partition": "A", "x": True, "y": 20}, + {"_id": 3, "partition": "A", "x": datetime(2023, 1, 1, tzinfo=timezone.utc), "y": 30}, + {"_id": 4, "partition": "A", "x": [1, 2], "y": 40}, + {"_id": 5, "partition": "A", "x": {"a": 1}, "y": 50}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # No valid numeric x values -> null + expected = [ + {"_id": 1, "partition": "A", "x": "text", "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": True, "y": 20, "result": None}, + { + "_id": 3, + "partition": "A", + "x": datetime(2023, 1, 1, tzinfo=timezone.utc), + "y": 30, + "result": None, + }, + {"_id": 4, "partition": "A", "x": [1, 2], "y": 40, "result": None}, + {"_id": 5, "partition": "A", "x": {"a": 1}, "y": 50, "result": None}, + ] + assertSuccess(result, expected, msg="all diverse non-numeric types return null") + + +# Property [Single Valid Pair Returns Null]: when only one numeric pair exists, covSamp = null + + +def test_covarianceSamp_single_valid_pair_returns_null(collection): + """$covarianceSamp returns null when only one valid numeric pair exists (N-1=0).""" + docs = [ + {"_id": 1, "partition": "A", "x": "a", "y": 2}, + {"_id": 2, "partition": "A", "x": 5, "y": 10}, + {"_id": 3, "partition": "A", "x": None, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Only 1 valid numeric pair (5,10) -> N=1 -> covSamp = null + expected = [ + {"_id": 1, "partition": "A", "x": "a", "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 5, "y": 10, "result": None}, + {"_id": 3, "partition": "A", "x": None, "y": 6, "result": None}, + ] + assertSuccess(result, expected, msg="single valid numeric pair -> covSamp = null (N-1=0)") + + +# Property [Mixed Types in Frame]: non-numeric values filtered per-frame, numerics participate + + +def test_covarianceSamp_mixed_numeric_non_numeric_sliding(collection): + """$covarianceSamp in sliding window with mix of numeric and non-numeric values.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": "skip", "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": None, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-2, 2]}, + expression=["$x", "$y"], + ) + # Window [-2, 2] (5-doc centered): + # Row 1: frame docs 1-3, valid pairs: (1,10),(3,30) -> mean_x=2,mean_y=20 + # covSamp = ((-1)(-10)+(1)(10))/1 = 20/1 = 20.0 + # Row 2: frame docs 1-4, valid pairs: (1,10),(3,30) -> covSamp = 20.0 + # Row 3: frame docs 1-5, valid pairs: (1,10),(3,30),(5,50) -> mean_x=3,mean_y=30 + # covSamp = ((-2)(-20)+(0)(0)+(2)(20))/2 = (40+0+40)/2 = 80/2 = 40.0 + # Row 4: frame docs 2-5, valid pairs: (3,30),(5,50) -> mean_x=4,mean_y=40 + # covSamp = ((-1)(-10)+(1)(10))/1 = 20.0 + # Row 5: frame docs 3-5, valid pairs: (3,30),(5,50) -> covSamp = 20.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": 20.0}, + {"_id": 2, "partition": "A", "x": "skip", "y": 20, "result": 20.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 40.0}, + {"_id": 4, "partition": "A", "x": None, "y": 40, "result": 20.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 20.0}, + ] + assertSuccess(result, expected, msg="mixed types in sliding window — non-numeric ignored") + + +def test_covarianceSamp_numeric_among_diverse_types_cumulative(collection): + """$covarianceSamp cumulative window with numerics scattered among diverse types.""" + docs = [ + {"_id": 1, "partition": "A", "x": "text", "y": 2}, + {"_id": 2, "partition": "A", "x": 1, "y": 2}, + {"_id": 3, "partition": "A", "x": datetime(2023, 6, 1, tzinfo=timezone.utc), "y": 4}, + {"_id": 4, "partition": "A", "x": 3, "y": 6}, + {"_id": 5, "partition": "A", "x": True, "y": 8}, + {"_id": 6, "partition": "A", "x": 5, "y": 10}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "current"]}, + expression=["$x", "$y"], + ) + # Cumulative, only numeric x and numeric y pairs count: + # Row 1: no valid pair -> null + # Row 2: [(1,2)] -> single pair -> null (N-1=0) + # Row 3: [(1,2)] -> datetime x ignored, still single pair -> null + # Row 4: [(1,2),(3,6)] -> mean_x=2, mean_y=4, covSamp = ((-1)(-2)+(1)(2))/1 = 4.0 + # Row 5: [(1,2),(3,6)] -> True ignored, still 2 pairs -> 4.0 + # Row 6: [(1,2),(3,6),(5,10)] -> mean_x=3, mean_y=6 + # covSamp = ((-2)(-4)+(0)(0)+(2)(4))/2 = 16/2 = 8.0 + expected = [ + {"_id": 1, "partition": "A", "x": "text", "y": 2, "result": None}, + {"_id": 2, "partition": "A", "x": 1, "y": 2, "result": None}, + { + "_id": 3, + "partition": "A", + "x": datetime(2023, 6, 1, tzinfo=timezone.utc), + "y": 4, + "result": None, + }, + {"_id": 4, "partition": "A", "x": 3, "y": 6, "result": 4.0}, + {"_id": 5, "partition": "A", "x": True, "y": 8, "result": 4.0}, + {"_id": 6, "partition": "A", "x": 5, "y": 10, "result": 8.0}, + ] + assertSuccess(result, expected, msg="cumulative window with numerics among diverse types") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_numeric_precision.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_numeric_precision.py new file mode 100644 index 000000000..91ce3949e --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_numeric_precision.py @@ -0,0 +1,1410 @@ +""" +Tests for $covarianceSamp numeric type mixing, overflow edge cases, +algorithmic precision validation, and Decimal128 type handling. + +Covers: Int32/Int64/Double mixing, Int64 near MAX_LONG (overflow risk when +squaring), catastrophic cancellation in variance calculation, known exact +results, very small differences, consistency between window modes, +Decimal128 (NumberDecimal) values, high-precision Decimal128, mixed Decimal128 +with other numeric types, and Decimal128 special values (NaN, Infinity). + +Server behavior (verified): +- When ANY input value is Decimal128, the server returns Decimal128 type results +- $covarianceSamp divides by N-1; single element returns null +- 1e308 identical values: returns NaN (overflow in intermediate computation) +""" + +from bson import Decimal128, Int64 + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertResult, assertSuccess, assertSuccessNaN +from documentdb_tests.framework.executor import execute_command +from documentdb_tests.framework.property_checks import Gt, Lte, PerDoc +from documentdb_tests.framework.test_constants import ( + DECIMAL128_INFINITY, + DECIMAL128_LARGE_EXPONENT, + DECIMAL128_MAX, + DECIMAL128_MIN, + DECIMAL128_NEGATIVE_INFINITY, + DECIMAL128_NEGATIVE_ZERO, + DECIMAL128_SMALL_EXPONENT, + DOUBLE_NEAR_MAX, + DOUBLE_NEGATIVE_ZERO, + FLOAT_INFINITY, + FLOAT_NAN, + FLOAT_NEGATIVE_INFINITY, +) + +# Property [Numeric Type Mixing]: Int32, Int64, Double coexist correctly + + +def test_covarianceSamp_all_int32_values(collection): + """$covarianceSamp with all Int32 values produces Double result.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1,2,3], y=[2,4,6]: covSamp = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 2.0}, + ] + assertSuccess(result, expected, msg="all Int32 values produce correct Double result") + + +def test_covarianceSamp_all_int64_values(collection): + """$covarianceSamp with all Int64 values produces correct result.""" + docs = [ + {"_id": 1, "partition": "A", "x": Int64(1), "y": Int64(2)}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": Int64(4)}, + {"_id": 3, "partition": "A", "x": Int64(3), "y": Int64(6)}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": Int64(1), "y": Int64(2), "result": 2.0}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": Int64(4), "result": 2.0}, + {"_id": 3, "partition": "A", "x": Int64(3), "y": Int64(6), "result": 2.0}, + ] + assertSuccess(result, expected, msg="all Int64 values compute correctly") + + +def test_covarianceSamp_mixed_int32_int64_double(collection): + """$covarianceSamp with mixed Int32 + Int64 + Double in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2.0}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": 4}, + {"_id": 3, "partition": "A", "x": 3.0, "y": Int64(6)}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2.0, "result": 2.0}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": 4, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3.0, "y": Int64(6), "result": 2.0}, + ] + assertSuccess(result, expected, msg="mixed Int32 + Int64 + Double type promotion works") + + +# Property [Large Value Handling]: near-overflow and large-spread values compute without overflow + + +def test_covarianceSamp_large_int64_near_max(collection): + """$covarianceSamp with Int64 values near MAX_LONG — squaring would overflow 64-bit.""" + docs = [ + { + "_id": 1, + "partition": "A", + "x": Int64(9223372036854775806), + "y": Int64(9223372036854775806), + }, + { + "_id": 2, + "partition": "A", + "x": Int64(9223372036854775807), + "y": Int64(9223372036854775807), + }, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Both values round to the same float64 at this scale -> covSamp = 0.0 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Int64(9223372036854775806), + "y": Int64(9223372036854775806), + "result": 0.0, + }, + { + "_id": 2, + "partition": "A", + "x": Int64(9223372036854775807), + "y": Int64(9223372036854775807), + "result": 0.0, + }, + ] + assertSuccess(result, expected, msg="Int64 near MAX_LONG does not overflow") + + +def test_covarianceSamp_large_int64_spread(collection): + """$covarianceSamp with widely spread Int64 values — tests numeric stability.""" + docs = [ + {"_id": 1, "partition": "A", "x": Int64(0), "y": Int64(0)}, + { + "_id": 2, + "partition": "A", + "x": Int64(4611686018427387903), + "y": Int64(4611686018427387903), + }, + { + "_id": 3, + "partition": "A", + "x": Int64(9223372036854775807), + "y": Int64(9223372036854775807), + }, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # When x==y, covSamp(x,y) = varSamp(x) > 0 + checks = PerDoc( + {"result": Gt(0)}, + {"result": Gt(0)}, + {"result": Gt(0)}, + ) + assertResult(result, expected=checks, msg="Large Int64 spread produces positive result") + + +def test_covarianceSamp_very_large_value(collection): + """$covarianceSamp with very large numeric value (1e308) — all identical returns NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1e308, "y": 1e308}, + {"_id": 2, "partition": "A", "x": 1e308, "y": 1e308}, + {"_id": 3, "partition": "A", "x": 1e308, "y": 1e308}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns NaN due to overflow in intermediate computation + expected = [ + {"_id": 1, "partition": "A", "x": 1e308, "y": 1e308, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": 1e308, "y": 1e308, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 1e308, "y": 1e308, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="very large identical values overflow to NaN") + + +def test_covarianceSamp_alternating_large_values(collection): + """$covarianceSamp with alternating sign large values — stress accumulator.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1e15, "y": 1e15}, + {"_id": 2, "partition": "A", "x": -1e15, "y": -1e15}, + {"_id": 3, "partition": "A", "x": 1e15, "y": 1e15}, + {"_id": 4, "partition": "A", "x": -1e15, "y": -1e15}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # When x==y: covSamp = varSamp. Mean=0, varSamp = (4*1e30)/3 = 4e30/3 + expected = [ + {"_id": 1, "partition": "A", "x": 1e15, "y": 1e15, "result": 1.3333333333333332e30}, + {"_id": 2, "partition": "A", "x": -1e15, "y": -1e15, "result": 1.3333333333333332e30}, + {"_id": 3, "partition": "A", "x": 1e15, "y": 1e15, "result": 1.3333333333333332e30}, + {"_id": 4, "partition": "A", "x": -1e15, "y": -1e15, "result": 1.3333333333333332e30}, + ] + assertSuccess(result, expected, msg="alternating large values produce correct covarianceSamp") + + +# Property [Algorithmic Precision]: known exact results and catastrophic cancellation handling + + +def test_covarianceSamp_known_exact_result(collection): + """$covarianceSamp with known exact result: covSamp([1,2,3,4],[2,4,6,8]) = 10/3.""" + docs = [ + {"_id": i, "partition": "A", "x": x, "y": y} + for i, (x, y) in enumerate([(1, 2), (2, 4), (3, 6), (4, 8)], 1) + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1,2,3,4], y=[2,4,6,8]: mean_x=2.5, mean_y=5 + # covSamp = ((-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3))/3 = (4.5+0.5+0.5+4.5)/3 = 10/3 + expected = [ + {"_id": i, "partition": "A", "x": x, "y": y, "result": 3.3333333333333335} + for i, (x, y) in enumerate([(1, 2), (2, 4), (3, 6), (4, 8)], 1) + ] + assertSuccess(result, expected, msg="covarianceSamp of [1,2,3,4],[2,4,6,8] must be 10/3") + + +def test_covarianceSamp_identical_values_exactly_zero(collection): + """$covarianceSamp of identical (x,y) pairs where y is constant must be exactly 0.0.""" + docs = [ + {"_id": 1, "partition": "A", "x": 3.0, "y": 7.0}, + {"_id": 2, "partition": "A", "x": 3.0, "y": 7.0}, + {"_id": 3, "partition": "A", "x": 3.0, "y": 7.0}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 3.0, "y": 7.0, "result": 0.0}, + {"_id": 2, "partition": "A", "x": 3.0, "y": 7.0, "result": 0.0}, + {"_id": 3, "partition": "A", "x": 3.0, "y": 7.0, "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical pairs produce exactly 0.0") + + +def test_covarianceSamp_catastrophic_cancellation(collection): + """$covarianceSamp with large offset values — naive algorithm fails.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1000000001, "y": 1000000002}, + {"_id": 2, "partition": "A", "x": 1000000002, "y": 1000000004}, + {"_id": 3, "partition": "A", "x": 1000000003, "y": 1000000006}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x = [N, N+1, N+2], y = [M, M+2, M+4] where N,M are large + # After centering: x-offsets = [-1, 0, 1], y-offsets = [-2, 0, 2] + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1000000001, "y": 1000000002, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 1000000002, "y": 1000000004, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 1000000003, "y": 1000000006, "result": 2.0}, + ] + assertSuccess( + result, + expected, + msg="catastrophic cancellation handled — correct covSamp for large offset values", + ) + + +def test_covarianceSamp_very_small_differences(collection): + """$covarianceSamp with values that differ by very small amounts.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1.0000001, "y": 2.0000002}, + {"_id": 2, "partition": "A", "x": 1.0000002, "y": 2.0000004}, + {"_id": 3, "partition": "A", "x": 1.0000003, "y": 2.0000006}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # The covarianceSamp should be positive and tiny + checks = PerDoc( + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + ) + assertResult( + result, expected=checks, msg="Small differences produce very small positive covSamp" + ) + + +# Property [Single Element Frame]: single value produces null for sample covariance + + +def test_covarianceSamp_single_element_sliding_window(collection): + """$covarianceSamp returns null when sliding window frame has exactly one value.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [0, 0]}, + expression=["$x", "$y"], + ) + # Window [0, 0] — each frame has exactly one value -> covSamp = null (N-1=0) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": None}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": None}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": None}, + ] + assertSuccess(result, expected, msg="single element in sliding frame returns null") + + +# Property [Decimal128 Support]: Decimal128 values return Decimal128 type results. + + +def test_covarianceSamp_pure_decimal128_values(collection): + """$covarianceSamp with pure Decimal128 values returns Decimal128 type result.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type: covSamp = 4/2 = 2 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": Decimal128("2"), + "result": Decimal128("2"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2"), + "y": Decimal128("4"), + "result": Decimal128("2"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Decimal128("6"), + "result": Decimal128("2"), + }, + ] + assertSuccess(result, expected, msg="pure Decimal128 values return Decimal128 type result") + + +def test_covarianceSamp_decimal128_with_double(collection): + """$covarianceSamp with mixed Decimal128 and Double returns Decimal128 type.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2.0}, + {"_id": 2, "partition": "A", "x": 2.0, "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": 6.0}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2.0, "result": Decimal128("2.0")}, + {"_id": 2, "partition": "A", "x": 2.0, "y": Decimal128("4"), "result": Decimal128("2.0")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": 6.0, "result": Decimal128("2.0")}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Double returns Decimal128 type") + + +def test_covarianceSamp_decimal128_with_int32(collection): + """$covarianceSamp with mixed Decimal128 and Int32 returns Decimal128 type.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2, "result": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": 2, "y": Decimal128("4"), "result": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": 6, "result": Decimal128("2")}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int32 returns Decimal128 type") + + +def test_covarianceSamp_decimal128_with_int64(collection): + """$covarianceSamp with mixed Decimal128 and Int64 returns Decimal128 type.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Int64(2)}, + {"_id": 2, "partition": "A", "x": Int64(2), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Int64(6)}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": Int64(2), + "result": Decimal128("2"), + }, + { + "_id": 2, + "partition": "A", + "x": Int64(2), + "y": Decimal128("4"), + "result": Decimal128("2"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Int64(6), + "result": Decimal128("2"), + }, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int64 returns Decimal128 type") + + +def test_covarianceSamp_decimal128_all_types_mixed(collection): + """$covarianceSamp with Decimal128 + Double + Int32 + Int64 all in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": 2.0}, + {"_id": 2, "partition": "A", "x": 2.0, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": Int64(6)}, + {"_id": 4, "partition": "A", "x": Int64(4), "y": Decimal128("8")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1,2,3,4], y=[2,4,6,8] -> covSamp = 10/3 + # Server returns Decimal128 type when any input is Decimal128 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": 2.0, + "result": Decimal128("3.333333333333333333333333333333333"), + }, + { + "_id": 2, + "partition": "A", + "x": 2.0, + "y": 4, + "result": Decimal128("3.333333333333333333333333333333333"), + }, + { + "_id": 3, + "partition": "A", + "x": 3, + "y": Int64(6), + "result": Decimal128("3.333333333333333333333333333333333"), + }, + { + "_id": 4, + "partition": "A", + "x": Int64(4), + "y": Decimal128("8"), + "result": Decimal128("3.333333333333333333333333333333333"), + }, + ] + assertSuccess(result, expected, msg="all four numeric types mixed returns Decimal128 type") + + +def test_covarianceSamp_decimal128_sliding_window(collection): + """$covarianceSamp with Decimal128 values in a sliding window.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + {"_id": 4, "partition": "A", "x": Decimal128("4"), "y": Decimal128("8")}, + {"_id": 5, "partition": "A", "x": Decimal128("5"), "y": Decimal128("10")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Window [-1, 0]: + # Row 1: [(1,2)] -> single pair -> null (N-1=0) + # Row 2: [(1,2),(2,4)] -> covSamp = 1/1 = Decimal128("1") + # Row 3: [(2,4),(3,6)] -> Decimal128("1") + # Row 4: [(3,6),(4,8)] -> Decimal128("1") + # Row 5: [(4,8),(5,10)] -> Decimal128("1") + expected = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2"), "result": None}, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2"), + "y": Decimal128("4"), + "result": Decimal128("1"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Decimal128("6"), + "result": Decimal128("1"), + }, + { + "_id": 4, + "partition": "A", + "x": Decimal128("4"), + "y": Decimal128("8"), + "result": Decimal128("1"), + }, + { + "_id": 5, + "partition": "A", + "x": Decimal128("5"), + "y": Decimal128("10"), + "result": Decimal128("1"), + }, + ] + assertSuccess(result, expected, msg="Decimal128 sliding window returns Decimal128 type") + + +def test_covarianceSamp_decimal128_identical_values(collection): + """$covarianceSamp with identical Decimal128 value pairs returns Decimal128('0E+12').""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("42.5"), "y": Decimal128("99.9")}, + {"_id": 2, "partition": "A", "x": Decimal128("42.5"), "y": Decimal128("99.9")}, + {"_id": 3, "partition": "A", "x": Decimal128("42.5"), "y": Decimal128("99.9")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # Server returns Decimal128("0E+12") for identical values, not float 0.0 + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("42.5"), + "y": Decimal128("99.9"), + "result": Decimal128("0E+12"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("42.5"), + "y": Decimal128("99.9"), + "result": Decimal128("0E+12"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("42.5"), + "y": Decimal128("99.9"), + "result": Decimal128("0E+12"), + }, + ] + assertSuccess( + result, expected, msg="identical Decimal128 value pairs return Decimal128('0E+12')" + ) + + +# Property [Decimal128 Special Values]: Decimal128 NaN and Infinity handling + + +def test_covarianceSamp_decimal128_nan_special(collection): + """$covarianceSamp with Decimal128 NaN — NaN is numeric and poisons the calculation.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("NaN"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + { + "_id": 1, + "partition": "A", + "x": Decimal128("1"), + "y": Decimal128("2"), + "result": FLOAT_NAN, + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("NaN"), + "y": Decimal128("4"), + "result": FLOAT_NAN, + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3"), + "y": Decimal128("6"), + "result": FLOAT_NAN, + }, + ] + assertSuccessNaN( + result, expected, msg="Decimal128 NaN is numeric and poisons covarianceSamp to NaN" + ) + + +def test_covarianceSamp_decimal128_infinity_special(collection): + """$covarianceSamp with Decimal128 Infinity special value.""" + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("Infinity"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": Decimal128("6")}, + ] + collection.insert_many(docs) + extra_stages = [ + { + "$addFields": { + "has_result": {"$ne": ["$result", None]}, + } + }, + {"$project": {"_id": 1, "has_result": 1}}, + ] + pipeline = [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$covarianceSamp": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ] + extra_stages + result = execute_command( + collection, + {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}}, + ) + expected = [ + {"_id": 1, "has_result": True}, + {"_id": 2, "has_result": True}, + {"_id": 3, "has_result": True}, + ] + assertSuccess(result, expected, msg="Decimal128 Infinity produces a non-null result") + + +# Property [Decimal128 Precision Boundaries]: boundary values from test_constants + + +def test_covarianceSamp_decimal128_min_values(collection): + """$covarianceSamp with DECIMAL128_MIN values — overflow in intermediate computation.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MIN, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MIN, "y": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MIN, "y": Decimal128("3")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # All x values identical, but intermediate computation overflows -> Infinity + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_MIN, + "y": Decimal128("1"), + "result": Decimal128("Infinity"), + }, + { + "_id": 2, + "partition": "A", + "x": DECIMAL128_MIN, + "y": Decimal128("2"), + "result": Decimal128("Infinity"), + }, + { + "_id": 3, + "partition": "A", + "x": DECIMAL128_MIN, + "y": Decimal128("3"), + "result": Decimal128("Infinity"), + }, + ] + assertSuccess( + result, expected, msg="DECIMAL128_MIN overflows to Infinity in intermediate computation" + ) + + +def test_covarianceSamp_decimal128_large_exponent(collection): + """$covarianceSamp with DECIMAL128_LARGE_EXPONENT values — high exponent Decimal128.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_LARGE_EXPONENT, "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2E+6144"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3E+6144"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1E+6144, 2E+6144, 3E+6144], y=[2,4,6]: covSamp = 4/2 * 1E+6144 = 2E+6144 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_LARGE_EXPONENT, + "y": Decimal128("2"), + "result": Decimal128("2.000000000000000000000000000000000E+6144"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2E+6144"), + "y": Decimal128("4"), + "result": Decimal128("2.000000000000000000000000000000000E+6144"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3E+6144"), + "y": Decimal128("6"), + "result": Decimal128("2.000000000000000000000000000000000E+6144"), + }, + ] + assertSuccess(result, expected, msg="DECIMAL128_LARGE_EXPONENT produces positive covSamp") + + +def test_covarianceSamp_decimal128_small_exponent(collection): + """$covarianceSamp with DECIMAL128_SMALL_EXPONENT values — very small Decimal128.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_SMALL_EXPONENT, "y": Decimal128("2")}, + {"_id": 2, "partition": "A", "x": Decimal128("2E-6143"), "y": Decimal128("4")}, + {"_id": 3, "partition": "A", "x": Decimal128("3E-6143"), "y": Decimal128("6")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1E-6143, 2E-6143, 3E-6143], y=[2,4,6]: covSamp = 4/2 * 1E-6143 = 2E-6143 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_SMALL_EXPONENT, + "y": Decimal128("2"), + "result": Decimal128("2E-6143"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("2E-6143"), + "y": Decimal128("4"), + "result": Decimal128("2E-6143"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("3E-6143"), + "y": Decimal128("6"), + "result": Decimal128("2E-6143"), + }, + ] + assertSuccess(result, expected, msg="DECIMAL128_SMALL_EXPONENT produces positive covSamp") + + +def test_covarianceSamp_decimal128_negative_zero(collection): + """$covarianceSamp with DECIMAL128_NEGATIVE_ZERO — treated as numeric zero.""" + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_NEGATIVE_ZERO, "y": Decimal128("10")}, + {"_id": 2, "partition": "A", "x": Decimal128("10"), "y": Decimal128("20")}, + {"_id": 3, "partition": "A", "x": Decimal128("20"), "y": Decimal128("30")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[-0, 10, 20] -> mean_x=10, y=[10, 20, 30] -> mean_y=20 + # covSamp = ((-10)(-10)+(0)(0)+(10)(10))/2 = (100+0+100)/2 = 200/2 = 100.0 + expected = [ + { + "_id": 1, + "partition": "A", + "x": DECIMAL128_NEGATIVE_ZERO, + "y": Decimal128("10"), + "result": Decimal128("1E+2"), + }, + { + "_id": 2, + "partition": "A", + "x": Decimal128("10"), + "y": Decimal128("20"), + "result": Decimal128("1E+2"), + }, + { + "_id": 3, + "partition": "A", + "x": Decimal128("20"), + "y": Decimal128("30"), + "result": Decimal128("1E+2"), + }, + ] + assertSuccess(result, expected, msg="DECIMAL128_NEGATIVE_ZERO treated as numeric zero") + + +# Property [Negative Zero]: -0.0 treated as numeric zero + + +def test_covarianceSamp_negative_zero(collection): + """$covarianceSamp treats -0.0 as numeric zero — participates in computation.""" + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEGATIVE_ZERO, "y": 10}, + {"_id": 2, "partition": "A", "x": 10, "y": 20}, + {"_id": 3, "partition": "A", "x": 20, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[-0, 10, 20] -> mean_x=10, y=[10, 20, 30] -> mean_y=20 + # covSamp = ((-10)(-10)+(0)(0)+(10)(10))/2 = (100+0+100)/2 = 200/2 = 100.0 + expected = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEGATIVE_ZERO, "y": 10, "result": 100.0}, + {"_id": 2, "partition": "A", "x": 10, "y": 20, "result": 100.0}, + {"_id": 3, "partition": "A", "x": 20, "y": 30, "result": 100.0}, + ] + assertSuccess(result, expected, msg="-0.0 treated as numeric zero in covarianceSamp") + + +# Property [Basic Numeric]: standard numeric inputs handled correctly + + +def test_covarianceSamp_negative_numbers(collection): + """$covarianceSamp handles negative numbers correctly.""" + docs = [ + {"_id": 1, "partition": "A", "x": -10, "y": -20}, + {"_id": 2, "partition": "A", "x": 0, "y": 0}, + {"_id": 3, "partition": "A", "x": 10, "y": 20}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[-10,0,10] mean=0, y=[-20,0,20] mean=0 + # covSamp = ((-10)(-20)+(0)(0)+(10)(20))/2 = (200+0+200)/2 = 400/2 = 200.0 + expected = [ + {"_id": 1, "partition": "A", "x": -10, "y": -20, "result": 200.0}, + {"_id": 2, "partition": "A", "x": 0, "y": 0, "result": 200.0}, + {"_id": 3, "partition": "A", "x": 10, "y": 20, "result": 200.0}, + ] + assertSuccess(result, expected, msg="negative numbers handled correctly") + + +def test_covarianceSamp_decimals(collection): + """$covarianceSamp handles floating-point (double) values.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1.5, "y": 3.0}, + {"_id": 2, "partition": "A", "x": 2.5, "y": 5.0}, + {"_id": 3, "partition": "A", "x": 3.5, "y": 7.0}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + # x=[1.5,2.5,3.5] mean=2.5, y=[3,5,7] mean=5 + # covSamp = ((-1)(-2)+(0)(0)+(1)(2))/2 = 4/2 = 2.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1.5, "y": 3.0, "result": 2.0}, + {"_id": 2, "partition": "A", "x": 2.5, "y": 5.0, "result": 2.0}, + {"_id": 3, "partition": "A", "x": 3.5, "y": 7.0, "result": 2.0}, + ] + assertSuccess(result, expected, msg="floating-point values handled correctly") + + +# --------------------------------------------------------------------------- +# Property [Intermediate Overflow]: TEST_COVERAGE.md §22 overflow requirements +# +# Covariance is computed by an online (Welford-style) update, so an +# *intermediate* value can overflow independently of whether the final result +# is representable. The four cases below are deliberately kept separate -- +# a single "large values" test cannot distinguish them, and each has a +# different failure mode: +# +# 1. No overflow in the formula -- identical large x, so every deviation is +# exactly 0 (n>=2; a single-element frame is +# null via the n-1 divisor). +# 2. Deviation overflow -- opposing magnitudes in one column, so +# x_i - mean_x overflows. The expected sign +# is asserted, not merely non-finiteness. +# 3. Product overflow only -- deviations stay finite, their product +# overflows. +# 4. Result exceeds range -- true result beyond the type maximum. +# +# A literal Infinity *input* exercises none of these: non-finite inputs are +# short-circuited before the online update runs. Those live in the +# special_floats file. +# +# All expectations below were verified against the reference server. +# --------------------------------------------------------------------------- + + +# --- Case 1: no overflow -- identical large values, deviations exactly 0 --- + + +def test_covarianceSamp_identical_decimal128_max_three_docs(collection): + """$covarianceSamp with 3 identical DECIMAL128_MAX x values -- deviations are exactly 0. + + Nothing in (1/(n-1))*sum((x_i - mean_x)(y_i - mean_y)) overflows: every x_i is the + same value, so every x_i - mean_x is exactly 0 at any magnitude and the formula + never requires sum(x_i). The server reports -Infinity because it derives the mean + from a running sum that overflows; its sign follows the y ordering rather than + the data (see the y-reversed test below). + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("3")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="identical DECIMAL128_MAX x values: The server reports -Infinity from sum-derived mean", + ) + + +def test_covarianceSamp_identical_decimal128_max_y_reversed(collection): + """$covarianceSamp identical DECIMAL128_MAX x with descending y -- sign flips. + + Same x column as the previous test, same true answer (0), but reversing y + flips The server's reported infinity from -Infinity to +Infinity. The sign + tracks the y ordering, not the covariance, which is what identifies it as an + overflow artifact rather than a semantic. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("3")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("2")}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="reversing y flips the reported infinity sign for identical x", + ) + + +def test_covarianceSamp_identical_decimal128_max_two_docs(collection): + """$covarianceSamp with 2 identical DECIMAL128_MAX x values -- no overflow yet. + + Count-dependence check. The true answer is 0 for any number of identical x + values, but The server's running sum only overflows once a third value is added, + so n=2 returns 0 while n=3 returns -Infinity. This is the baseline that makes + the n=3 result meaningful. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("2")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": Decimal128("0E+14")} for doc in docs] + assertSuccess( + result, + expected, + msg="two identical DECIMAL128_MAX x values do not overflow the running sum", + ) + + +def test_covarianceSamp_identical_decimal128_max_in_y(collection): + """$covarianceSamp with identical DECIMAL128_MAX in the y position. + + Mirror of the x-side test: the same artifact must be probed per expression + position, since the two arguments are accumulated separately. + """ + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1"), "y": DECIMAL128_MAX}, + {"_id": 2, "partition": "A", "x": Decimal128("2"), "y": DECIMAL128_MAX}, + {"_id": 3, "partition": "A", "x": Decimal128("3"), "y": DECIMAL128_MAX}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="identical DECIMAL128_MAX y values: same artifact on the y side", + ) + + +def test_covarianceSamp_identical_double_near_max_in_y(collection): + """$covarianceSamp with identical DOUBLE_NEAR_MAX in y -- double path, y side. + + The double path shows the same count dependence as decimal: two identical + 1e308 x values return 0 (below), while the y-side sum here overflows. + """ + docs = [ + {"_id": 1, "partition": "A", "x": 1.0, "y": DOUBLE_NEAR_MAX}, + {"_id": 2, "partition": "A", "x": 2.0, "y": DOUBLE_NEAR_MAX}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="identical DOUBLE_NEAR_MAX y values overflow the running sum", + ) + + +def test_covarianceSamp_identical_double_near_max_two_docs(collection): + """$covarianceSamp with 2 identical DOUBLE_NEAR_MAX x values -- returns 0. + + Double-path baseline for case 1: x=[1e308, 1e308] does not overflow the + running sum, so the result is exactly 0. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": 1.0}, + {"_id": 2, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": 2.0}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": 0.0} for doc in docs] + assertSuccess( + result, + expected, + msg="two identical DOUBLE_NEAR_MAX x values give exactly 0", + ) + + +# --- Case 2: deviation overflow -- SIGN-critical --- + + +def test_covarianceSamp_deviation_overflow_positive_decimal128(collection): + """$covarianceSamp with x=y=[DECIMAL128_MAX, DECIMAL128_MIN] -- expects +Infinity. + + Opposing maximum magnitudes in the same column make x_i - mean_x overflow. + x and y move together, so the limit is +Infinity. The sign is asserted + rather than just non-finiteness, since -Infinity is also non-finite and + would satisfy a weaker assertion. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MIN, "y": DECIMAL128_MIN}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="positively-correlated deviation overflow gives +Infinity", + ) + + +def test_covarianceSamp_deviation_overflow_negative_decimal128(collection): + """$covarianceSamp anti-correlated DECIMAL128 extremes -- expects -Infinity. + + Same magnitudes as the previous test with y inverted, so the limit is + -Infinity. Paired with that test, this pins the sign to the direction of the + data rather than to the magnitudes. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MIN}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MIN, "y": DECIMAL128_MAX}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="anti-correlated deviation overflow gives -Infinity", + ) + + +def test_covarianceSamp_deviation_overflow_positive_double(collection): + """$covarianceSamp with x=y=[1e308, -1e308] -- double path, expects +Infinity. + + The deviation -1e308 - 1e308 overflows to -inf in the double path exactly as + it does in decimal, so the case is covered for both numeric types. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": DOUBLE_NEAR_MAX}, + {"_id": 2, "partition": "A", "x": -DOUBLE_NEAR_MAX, "y": -DOUBLE_NEAR_MAX}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="double-path positively-correlated deviation overflow gives +Infinity", + ) + + +def test_covarianceSamp_deviation_overflow_negative_double(collection): + """$covarianceSamp anti-correlated 1e308 extremes -- double path, expects -Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": DOUBLE_NEAR_MAX, "y": -DOUBLE_NEAR_MAX}, + {"_id": 2, "partition": "A", "x": -DOUBLE_NEAR_MAX, "y": DOUBLE_NEAR_MAX}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="double-path anti-correlated deviation overflow gives -Infinity", + ) + + +def test_covarianceSamp_deviation_overflow_representable_result(collection): + """$covarianceSamp with x=[DECIMAL128_MAX, 0] -- large magnitude, exact result. + + Deviations reach half of DECIMAL128_MAX without overflowing, and the result + -5.0E+6144 (the n-1 divisor doubles the population value) is representable, so + it is returned exactly. This establishes that full-range magnitudes alone do + not trigger the overflow cases above. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": Decimal128("1")}, + {"_id": 2, "partition": "A", "x": Decimal128("0"), "y": Decimal128("2")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {**doc, "result": Decimal128("-5.000000000000000000000000000000000E+6144")} for doc in docs + ] + assertSuccess( + result, + expected, + msg="DECIMAL128_MAX-scale deviations with a representable result are exact", + ) + + +# --- Case 3: product overflow with finite deviations (control) --- + + +def test_covarianceSamp_product_overflow_finite_deviations_double(collection): + """$covarianceSamp with x=y=[1e200, -1e200] -- product overflows, deviations do not. + + Control for the case-2 sign tests. Here 1e200 - (-1e200) = 2e200 stays finite + and only the deviation *product* overflows, giving +Infinity. Keeping this + separate from case 2 distinguishes an overflow in the deviation step from one + in the product step. + """ + docs = [ + {"_id": 1, "partition": "A", "x": 1e200, "y": 1e200}, + {"_id": 2, "partition": "A", "x": -1e200, "y": -1e200}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="product overflow with finite deviations preserves the sign", + ) + + +def test_covarianceSamp_product_overflow_finite_deviations_negative(collection): + """$covarianceSamp anti-correlated 1e200 -- product overflows to -Infinity. + + Negative-direction half of the control pair. + """ + docs = [ + {"_id": 1, "partition": "A", "x": 1e200, "y": -1e200}, + {"_id": 2, "partition": "A", "x": -1e200, "y": 1e200}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": FLOAT_NEGATIVE_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="anti-correlated product overflow gives -Infinity", + ) + + +def test_covarianceSamp_product_overflow_finite_deviations_decimal128(collection): + """$covarianceSamp with x=y=[1E+3100, -1E+3100] -- decimal product overflow. + + Decimal128 half of the control: deviations reach 2E+3100 (well inside range) + while their product exceeds E+6144. + """ + docs = [ + {"_id": 1, "partition": "A", "x": Decimal128("1E+3100"), "y": Decimal128("1E+3100")}, + {"_id": 2, "partition": "A", "x": Decimal128("-1E+3100"), "y": Decimal128("-1E+3100")}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": DECIMAL128_INFINITY} for doc in docs] + assertSuccess( + result, + expected, + msg="decimal product overflow with finite deviations preserves the sign", + ) + + +# --- Case 4: true result exceeds the type range --- + + +def test_covarianceSamp_result_exceeds_decimal128_range(collection): + """$covarianceSamp with x=y=DECIMAL128_MAX repeated -- server returns NaN. + + Identical values in both columns, so the formula gives 0, but The server's + sum-derived mean overflows in both accumulators and the indeterminate + Infinity - Infinity yields NaN. + """ + docs = [ + {"_id": 1, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + {"_id": 2, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + {"_id": 3, "partition": "A", "x": DECIMAL128_MAX, "y": DECIMAL128_MAX}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [{**doc, "result": Decimal128("NaN")} for doc in docs] + assertSuccessNaN( + result, + expected, + msg="DECIMAL128_MAX in both columns overflows both means to NaN", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_order_independence.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_order_independence.py new file mode 100644 index 000000000..4f4a5d9a0 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_order_independence.py @@ -0,0 +1,139 @@ +""" +Tests for $covarianceSamp order independence in window context. + +Verifies that $covarianceSamp produces the same result regardless of sortBy direction, +confirming it is an order-independent operator. Sample covariance is a +symmetric statistic over the frame — it depends only on which documents are in +the frame, not on their processing order. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + COVAR_DOCS, + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +UNBOUNDED_WINDOW = {"documents": ["unbounded", "unbounded"]} + +# Property [Order Independence]: $covarianceSamp produces same result regardless of sort direction + + +def test_covarianceSamp_whole_partition_ascending_sort(collection): + """$covarianceSamp whole partition with ascending sort.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": 1}, + ) + # covSamp of (x,y) where y=2x: covSamp = 20/4 = 5.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 5.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 5.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 5.0}, + ] + assertSuccess(result, expected, msg="ascending sort produces correct covarianceSamp") + + +def test_covarianceSamp_whole_partition_descending_sort(collection): + """$covarianceSamp whole partition with descending sort produces same result as ascending.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": -1}, + extra_stages=[{"$sort": {"_id": 1}}], + ) + # Same result regardless of sort direction — order-independent operator + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 5.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 5.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 5.0}, + ] + assertSuccess( + result, expected, msg="descending sort produces same covarianceSamp — order independent" + ) + + +def test_covarianceSamp_sort_by_value_vs_sort_by_id(collection): + """$covarianceSamp whole partition: sort by value field vs sort by _id gives same result.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"x": -1}, + extra_stages=[{"$sort": {"_id": 1}}], + ) + # Sorting by x descending should not affect whole-partition covarianceSamp + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": 5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": 5.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 8, "result": 5.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 10, "result": 5.0}, + ] + assertSuccess( + result, + expected, + msg="sort by value field produces same result as sort by _id — order independent", + ) + + +NEGATIVE_COVAR_DOCS = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 8}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + {"_id": 4, "partition": "A", "x": 4, "y": 4}, + {"_id": 5, "partition": "A", "x": 5, "y": 2}, +] + +# x=[1,2,3,4,5] mean=3, y=[10,8,6,4,2] mean=6 +# covSamp = ((-2)(4)+(-1)(2)+(0)(0)+(1)(-2)+(2)(-4))/4 = (-8-2+0-2-8)/4 = -20/4 = -5.0 +NEGATIVE_COVAR_EXPECTED = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": -5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 8, "result": -5.0}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": -5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 4, "result": -5.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 2, "result": -5.0}, +] + + +def test_covarianceSamp_negative_correlation_ascending_sort(collection): + """$covarianceSamp with negative correlation gives correct result with ascending sort.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=NEGATIVE_COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": 1}, + ) + assertSuccess( + result, NEGATIVE_COVAR_EXPECTED, msg="negative correlation ascending sort gives -5.0" + ) + + +def test_covarianceSamp_negative_correlation_descending_sort(collection): + """$covarianceSamp with negative correlation gives same result with descending sort.""" + result = run_window_operator( + collection, + "$covarianceSamp", + docs=NEGATIVE_COVAR_DOCS, + expression=["$x", "$y"], + window=UNBOUNDED_WINDOW, + sort_by={"_id": -1}, + extra_stages=[{"$sort": {"_id": 1}}], + ) + assertSuccess( + result, NEGATIVE_COVAR_EXPECTED, msg="negative correlation descending sort gives same -5.0" + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_special_floats.py b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_special_floats.py new file mode 100644 index 000000000..f60d7f7a1 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/covarianceSamp/test_window_covarianceSamp_special_floats.py @@ -0,0 +1,542 @@ +""" +Tests for $covarianceSamp with special float values (NaN, Infinity, -Infinity). + +Covers: Infinity as numeric participant, -Infinity, NaN values, +sliding window behavior with special floats, and cumulative window behavior. + +$covarianceSamp semantics for special floats (verified against server 8.2.4): +- NaN and Infinity are numeric values (not ignored like null/missing) +- Single value (N=1) always returns null for covarianceSamp (regardless of value type) +- In non-removable windows (whole partition): Inf produces Infinity, -Inf produces + -Infinity, NaN produces NaN, Inf+(-Inf) cancels to 0.0, all-Inf produces null +- In cumulative windows (unbounded, current): single value = null, then propagates +- In sliding/removable windows: single-value frames return null, pairs with Inf + can return 0.0, clean frames compute normally +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess, assertSuccessNaN +from documentdb_tests.framework.test_constants import ( + FLOAT_INFINITY, + FLOAT_NAN, + FLOAT_NEGATIVE_INFINITY, +) + +# Property [Infinity Non-Removable Window]: Infinity in non-removable whole partition windows + + +def test_covarianceSamp_positive_infinity_whole_partition(collection): + """$covarianceSamp with Infinity in x, whole partition returns Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 4, "result": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_INFINITY}, + ] + assertSuccess(result, expected, msg="Infinity in x produces Infinity for whole partition") + + +def test_covarianceSamp_positive_infinity_in_y(collection): + """$covarianceSamp with Infinity in second expression (y) produces Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_INFINITY, "result": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_INFINITY}, + ] + assertSuccess(result, expected, msg="Infinity in y produces Infinity in whole partition") + + +def test_covarianceSamp_negative_infinity_in_y(collection): + """$covarianceSamp with -Infinity in second expression (y) produces -Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_NEGATIVE_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + { + "_id": 1, + "partition": "A", + "x": 1, + "y": FLOAT_NEGATIVE_INFINITY, + "result": FLOAT_NEGATIVE_INFINITY, + }, + {"_id": 2, "partition": "A", "x": 2, "y": 4, "result": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NEGATIVE_INFINITY}, + ] + assertSuccess(result, expected, msg="-Infinity in y produces -Infinity in whole partition") + + +def test_covarianceSamp_opposing_infinity_same_pair(collection): + """$covarianceSamp with +Inf and -Inf in the same (x, y) pair produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": FLOAT_NEGATIVE_INFINITY}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + { + "_id": 1, + "partition": "A", + "x": FLOAT_INFINITY, + "y": FLOAT_NEGATIVE_INFINITY, + "result": FLOAT_NAN, + }, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="opposing infinities in same pair produce NaN") + + +def test_covarianceSamp_both_inf_signs_separate_rows(collection): + """$covarianceSamp with +Inf and -Inf in separate rows, count_finite>=2, produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": FLOAT_NAN}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="both inf signs in separate rows with count_finite>=2 produce NaN" + ) + + +def test_covarianceSamp_mixed_inf_types_across_rows(collection): + """$covarianceSamp with opposing-sign pair + same-sign pair + finite: count_finite=1 -> null.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "x": 5, "y": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": FLOAT_INFINITY, "result": None}, + {"_id": 2, "partition": "A", "x": 5, "y": FLOAT_NEGATIVE_INFINITY, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": None}, + ] + assertSuccess(result, expected, msg="mixed inf types with count_finite=1 returns null for Samp") + + +def test_covarianceSamp_opposing_inf_signs_across_columns(collection): + """$covarianceSamp with -Inf in x (one row) and +Inf in y (another row) produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": FLOAT_INFINITY}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="opposing inf signs across x and y columns produce NaN") + + +def test_covarianceSamp_negative_infinity_whole_partition(collection): + """$covarianceSamp with -Infinity in x, whole partition returns -Infinity.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_NEGATIVE_INFINITY}, + { + "_id": 2, + "partition": "A", + "x": FLOAT_NEGATIVE_INFINITY, + "y": 4, + "result": FLOAT_NEGATIVE_INFINITY, + }, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NEGATIVE_INFINITY}, + ] + assertSuccess(result, expected, msg="-Infinity in x produces -Infinity for whole partition") + + +def test_covarianceSamp_inf_and_neg_inf_in_same_frame(collection): + """$covarianceSamp with both Infinity and -Infinity in x returns null.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": 10, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 10, "y": 30, "result": None}, + ] + assertSuccess( + result, + expected, + msg="Inf + -Inf in same frame: only 1 finite pair, covarianceSamp returns null", + ) + + +def test_covarianceSamp_all_infinity_values(collection): + """$covarianceSamp where all x values are Infinity returns null (Inf-Inf=NaN internally).""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 20}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": FLOAT_INFINITY, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30, "result": None}, + ] + assertSuccess( + result, expected, msg="All Inf x values: Inf-Inf=NaN internally, server returns null" + ) + + +def test_covarianceSamp_infinity_cumulative_window(collection): + """$covarianceSamp cumulative [unbounded, current] with Infinity in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "current"]}, + expression=["$x", "$y"], + ) + # Row 1: single Inf value -> null (N=1, covSamp undefined) + # Row 2: frame=[(Inf,10),(2,20)] -> null (only 1 finite pair, covSamp needs N-1>=1 finite) + # Row 3: frame=[(Inf,10),(2,20),(3,30)] -> Infinity (2 finite pairs + Inf propagates) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": FLOAT_INFINITY}, + ] + assertSuccess( + result, + expected, + msg="Cumulative: single Inf=null, 2 values with Inf=null, 3 values=Infinity", + ) + + +def test_covarianceSamp_single_infinity_value(collection): + """$covarianceSamp with single Infinity value in whole partition returns null.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + ] + assertSuccess(result, expected, msg="Single Inf value: covarianceSamp returns null") + + +# Property [NaN Non-Removable Window]: NaN in non-removable windows produces NaN + + +def test_covarianceSamp_nan_value_whole_partition(collection): + """$covarianceSamp with NaN in non-removable window produces NaN (NaN is numeric, poisons).""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": FLOAT_NAN, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": FLOAT_NAN, "y": 4, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="NaN is numeric; non-removable window produces NaN") + + +def test_covarianceSamp_nan_in_y_whole_partition(collection): + """$covarianceSamp with NaN in second expression (y) produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + expression=["$x", "$y"], + ) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": 2, "y": FLOAT_NAN, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "x": 3, "y": 6, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="NaN in y expression poisons non-removable window to NaN" + ) + + +# Property [Special Floats Sliding Window]: special floats in removable/sliding windows + + +def test_covarianceSamp_infinity_sliding(collection): + """$covarianceSamp sliding window [-1,0] with Infinity in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(Inf,10)] -> null (single value, covSamp undefined) + # Row 2: frame=[(Inf,10),(2,20)] -> null (only 1 finite pair, covSamp needs >= 2 finite) + # Row 3: frame=[(2,20),(3,30)] -> covSamp = 5.0 + # Row 4: frame=[(3,30),(4,40)] -> covSamp = 5.0 + # Row 5: frame=[(4,40),(5,50)] -> covSamp = 5.0 + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 5.0}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 5.0}, + ] + assertSuccess( + result, + expected, + msg="Sliding window: null for single Inf, null for Inf pair, then recovers", + ) + + +def test_covarianceSamp_neg_infinity_sliding(collection): + """$covarianceSamp sliding window [-1,0] with -Infinity in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(-Inf,10)] -> null (single value, covSamp undefined) + # Row 2: frame=[(-Inf,10),(2,20)] -> null (only 1 finite pair, covSamp needs >= 2 finite) + # Row 3: frame=[(2,20),(3,30)] -> covSamp = 5.0 + # Row 4: frame=[(3,30),(4,40)] -> covSamp = 5.0 + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_NEGATIVE_INFINITY, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 5.0}, + ] + assertSuccess( + result, + expected, + msg="Sliding window: null for single -Inf, null for -Inf pair, then recovers", + ) + + +def test_covarianceSamp_nan_sliding(collection): + """$covarianceSamp sliding window [-1,0] with NaN in first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": FLOAT_NAN, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(NaN,10)] -> null (single value, covSamp undefined) + # Row 2: frame=[(NaN,10),(2,20)] -> null (only 1 finite pair, covSamp needs >= 2 finite) + # Row 3: frame=[(2,20),(3,30)] -> covSamp = 5.0 + # Row 4: frame=[(3,30),(4,40)] -> covSamp = 5.0 + expected = [ + {"_id": 1, "partition": "A", "x": FLOAT_NAN, "y": 10, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 5.0}, + ] + assertSuccessNaN( + result, + expected, + msg="Sliding window: null for single NaN, null for NaN pair, then recovers", + ) + + +def test_covarianceSamp_infinity_centered_sliding(collection): + """$covarianceSamp centered sliding window [-1, 1] with Infinity in middle.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + {"_id": 5, "partition": "A", "x": 5, "y": 50}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 1]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(1,10),(2,20)] (n=2) -> covSamp = 5.0 + # Row 2: frame=[(1,10),(2,20),(Inf,30)] (n=3) -> Infinity (Inf propagates in 3-elem frame) + # Row 3: frame=[(2,20),(Inf,30),(4,40)] (n=3) -> Infinity + # Row 4: frame=[(Inf,30),(4,40),(5,50)] (n=3) -> Infinity + # Row 5: frame=[(4,40),(5,50)] (n=2) -> covSamp = 5.0 (with possible FP rounding) + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": 10, "result": 5.0}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "x": FLOAT_INFINITY, "y": 30, "result": FLOAT_INFINITY}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": FLOAT_INFINITY}, + {"_id": 5, "partition": "A", "x": 5, "y": 50, "result": 5.000000000000021}, + ] + assertSuccess( + result, + expected, + msg="Centered sliding: Inf propagates in 3-elem frames, clean 2-elem frames = 5.0", + ) + + +def test_covarianceSamp_nan_in_y_sliding(collection): + """$covarianceSamp sliding window [-1,0] with NaN in y of first row.""" + docs = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_NAN}, + {"_id": 2, "partition": "A", "x": 2, "y": 20}, + {"_id": 3, "partition": "A", "x": 3, "y": 30}, + {"_id": 4, "partition": "A", "x": 4, "y": 40}, + ] + result = run_window_operator( + collection, + "$covarianceSamp", + docs, + {"documents": [-1, 0]}, + expression=["$x", "$y"], + ) + # Row 1: frame=[(1,NaN)] -> null (single pair, covSamp undefined) + # Row 2: frame=[(1,NaN),(2,20)] -> null (only 1 finite pair, covSamp needs >= 2 finite) + # Row 3: frame=[(2,20),(3,30)] -> covSamp = 5.0 + # Row 4: frame=[(3,30),(4,40)] -> covSamp = 5.0 + expected = [ + {"_id": 1, "partition": "A", "x": 1, "y": FLOAT_NAN, "result": None}, + {"_id": 2, "partition": "A", "x": 2, "y": 20, "result": None}, + {"_id": 3, "partition": "A", "x": 3, "y": 30, "result": 5.0}, + {"_id": 4, "partition": "A", "x": 4, "y": 40, "result": 5.0}, + ] + assertSuccessNaN( + result, + expected, + msg="Sliding window: null for single NaN-y, null for NaN-y pair, then recovers", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py b/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py index 0fcd38e2b..9573e53d4 100644 --- a/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py +++ b/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py @@ -28,6 +28,14 @@ class WindowTestCase(BaseTestCase): {"_id": 5, "partition": "A", "value": 50}, ] +COVAR_DOCS: list[dict[str, Any]] = [ + {"_id": 1, "partition": "A", "x": 1, "y": 2}, + {"_id": 2, "partition": "A", "x": 2, "y": 4}, + {"_id": 3, "partition": "A", "x": 3, "y": 6}, + {"_id": 4, "partition": "A", "x": 4, "y": 8}, + {"_id": 5, "partition": "A", "x": 5, "y": 10}, +] + def run_window_operator( collection,