Skip to content

feat: simd double unary math ops #19768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
clintropolis wants to merge 9 commits into
base: master
Choose a base branch
from
Open

feat: simd double unary math ops #19768

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3251e17
feat: simd double unary math ops
clintropolis Jul 27, 2026
18228ec
add druid.expressions.useVectorMathApi to gate mathlib ops
clintropolis Jul 30, 2026
c7203e1
Merge remote-tracking branch 'upstream/master' into simd-double-math-ops
clintropolis Jul 30, 2026
570a9b2
spelling exceptions
clintropolis Aug 4, 2026
4367685
Merge remote-tracking branch 'upstream/master' into simd-double-math-ops
clintropolis Aug 14, 2026
1f7647b
Merge remote-tracking branch 'upstream/master' into simd-double-math-ops
clintropolis Aug 15, 2026
83a7e8b
adjust for review
clintropolis Aug 27, 2026
2b81c4e
Merge remote-tracking branch 'upstream/master' into simd-double-math-ops
clintropolis Aug 27, 2026
2032adb
better equivalent check
clintropolis Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 55 additions & 2 deletions benchmarks/src/test/java/org/apache/druid/benchmark/query/SqlExpressionBenchmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,34 @@ public class SqlExpressionBenchmark extends SqlBaseQueryBenchmark
"SELECT SUM(ABS(long1 - long4)) FROM expressions",
// 69,70: unary sqrt on double and long inputs (long input exercises the SIMD widening path)
"SELECT SUM(SQRT(double1)) FROM expressions",
"SELECT SUM(SQRT(long1)) FROM expressions"
"SELECT SUM(SQRT(long1)) FROM expressions",
// 71-96: VO_MATHLIB unary ops exposed via SQL and covered by SIMD dispatch.
"SELECT SUM(LN(double1)) FROM expressions",
"SELECT SUM(LN(long1)) FROM expressions",
"SELECT SUM(EXP(double1)) FROM expressions",
"SELECT SUM(EXP(long1)) FROM expressions",
"SELECT SUM(LOG10(double1)) FROM expressions",
"SELECT SUM(LOG10(long1)) FROM expressions",
"SELECT SUM(CBRT(double1)) FROM expressions",
"SELECT SUM(CBRT(long1)) FROM expressions",
"SELECT SUM(SIN(double1)) FROM expressions",
"SELECT SUM(SIN(long1)) FROM expressions",
"SELECT SUM(COS(double1)) FROM expressions",
"SELECT SUM(COS(long1)) FROM expressions",
"SELECT SUM(TAN(double1)) FROM expressions",
"SELECT SUM(TAN(long1)) FROM expressions",
"SELECT SUM(ASIN(double1)) FROM expressions",
"SELECT SUM(ASIN(long1)) FROM expressions",
"SELECT SUM(ACOS(double1)) FROM expressions",
"SELECT SUM(ACOS(long1)) FROM expressions",
"SELECT SUM(ATAN(double1)) FROM expressions",
"SELECT SUM(ATAN(long1)) FROM expressions",
"SELECT SUM(SINH(double1)) FROM expressions",
"SELECT SUM(SINH(long1)) FROM expressions",
"SELECT SUM(COSH(double1)) FROM expressions",
"SELECT SUM(COSH(long1)) FROM expressions",
"SELECT SUM(TANH(double1)) FROM expressions",
"SELECT SUM(TANH(long1)) FROM expressions"
);

@Param({
Expand Down Expand Up @@ -266,7 +293,33 @@ public class SqlExpressionBenchmark extends SqlBaseQueryBenchmark
"67",
"68",
"69",
"70"
"70",
"71",
"72",
"73",
"74",
"75",
"76",
"77",
"78",
"79",
"80",
"81",
"82",
"83",
"84",
"85",
"86",
"87",
"88",
"89",
"90",
"91",
"92",
"93",
"94",
"95",
"96"
})
private String query;

Expand Down
1 change: 1 addition & 0 deletions docs/configuration/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -652,6 +652,7 @@ the `expression` aggregator/post-aggregator, and any SQL functions that lower to
|`druid.expressions.homogenizeNullMultiValueStringArrays`|If true, multi-value string expression input values of `null`, `[]`, and `[null]` are all coerced to `[null]`. Provided for backwards compatibility with Druid 0.22 and earlier. If false (the default), this coercion only happens when single-value expressions are implicitly mapped across multi-value rows, so the single-valued expression is evaluated with an input of `null`.|false|
|`druid.expressions.allowVectorizeFallback`|If true, the vectorized query engine handles expressions without a native vectorized implementation using a fallback processor that invokes the scalar expression evaluator in a loop. If false, such expressions cannot be vectorized and the query falls back to the non-vectorized engine.|true|
|`druid.expressions.useVectorApi`|If true, vectorized expression vector processors and numeric vector aggregators dispatch to SIMD specializations backed by the JDK incubator Vector API (`jdk.incubator.vector`) where available. Requires `--add-modules=jdk.incubator.vector` on the JVM command line (see [strong encapsulation](../operations/java.md#strong-encapsulation)). Off by default while the Vector API remains an incubator JDK feature.|false|
|`druid.expressions.useVectorMathApi`|When `useVectorApi` is true, controls whether math expressions whose SIMD path is backed by the JDK's VO_MATHLIB routing (Intel SVML on x86, SLEEF on Arm) dispatch to that SIMD path. Results can differ from the scalar `Math.<op>` by a few ulps (the parity test bounds this at 2 ulps), and the exact bits can shift once the JIT promotes the SIMD loop from C1 to C2 during a long-running query. In general applications should not rely on bit-for-bit equality of floating-point results, and round or truncate first when using for grouping or equality comparisons. This flag exists as an escape hatch for legacy queries that rely on `Math.<op>` bits without rounding. Has no effect unless `useVectorApi` is also true. On by default when `useVectorApi` is enabled.|true|

### Double column storage

Expand Down
25 changes: 22 additions & 3 deletions processing/src/main/java/org/apache/druid/math/expr/ExpressionProcessing.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,19 +45,19 @@ public class ExpressionProcessing
@VisibleForTesting
public static void initializeForTests()
{
INSTANCE = new ExpressionProcessingConfig(null, null, null, null);
INSTANCE = new ExpressionProcessingConfig(null, null, null, null, null);
}

@VisibleForTesting
public static void initializeForHomogenizeNullMultiValueStrings()
{
INSTANCE = new ExpressionProcessingConfig(null, true, null, null);
INSTANCE = new ExpressionProcessingConfig(null, true, null, null, null);
}

@VisibleForTesting
public static void initializeForVectorApiTests()
{
INSTANCE = new ExpressionProcessingConfig(null, null, null, true);
INSTANCE = new ExpressionProcessingConfig(null, null, null, true, true);

@FrankChen021 FrankChen021 Aug 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep exact vector-math tests from enabling approximate SIMD

initializeForVectorApiTests() enables approximate math SIMD, but the inherited exact-comparison suite uses Assert.assertEquals for functions such as sin and exp. This PR documents VO_MATHLIB ULP differences and tier-transition changes, so supported JDK/hardware combinations can now fail these tests. Keep this helper's math flag off, or make the consistency assertions tolerance-aware.

@gianm gianm Aug 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it could be a legit issue with VectorExprResultConsistencyVectorApiTest, possibly dependent on the order in which things are run.

@FrankChen021 FrankChen021 Aug 15, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue remains at current HEAD: initializeForVectorApiTests() still enables useVectorMathApi at line 60, while the inherited consistency suite compares SIMD and scalar math with exact equality. Please disable this flag in that helper or make the affected assertions tolerance-aware.

Reviewed 42 of 42 changed files.

}

/**
Expand Down Expand Up @@ -99,6 +99,25 @@ public static boolean useVectorApi()
return INSTANCE.useVectorApi();
}

/**
* Whether SIMD dispatch is allowed for math ops backed by the JDK's VO_MATHLIB path (LOG, EXP, SIN, etc). On by
* default whenever {@link #useVectorApi()} is on; can be turned off independently via
* {@link ExpressionProcessingConfig#USE_VECTOR_MATH_API}=false. Has no effect unless {@link #useVectorApi()} is
* also on.
*
* <p>These ops route through Intel SVML / Arm SLEEF once the JIT compiles the vector loop to C2; before that
* compilation, they fall back to per-lane {@link Math} calls. The two paths can differ by a few ulps (bounded
* at 2 ulps by {@code SimdVoMathlibParityTest.MAX_ULPS}), so a long-running query can produce different bits
* for the same input across the C1→C2 tier transition. Bit-for-bit equality of floating-point results is
* fragile in general, but this flag is an escape hatch for legacy queries relying on the scalar {@link Math} bits
* directly.
*/
public static boolean useVectorMathApi()
{
checkInitialized();
return INSTANCE.useVectorMathApi();
}

private static void checkInitialized()
{
// this should only be null in a unit test context, in production this will be injected by the null handling module
Expand Down
13 changes: 12 additions & 1 deletion processing/src/main/java/org/apache/druid/math/expr/ExpressionProcessingConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ public class ExpressionProcessingConfig
"druid.expressions.homogenizeNullMultiValueStringArrays";
public static final String ALLOW_VECTORIZE_FALLBACK = "druid.expressions.allowVectorizeFallback";
public static final String USE_VECTOR_API = "druid.expressions.useVectorApi";
public static final String USE_VECTOR_MATH_API = "druid.expressions.useVectorMathApi";

@JsonProperty("processArraysAsMultiValueStrings")
private final boolean processArraysAsMultiValueStrings;
Expand All @@ -48,12 +49,16 @@ public class ExpressionProcessingConfig
@JsonProperty("useVectorApi")
private final boolean useVectorApi;

@JsonProperty("useVectorMathApi")
private final boolean useVectorMathApi;

@JsonCreator
public ExpressionProcessingConfig(
@JsonProperty("processArraysAsMultiValueStrings") @Nullable Boolean processArraysAsMultiValueStrings,
@JsonProperty("homogenizeNullMultiValueStringArrays") @Nullable Boolean homogenizeNullMultiValueStringArrays,
@JsonProperty("allowVectorizeFallback") @Nullable Boolean allowVectorizeFallback,
@JsonProperty("useVectorApi") @Nullable Boolean useVectorApi
@JsonProperty("useVectorApi") @Nullable Boolean useVectorApi,
@JsonProperty("useVectorMathApi") @Nullable Boolean useVectorMathApi
)
{
this.processArraysAsMultiValueStrings = getWithPropertyFallbackFalse(
Expand All @@ -70,6 +75,7 @@ public ExpressionProcessingConfig(
"true"
);
this.useVectorApi = getWithPropertyFallbackFalse(useVectorApi, USE_VECTOR_API);
this.useVectorMathApi = getWithPropertyFallback(useVectorMathApi, USE_VECTOR_MATH_API, "true");
}

public boolean processArraysAsMultiValueStrings()
Expand All @@ -92,6 +98,11 @@ public boolean useVectorApi()
return useVectorApi;
}

public boolean useVectorMathApi()
{
return useVectorMathApi;
}

private static boolean getWithPropertyFallbackFalse(@Nullable Boolean value, String property)
{
return getWithPropertyFallback(value, property, "false");
Expand Down
8 changes: 4 additions & 4 deletions ...ain/java/org/apache/druid/math/expr/vector/SimpleVectorMathBivariateProcessorFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ protected SimpleVectorMathBivariateProcessorFactory(
@Override
public final ExprVectorProcessor<long[]> longsProcessor(Expr.VectorInputBindingInspector inspector, Expr left, Expr right)
{
if (simdOp != null && simdOp.supportsLongLong() && ExpressionProcessing.useVectorApi()) {
if (simdOp != null && simdOp.supportsLongLong() && simdOp.isSimdEnabled()) {
return SimdProcessors.makeLongLong(
left.asVectorProcessor(inspector),
right.asVectorProcessor(inspector),
Expand All @@ -100,7 +100,7 @@ public final ExprVectorProcessor<double[]> longDoubleProcessor(
Expr right
)
{
if (simdOp != null && ExpressionProcessing.useVectorApi()) {
if (simdOp != null && simdOp.isSimdEnabled()) {
return SimdProcessors.makeLongDouble(
left.asVectorProcessor(inspector),
right.asVectorProcessor(inspector),
Expand All @@ -122,7 +122,7 @@ public final ExprVectorProcessor<double[]> doubleLongProcessor(
Expr right
)
{
if (simdOp != null && ExpressionProcessing.useVectorApi()) {
if (simdOp != null && simdOp.isSimdEnabled()) {
return SimdProcessors.makeDoubleLong(
left.asVectorProcessor(inspector),
right.asVectorProcessor(inspector),
Expand All @@ -144,7 +144,7 @@ public final ExprVectorProcessor<double[]> doublesProcessor(
Expr right
)
{
if (simdOp != null && ExpressionProcessing.useVectorApi()) {
if (simdOp != null && simdOp.isSimdEnabled()) {
return SimdProcessors.makeDoubleDouble(
left.asVectorProcessor(inspector),
right.asVectorProcessor(inspector),
Expand Down
8 changes: 5 additions & 3 deletions ...a/org/apache/druid/math/expr/vector/SimpleVectorMathUnivariateDoubleProcessorFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@
*
* If a non-null {@link SimdSupportedUnaryOp} is supplied to the constructor and
* {@link ExpressionProcessing#useVectorApi()} is true, this factory will return SIMD-specialized processors backed
* by the JDK incubator {@code jdk.incubator.vector} API instead of the standard scalar implementations.
* by the JDK incubator {@code jdk.incubator.vector} API instead of the standard scalar implementations. For ops
* whose SIMD path routes through VO_MATHLIB (SVML/SLEEF), dispatch additionally requires
* {@link ExpressionProcessing#useVectorMathApi()}, see the flag javadoc for the semantic caveats.
*/
public class SimpleVectorMathUnivariateDoubleProcessorFactory extends VectorMathUnivariateDoubleProcessorFactory
{
Expand Down Expand Up @@ -67,7 +69,7 @@ protected SimpleVectorMathUnivariateDoubleProcessorFactory(
@Override
public final ExprVectorProcessor<double[]> longProcessor(Expr.VectorInputBindingInspector inspector, Expr arg)
{
if (simdOp != null && ExpressionProcessing.useVectorApi()) {
if (simdOp != null && simdOp.isSimdEnabled()) {
return SimdProcessors.makeLongToDoubleUnary(arg.asVectorProcessor(inspector), simdOp, longFunction);
}
return new DoubleUnivariateLongFunctionVectorProcessor(
Expand All @@ -79,7 +81,7 @@ public final ExprVectorProcessor<double[]> longProcessor(Expr.VectorInputBinding
@Override
public final ExprVectorProcessor<double[]> doubleProcessor(Expr.VectorInputBindingInspector inspector, Expr arg)
{
if (simdOp != null && ExpressionProcessing.useVectorApi()) {
if (simdOp != null && simdOp.isSimdEnabled()) {
return SimdProcessors.makeDoubleUnary(arg.asVectorProcessor(inspector), simdOp, doubleFunction);
}
return new DoubleUnivariateDoubleFunctionVectorProcessor(
Expand Down
30 changes: 15 additions & 15 deletions processing/src/main/java/org/apache/druid/math/expr/vector/VectorMathProcessors.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -554,7 +554,7 @@ public static final class Acos extends SimpleVectorMathUnivariateDoubleProcessor

public Acos()
{
super(Math::acos, Math::acos);
super(Math::acos, Math::acos, SimdSupportedUnaryOp.ACOS);
}
}

Expand All @@ -564,7 +564,7 @@ public static final class Asin extends SimpleVectorMathUnivariateDoubleProcessor

public Asin()
{
super(Math::asin, Math::asin);
super(Math::asin, Math::asin, SimdSupportedUnaryOp.ASIN);
}
}

Expand All @@ -574,7 +574,7 @@ public static final class Atan extends SimpleVectorMathUnivariateDoubleProcessor

public Atan()
{
super(Math::atan, Math::atan);
super(Math::atan, Math::atan, SimdSupportedUnaryOp.ATAN);
}
}

Expand All @@ -584,7 +584,7 @@ public static final class Cos extends SimpleVectorMathUnivariateDoubleProcessorF

public Cos()
{
super(Math::cos, Math::cos);
super(Math::cos, Math::cos, SimdSupportedUnaryOp.COS);
}
}

Expand All @@ -594,7 +594,7 @@ public static final class Cosh extends SimpleVectorMathUnivariateDoubleProcessor

public Cosh()
{
super(Math::cosh, Math::cosh);
super(Math::cosh, Math::cosh, SimdSupportedUnaryOp.COSH);
}
}

Expand All @@ -617,7 +617,7 @@ public static final class Sin extends SimpleVectorMathUnivariateDoubleProcessorF

public Sin()
{
super(Math::sin, Math::sin);
super(Math::sin, Math::sin, SimdSupportedUnaryOp.SIN);
}
}

Expand All @@ -627,7 +627,7 @@ public static final class Sinh extends SimpleVectorMathUnivariateDoubleProcessor

public Sinh()
{
super(Math::sinh, Math::sinh);
super(Math::sinh, Math::sinh, SimdSupportedUnaryOp.SINH);
}
}

Expand All @@ -637,7 +637,7 @@ public static final class Tan extends SimpleVectorMathUnivariateDoubleProcessorF

public Tan()
{
super(Math::tan, Math::tan);
super(Math::tan, Math::tan, SimdSupportedUnaryOp.TAN);
}
}

Expand All @@ -647,7 +647,7 @@ public static final class Tanh extends SimpleVectorMathUnivariateDoubleProcessor

public Tanh()
{
super(Math::tanh, Math::tanh);
super(Math::tanh, Math::tanh, SimdSupportedUnaryOp.TANH);
}
}

Expand All @@ -657,7 +657,7 @@ public static final class Cbrt extends SimpleVectorMathUnivariateDoubleProcessor

public Cbrt()
{
super(Math::cbrt, Math::cbrt);
super(Math::cbrt, Math::cbrt, SimdSupportedUnaryOp.CBRT);
}
}

Expand Down Expand Up @@ -687,7 +687,7 @@ public static final class Exp extends SimpleVectorMathUnivariateDoubleProcessorF

public Exp()
{
super(Math::exp, Math::exp);
super(Math::exp, Math::exp, SimdSupportedUnaryOp.EXP);
}
}

Expand All @@ -697,7 +697,7 @@ public static final class Expm1 extends SimpleVectorMathUnivariateDoubleProcesso

public Expm1()
{
super(Math::expm1, Math::expm1);
super(Math::expm1, Math::expm1, SimdSupportedUnaryOp.EXPM1);
}
}

Expand All @@ -720,7 +720,7 @@ public static final class Log extends SimpleVectorMathUnivariateDoubleProcessorF

public Log()
{
super(Math::log, Math::log);
super(Math::log, Math::log, SimdSupportedUnaryOp.LOG);
}
}

Expand All @@ -730,7 +730,7 @@ public static final class Log10 extends SimpleVectorMathUnivariateDoubleProcesso

public Log10()
{
super(Math::log10, Math::log10);
super(Math::log10, Math::log10, SimdSupportedUnaryOp.LOG10);
}
}
public static final class Log1p extends SimpleVectorMathUnivariateDoubleProcessorFactory
Expand All @@ -739,7 +739,7 @@ public static final class Log1p extends SimpleVectorMathUnivariateDoubleProcesso

public Log1p()
{
super(Math::log1p, Math::log1p);
super(Math::log1p, Math::log1p, SimdSupportedUnaryOp.LOG1P);
}
}

Expand Down
Loading
Loading