Add date_bin aggregation and telemetry key discovery to the Table Mode DAOs#115
Conversation
…e DAOs Implements the aggregation and key-discovery parts of the ThingsBoard Table Mode integration on top of the merged BaseDao / TimeseriesDao / LatestDao: - IoTDBTableTimeseriesDao: date_bin bucketed aggregation for the aggregated read path, covering fixed-width (millisecond) buckets and calendar buckets (WEEK / MONTH / QUARTER) with time-zone-aware boundaries computed on the Java side to avoid time-zone drift. - IoTDBTableLatestDao: telemetry key discovery (findAllKeysByEntityIds / findAllKeysByTenant) now unions the telemetry and telemetry_latest sources so latest-only keys are also discovered. Adds unit tests and container integration tests. Signed-off-by: Zihan Dai <99155080+PDGGK@users.noreply.github.com>
|
The aggregation implementation reads well overall — the 1. Nested result set in In both the milliseconds and calendar aggregation paths, // milliseconds path — outer dataSet still open:
try (ITableSession session = tableSessionPool.getSession();
SessionDataSet dataSet = session.executeQueryStatement(sql)) {
while (row.next()) {
KvEntry value = aggregatedEntry(aggregation, row,
new SumReSumContext(session, ...)); // may call exactLongSum
// exactLongSum: session.executeQueryStatement(reSumSql) — second open RS
}
}Whether IoTDB's The fix is straightforward: acquire a separate session from the pool inside 2. Empty entity list in If CI is still running on this commit — waiting for |
exactLongSum re-queried the bucket's raw long_v values on the same ITableSession that was iterating the outer aggregate result set. IoTDB Table Mode does not guarantee two concurrently open result sets on one session, so the re-sum fallback now checks out its own pooled session and SumReSumContext no longer carries the session. Addresses review feedback on apache#115. Signed-off-by: Zihan Dai <99155080+PDGGK@users.noreply.github.com>
|
Thanks for the careful review, @CritasWang. 1. Nested result set in 2. Empty entity list in |
|
Both resolved — thanks. 1. Nested result set: the fix is correct. 2. Empty entity list: my mistake — you're right, CI is fully green on 970feca (CodeQL, code-analyze java/go/ts, compile-check 8/11/17/21, dependency-check, todo-check). LGTM — ready to merge from my side. |
Summary
Implements the aggregation and telemetry key-discovery parts of the ThingsBoard Table Mode integration, on top of the merged BaseDao / TimeseriesDao / LatestDao (#110, #113).
Changes
IoTDBTableTimeseriesDao):date_binbucketed aggregation covering fixed-width (millisecond) buckets and calendar buckets (WEEK / MONTH / QUARTER). Calendar bucket boundaries are computed on the Java side (viaTimeUtils) to avoid time-zone drift.IoTDBTableLatestDao):findAllKeysByEntityIds/findAllKeysByTenantnow union thetelemetryandtelemetry_latestsources, so keys that only ever appear in the latest overlay are also discovered.Tests
IoTDBTableTimeseriesDaoTest(74) andIoTDBTableLatestDaoTest(40) cover the aggregation and key-discovery logic.IoTDBTableTimeseriesAggregationIT(11),IoTDBTableLatestDaoIT(19),IoTDBTableTimeseriesDaoIT(10). All green.