Skip to content

Load @db/sqlite on first use instead of at import - #23

Merged
lambdalisue merged 2 commits into
mainfrom
fix/lazy-sqlite-ffi
Sep 7, 2026
Merged

lambdalisue merged 2 commits into
mainfrom
fix/lazy-sqlite-ffi

Conversation

@lambdalisue

Copy link
Copy Markdown
Member

Summary

  • Import @db/sqlite inside createSqliteClient instead of at module scope, so the SQLite library is loaded only when a SQLite client is actually created.
  • Load @db/sqlite up front in the integration test, keeping its process-lifetime library out of the resource sanitizer's scope.

Why

@db/sqlite dlopens the SQLite library while it evaluates, so a static import made every consumer of this package pay that dlopen — and inherit its failure modes — even when no SQLite client was ever created. @probitas/probitas re-exports this package from its root module (mod.ts → client.ts → sql.ts → sql/sqlite.ts), which means importing @probitas/probitas at all was enough to trigger it. When the library failed to load, scenarios that never touch SQLite died with it.

That is not hypothetical. In probitas-test/probitas#108, updating the pinned nixpkgs moved CI onto an environment where the prebuilt libsqlite3.so that @db/sqlite downloads segfaults on dlopen, and every scenario subprocess died before running a step — none of them used SQLite.

Auditing every client module in @probitas/probitas (hooking both Deno.dlopen and process.dlopen, one module per process) showed SQLite is the only one that loads anything native at import time:

Module Import Native load at import
sql.ts 119ms FFI dlopen (via sqlite)
sql/sqlite.ts 55ms FFI dlopen
sql/duckdb.ts 36ms none
mongodb, sqs, grpc, connectrpc, mysql, redis, rabbitmq, postgres, deno_kv, graphql, http 7–69ms none

@probitas/client-sql-duckdb has the same static-import shape, but @duckdb/node-api defers its own native binding until DuckDBInstance.create(), so it costs nothing today. It is left alone here.

createSqliteClient already returned Promise<SqliteClient>, so making it async keeps the signature compatible with every caller. A load failure now surfaces as a catchable error from createSqliteClient rather than killing the module graph at import time.

Why not import defer. Deno 2.9.5 does implement deferred imports, and they work on ordinary modules. They cannot help here: @db/sqlite uses top-level await (await dlopen(...) in ffi.ts), and a subgraph containing an async module cannot be deferred — Deno evaluates it eagerly anyway. Verified directly before ruling it out.

The test change. Deferring the load is exactly what moves the dlopen from module evaluation (outside the resource sanitizer's scope) into a test. @db/sqlite keeps its library open for the life of the process, so the sanitizer reported it as a per-test leak. Loading it at the top of the test file puts that one-time load back outside the sanitizer's scope, which is accurate, and keeps leak detection on for everything else.

Test Plan

  • deno task verify on Deno 2.9.5 — 1411 passed, 0 failed, 44 ignored
  • With a deliberately broken DENO_SQLITE_PATH: importing the package no longer dlopens
  • With the same broken path: the dlopen happens on createSqliteClient and surfaces as a catchable SqliteError
  • With a working library: a real client still creates, queries and closes

🤖 Generated with Claude Code

https://claude.ai/code/session_01E6iErRVYiy7LzNGLT5Vzh1

@db/sqlite dlopens the SQLite library as it evaluates, so a static import made
every consumer pay that dlopen — and inherit its failure modes — even when no
SQLite client was ever created. @probitas/probitas re-exports this package from
its root module, so importing `@probitas/probitas` at all was enough to trigger
it, and a broken library took down scenarios that never touch SQLite.

Importing @db/sqlite inside createSqliteClient defers the load to the point
where SQLite is actually wanted. The function already returned a Promise, so
making it async keeps the signature compatible. A load failure now surfaces as
a catchable error from createSqliteClient rather than killing the module graph.

Deferred imports are not an option here: @db/sqlite uses top-level await, and a
subgraph containing it cannot be deferred.

The integration test now loads @db/sqlite up front. The dlopen used to happen
during module evaluation, outside the resource sanitizer's scope; deferring it
moves it inside a test, where the library @db/sqlite keeps open for the life of
the process is reported as a leak.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6iErRVYiy7LzNGLT5Vzh1

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The lazy-loading change preserves the public Promise<SqliteClient> contract while preventing import-time native side effects, and the test adjustment aligns with the sanitizer’s expected resource lifetime semantics.

Pull request overview

This PR defers loading @db/sqlite until a SQLite client is actually created, avoiding process-wide native library dlopen side effects during unrelated imports (notably when @probitas/probitas is imported but SQLite is never used).

Changes:

  • Converted the @db/sqlite module-scope import to a type-only import and moved the runtime import into createSqliteClient() via await import("@db/sqlite").
  • Updated the integration test to eagerly import "@db/sqlite" at file scope so the one-time, process-lifetime native handle is outside Deno’s per-test resource sanitizer scope.
File summaries
File Description
packages/probitas-client-sql-sqlite/client.ts Lazily imports @db/sqlite inside createSqliteClient to prevent native loading at module evaluation time.
packages/probitas-client-sql-sqlite/integration_test.ts Loads @db/sqlite once at test-file evaluation to avoid sanitizer false positives for a process-lifetime resource.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

LocalStack merged its community and pro images, and every image published
after that merge quits with exit code 55 unless LOCALSTACK_AUTH_TOKEN is set.
The test job died at container init before a single test ran, and compose.yaml
hits the same wall locally.

4.14.0 is the last community release, so it starts without a token. This is a
stopgap that freezes the SQS test environment at February 2026; the same pin
and the same follow-up apply to probitas-test/probitas.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6iErRVYiy7LzNGLT5Vzh1
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
packages/probitas-client-sql-sqlite/client.ts 80.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@lambdalisue
lambdalisue merged commit ae9f3c0 into main Sep 7, 2026
3 checks passed
@lambdalisue
lambdalisue deleted the fix/lazy-sqlite-ffi branch September 7, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants