BYTEA read path encodes the full base64 instead of a truncated preview (builtin caps at 10 KB)

[aesslinger]

Summary

The BYTEA read path in src/extract.rs base64-encodes the entire byte array into the BLOB:<size>:<mime>:<b64> wire format. The built-in driver truncates the base64 payload to the first 10,240 bytes (MAX_BLOB_PREVIEW_SIZE) for the read/preview path, while still reporting the full size in the header. Large BYTEA values therefore produce much larger JSON responses in this plugin than in the built-in, slowing the data grid and inflating transfers.

Builtin behavior (upstream main)

src-tauri/src/drivers/common/blob.rs has two functions:

• encode_blob(data) — truncated for the read/preview path. Used by extract/simple.rs:18 (Type::BYTEA => encode_blob(&b)). For data.len() > 10_240, only &data[..10_240] is base64-encoded; the BLOB: header still reports the true total_size, so the UI knows the real length. Smaller blobs encode in full.
• encode_blob_full(data) — no truncation, used by fetch_blob_as_data_url (file export) so files aren't silently truncated on save.

MAX_BLOB_PREVIEW_SIZE = 10_240 (blob.rs:5). DEFAULT_MAX_BLOB_SIZE = 100 MB (blob.rs:11) is the upload/in-memory guard, separate from the preview cap.

Plugin behavior (this repo)

• src/extract.rs:73 — the Type::BYTEA arm inlines format!("BLOB:{}:application/octet-stream:{}", v.len(), b64) where b64 is the base64 of the entire Vec<u8>. No truncation, no MAX_BLOB_PREVIEW_SIZE. The MIME type is also hardcoded to application/octet-stream rather than sniffed via infer::get (the builtin sniffs magic bytes).
• src/handlers/blob.rs:134 — encode_blob_full (used by fetch_blob_as_data_url) correctly encodes all bytes with MIME sniffing — this path matches the builtin. The gap is only the read/preview path in extract.rs.

Impact

• Severity: Medium. For BYTEA columns holding values over 10 KB, the plugin's data-grid response is much larger than the builtin's (full base64 vs 10 KB preview), with the same BLOB: size header. Functional but wasteful; can cause visible lag on large-bytea tables. The hardcoded MIME (application/octet-stream) also means the UI can't preview image/PDF blobs inline the way the builtin's sniffed MIME allows.
• Not caught by the parity suite (no large-BYTEA read test).

Fix

Port the builtin's truncated encode_blob (with MAX_BLOB_PREVIEW_SIZE = 10_240) and use it in the extract.rs Type::BYTEA arm, sniffing the MIME via infer::get like the builtin. Keep encode_blob_full for the file-export path (blob.rs). Add a unit test in extract_tests.rs: a 20 KB Vec<u8> produces a BLOB: header with size = 20480 but a base64 payload of only the first 10 KB.

Related

• Builtin reference: TabularisDB/tabularis drivers/common/blob.rs::{encode_blob, encode_blob_full, MAX_BLOB_PREVIEW_SIZE} + extract/simple.rs:18.