Add native CockroachDB connector support - #3861
viragtripathi wants to merge 5 commits into
Conversation
eb56599 to
31e5672
Compare
31e5672 to
f16c84e
Compare
f16c84e to
2f33f5d
Compare
|
Rebased onto current main and expanded this from a peer-only connector to a full source integration:
Tested against CockroachDB v25.4.13 (e2e suite, single node insecure) and a CockroachDB Cloud v26.2.1 cluster over verify-full TLS for connectivity, validation and schema introspection. A docs PR to peerdb-io/docs will follow once this merges. |
2f33f5d to
0fa9e5a
Compare
|
Updated: the rangefeed validation check queried crdb_internal.cluster_settings, which is gated behind allow_unsafe_internals from v26.1 onward, so on newer clusters the check silently returned nothing and mirror validation passed without verifying rangefeeds. Switched it (and the database variant lookup) to SHOW CLUSTER SETTING, which only needs VIEWCLUSTERSETTING, and added an e2e test covering the rangefeed validation path against a live cluster, including the disabled-setting error case. Planned follow-ups: changefeed sink transports (webhook and Kafka) for deployments that need changefeed job semantics, protected timestamps through large initial scans, and DB Console metrics. The changefeed envelope parsing and checkpointing in this PR are transport independent, so those land as additional transports rather than a redesign. |
0fa9e5a to
9789acb
Compare
9789acb to
b0de202
Compare
|
Continued in #4662 from an in repo branch so CI runs without the external contributor approval gate. Same five commit stack, no code changes. |
Summary
Implements full CockroachDB integration as a source peer, following the same patterns as existing PostgreSQL and MySQL connectors.
What's included
Backend:
Frontend:
Testing performed
Design Decisions
TLS Default: Set to
false(matching Postgres) for easier local development. Cloud users will enable it as needed.SSH Support: Backend implementation included but no UI exposed. CockroachDB Cloud deployments (primary use case) don't require SSH tunnels, keeping the UI simpler and more focused.
Note on SQL syntax
SQL syntax support (
CREATE PEER FROM COCKROACHDB) requires updates to the sqlparser dependency and will be added in a follow-up PR.Files changed
23 files changed, 936 insertions(+), 2 deletions(-)