Skip to content

Add native CockroachDB connector support - #3861

Closed
viragtripathi wants to merge 5 commits into
PeerDB-io:mainfrom
viragtripathi:feat/cockroachdb-integration
Closed

viragtripathi wants to merge 5 commits into
PeerDB-io:mainfrom
viragtripathi:feat/cockroachdb-integration

Conversation

@viragtripathi

@viragtripathi viragtripathi commented Jan 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Implements full CockroachDB integration as a source peer, following the same patterns as existing PostgreSQL and MySQL connectors.

What's included

Backend:

  • Native Go connector (490 lines) using pgx driver with CockroachDB-specific configuration
  • Schema introspection: GetSchemas, GetTables, GetColumns
  • Connection support for local and cloud deployments
  • TLS support (disabled by default for easier local development, matching Postgres behavior)
  • SSH tunnel support in backend (intentionally no UI - CockroachDB Cloud doesn't need SSH)
  • Version validation and variant detection (Cloud/Dedicated/Serverless)
  • Rust analyzer and catalog support for peer configuration

Frontend:

  • Complete UI integration for peer creation
  • Form validation and field helpers
  • CockroachDB logo and proper display names

Testing performed

  • ✅ UI peer creation with form validation
  • ✅ API peer creation and validation
  • ✅ Local CockroachDB connection (insecure mode)
  • ✅ Cloud CockroachDB connection (with TLS)
  • ✅ Schema introspection

Design Decisions

TLS Default: Set to false (matching Postgres) for easier local development. Cloud users will enable it as needed.

SSH Support: Backend implementation included but no UI exposed. CockroachDB Cloud deployments (primary use case) don't require SSH tunnels, keeping the UI simpler and more focused.

Note on SQL syntax

SQL syntax support (CREATE PEER FROM COCKROACHDB) requires updates to the sqlparser dependency and will be added in a follow-up PR.

Files changed

23 files changed, 936 insertions(+), 2 deletions(-)

@CLAassistant

CLAassistant commented Jan 21, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from eb56599 to 31e5672 Compare January 23, 2026 03:08
@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from 31e5672 to f16c84e Compare January 23, 2026 03:36
@serprex
serprex requested a review from jgao54 February 4, 2026 19:11
@serprex
serprex requested a review from Amogh-Bharadwaj April 6, 2026 04:06
@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from f16c84e to 2f33f5d Compare July 23, 2026 02:37
@viragtripathi
viragtripathi requested a review from a team as a code owner July 23, 2026 02:37
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 23, 2026 02:37 — with GitHub Actions Failure
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 23, 2026 02:37 — with GitHub Actions Failure
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 23, 2026 02:37 — with GitHub Actions Failure
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 23, 2026 02:37 — with GitHub Actions Failure
@viragtripathi

Copy link
Copy Markdown
Collaborator Author

Rebased onto current main and expanded this from a peer-only connector to a full source integration:

  • CREATE PEER ... FROM COCKROACHDB now goes through the in-tree parser (following Drop sqlparser-rs (0.45) fork in favor of upstream (0.61) #4145), so the sqlparser fork dependency is gone and I am closing Add CockroachDB PeerType support sqlparser-rs#39
  • QRep replication using AS OF SYSTEM TIME partitioned reads with watermark based partitioning
  • CDC via sinkless changefeeds: resolved timestamp checkpoints, cursor resume with backoff, gapless snapshot handoff via cluster_logical_timestamp(), delete key fallback from the changefeed message key, replay dedup, and SQLSTATE based error classification. Changefeed webhook sink support is a natural follow-up for higher throughput deployments.
  • Unit tests, an e2e suite, and a CockroachDB service container wired into CI, plus Tilt dev environment wiring matching the other source databases

Tested against CockroachDB v25.4.13 (e2e suite, single node insecure) and a CockroachDB Cloud v26.2.1 cluster over verify-full TLS for connectivity, validation and schema introspection. A docs PR to peerdb-io/docs will follow once this merges.

@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from 2f33f5d to 0fa9e5a Compare July 24, 2026 22:41
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 24, 2026 22:41 — with GitHub Actions Failure
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 24, 2026 22:41 — with GitHub Actions Failure
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 24, 2026 22:41 — with GitHub Actions Failure
@viragtripathi
viragtripathi had a problem deploying to external-contributor July 24, 2026 22:41 — with GitHub Actions Failure
@viragtripathi

Copy link
Copy Markdown
Collaborator Author

Updated: the rangefeed validation check queried crdb_internal.cluster_settings, which is gated behind allow_unsafe_internals from v26.1 onward, so on newer clusters the check silently returned nothing and mirror validation passed without verifying rangefeeds. Switched it (and the database variant lookup) to SHOW CLUSTER SETTING, which only needs VIEWCLUSTERSETTING, and added an e2e test covering the rangefeed validation path against a live cluster, including the disabled-setting error case.

Planned follow-ups: changefeed sink transports (webhook and Kafka) for deployments that need changefeed job semantics, protected timestamps through large initial scans, and DB Console metrics. The changefeed envelope parsing and checkpointing in this PR are transport independent, so those land as additional transports rather than a redesign.

@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from 0fa9e5a to 9789acb Compare August 5, 2026 04:53
@viragtripathi

Copy link
Copy Markdown
Collaborator Author

Continued in #4662 from an in repo branch so CI runs without the external contributor approval gate. Same five commit stack, no code changes.

This branch had an error being deployed

1 failed deployment
external-contributor — b0de202f Deployed Aug 5, 2026 by viragtripathi via gate #21696
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants