Skip to content

About

Foresee whether a pull request's PostgreSQL migrations lose data or fail, before merge. GitHub Action by Ekbasis.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Ekbasis Migration Guard

Before a pull request is merged: will its new PostgreSQL migrations lose data or fail on the existing database? The guard builds your schema from the base branch's migrations, runs each new migration on it, asks the Ekbasis model the two questions, and posts one comment on the pull request.

Pre-registered evaluation on 184 migrations the model never saw:

Ekbasis Regex rules
Verdict right (risky vs ok) 87.5% 69.6%
Recall on risky migrations 94.6% (53 of 56)
False alarms on safe migrations 15.6% 43.0%

Limits, method and data: docs/MIGRATION_GUARD.md. A low probability is not proof of safety.

Install in one minute

  1. Get an API key at openinterp.org/console. Price: $0.30 per 1M input tokens; a migration is about 4k tokens, so about $0.0012 per migration checked.
  2. Add it to the repository: Settings → Secrets and variables → Actions → New repository secret, name EKBASIS_API_KEY.
  3. Add .github/workflows/migration-guard.yml:
name: migration guard
on:
  pull_request:                     # never pull_request_target
    paths: ["**/migrations/**", "**/*.sql"]
permissions:
  contents: read
  pull-requests: write              # the comment
jobs:
  guard:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0            # the base branch's migrations build the schema
      - uses: OpenInterpretability/migration-guard@v1
        with:
          api-key: ${{ secrets.EKBASIS_API_KEY }}
          # db-stats-url: ${{ secrets.READ_REPLICA_URL }}   # optional: planner statistics only
          # mode: both              # comment | check | both
          # fail-on: risky          # risky | never

Migrations are found by default for Prisma (migrations/*/migration.sql), Diesel (migrations/*/up.sql), golang-migrate and others (*.up.sql, *migrations/*.sql, *migrate/*.sql). Set paths (one glob per line) for your own layout.

What the comment looks like

One comment per pull request, updated on each push. A pull request that drops a column:

Ekbasis migration guard

Risky: at least one migration may lose data or fail on the existing database.

Migration Verdict Why
db/migrations/0002_drop_phone.up.sql (golang-migrate, one transaction) risky may lose existing data (96%)

A pull request that adds a nullable column and an index:

No data loss or failure foreseen.

Migration Verdict Why
db/migrations/0002_add_shipped_at.up.sql (golang-migrate, one transaction) ok no data loss or failure foreseen (lose 0%, fail 1%)

With mode: check or both and fail-on: risky, the job fails on risky and on cannot foresee, so it can be a required status check. The result also goes to the job summary and to the outputs verdict (ok, risky, cannot-foresee) and result (path of the JSON).

Inputs

Input Default
api-key required Ekbasis API key, from a secret
api-url https://openinterp.org/api/v1 the hosted API, or your own server
base-ref the PR's base branch the PR merges into
paths Prisma, Diesel, *.up.sql, *migrations/*.sql one fnmatch glob per line
lost-threshold / fail-threshold 0.5 risky when P(data lost) / P(fails) reaches this
autocommit false true for psql -f semantics; default is one transaction per file
mode both comment, check or both
fail-on risky risky or never
db-stats-url empty read-only replica, from a secret; planner statistics only
postgres-image postgres:16 the scratch server
github-token github.token writes the comment
verdict-cache true reuses the answer for a migration already checked in this pull request (Actions cache, 30 days): no call, no tokens
setup-sql empty SQL file run by the scratch admin before the base's migrations (a schema or role your platform creates outside migrations); read from the base branch
redis auto django/alembic/rails: a throwaway Redis on the generator's isolated network, when the project's migrations need one
policy rule rule: a rule from the PostgreSQL docs decides, the model's second opinion can only mark a migration review; model: the model decides (before 1.6.0)
framework sql alembic, or django (experimental): the framework renders the SQL; the pull request's code runs only in a container with no network and no secrets
on-code warn django/alembic: a migration with code that is not SQL is listed for review by hand (warn) or fails the check (block)
image, setup, workdir, env, migrate-command, alembic-command the framework generator; see the docs. Never put secrets in env

Security

  • Trigger on pull_request, never on pull_request_target with a checkout of the pull request's code. Under pull_request_target, untrusted code runs with your secrets and a write token.
  • Forks get no secrets. On a fork's pull request the key is empty, the verdict is "cannot foresee", and the comment step only warns (the fork's token is read-only); the result is still in the job summary. Choose one:
    • fail-on: ${{ github.event.pull_request.head.repo.full_name == github.repository && 'risky' || 'never' }}
    • or run the job only for your own branches: if: github.event.pull_request.head.repo.full_name == github.repository
  • Minimal permissions: contents: read and pull-requests: write, nothing else.
  • The pull request's SQL is untrusted, and it runs, but only in a throwaway PostgreSQL container on an internal Docker network (no route out, no published port), as a role made for the check that is not a superuser and cannot create databases or roles. COPY ... PROGRAM, pg_read_file, untrusted languages and the like fail with "permission denied", and the guard reports "cannot foresee". It never retries with more privileges.
  • Pinned code. Each release of this action pins one release of ekbasis by full commit SHA (see CHANGELOG.md and .github/ekbasis-pin). For the strictest setup, pin this action by its own commit SHA instead of @v1.

What is sent to the API

  • the schema: DDL of the tables a new migration names and of the tables linked to them by foreign keys, as built from your migration files;
  • the new migration's SQL, with comments removed;
  • optional planner statistics when you set db-stats-url: estimated rows per table (pg_class.reltuples), NULL fraction and distinct count per column (pg_stats.null_frac, n_distinct).

Never data values. No row is read, and most_common_vals is never queried. The API key and the replica URL travel only as environment variables, and the replica's URL and password are removed from any error the guard prints.

Self-host with the open weights

The model is open (Apache-2.0): Ekbasis-27B (bf16, 80 GB GPU), FP8 (48 GB) and INT4 (32 GB). Serve it as in the ekbasis quick start, then point the action at it:

      - uses: OpenInterpretability/migration-guard@v1
        with:
          api-url: https://ekbasis.internal.example.com   # reachable from the runner (e.g. a self-hosted runner)
          api-key: ${{ secrets.EKBASIS_API_KEY }}         # any value if your server does not check keys

Your schema and SQL then never reach OpenInterpretability.

License

Apache-2.0. See LICENSE.

About

Foresee whether a pull request's PostgreSQL migrations lose data or fail, before merge. GitHub Action by Ekbasis.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors