GCP Notebook Delivery Pipeline

A compact example of managing a GCS-to-BigQuery ingestion notebook as versioned code and publishing it to Google Cloud Storage through GitHub Actions.

Repository layout

.
├── .github/workflows/main.yml
├── notebooks/
│   └── ingest_gcs_to_bq.ipynb
└── README.md

Workflow

Push to master
    ↓
GitHub Actions checks out the repository
    ↓
Authenticate to Google Cloud
    ↓
Upload the configured notebook folder to GCS
    ↓
Open or copy the notebook into the target Vertex AI Workbench environment

The workflow is intentionally separated from notebook execution: GitHub is the source of truth, Cloud Storage is the delivery layer, and the notebook can then be used in the appropriate development, pre-production, or production workbench.

Included notebook

notebooks/ingest_gcs_to_bq.ipynb contains the GCS-to-BigQuery ingestion workflow.

Required repository secrets

Secret	Purpose
GCP_SERVICE_ACCOUNT_KEY	Google Cloud service-account credentials used by the workflow
GH_FOLDER_PATH	Repository folder or file path to upload
GCS_PATH	Destination Cloud Storage bucket/path

Deployment behavior

Every push to the master branch triggers .github/workflows/main.yml. The action authenticates to Google Cloud and uploads the configured source path to the configured GCS destination.

Environment strategy

For separate dev, pqt, and prod environments, use GitHub Environments or environment-specific repositories/secrets so that each deployment points to the correct bucket and service account. Keep the notebook version consistent while isolating credentials and destinations.

Technology

Python notebooks · Google Cloud Storage · BigQuery · Vertex AI Workbench · GitHub Actions

Security note

For a production setup, prefer Workload Identity Federation over long-lived service-account JSON keys and grant the workflow identity access only to the required bucket and resources.