Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ This repository (`google/osv.dev`) contains the backend services, database model
It is structured as a multi-language monorepo:
- `go/`: Go services and utilities (importers, exporter, internal libraries). **This is the primary target for active migrations from Python.**
- `osv/`: Core Python library containing models, repository helpers, and ecosystem-specific logic. *Note: Some parts are deprecated as we migrate logic to Go.*
- `gcp/`: GCP deployment configurations, Cloud Functions, API server, and workers. (Website frontend uses `pnpm` and Hugo).
- `gcp/`: GCP deployment configurations, Cloud Functions, and workers.
- `website/`: Frontend assets for the osv.dev website (`frontend3` using `pnpm`) and blog posts (`blog` using Hugo).
- `vulnfeeds/`: Vulnerability feed utilities (*independent Go module*).
- `bindings/`: API bindings (*contains an independent Go module under `bindings/go`*).

Expand Down Expand Up @@ -73,7 +74,7 @@ The project uses `poetry` for Python dependency management, `pnpm` for website f
```
- **Install Website Dependencies** (for frontend development):
```bash
cd gcp/website/frontend3 && pnpm install
cd website/frontend3 && pnpm install
```
- **Initialize Git Submodules**:
```bash
Expand Down Expand Up @@ -302,9 +303,9 @@ Contains deployment setups, workers running in GKE, Cloud Functions, and the use
- **Deployment Target**: **Google Cloud Run** (managed via Cloud Deploy pipeline `osv-api` deploying to `osv-grpc-backend`).
- *Note*: Fully migrated from Python to Go. Protobuf definitions and descriptor files are located under `proto/v1/`.

### 2. Website (`go/cmd/website/`, `gcp/website/`)
- **Status**: **Active (Go / Python)**.
- Migrated to Go backend under `go/cmd/website/` and `go/internal/website/`. Frontend assets (Hugo blog, pnpm frontend3) are located under `website/` (symlinked to `gcp/website/`).
### 2. Website (`go/cmd/website/`, `website/`)
- **Status**: **Active (Go)**.
- Fully migrated to Go backend under `go/cmd/website/` and `go/internal/website/`. Frontend assets (Hugo blog, pnpm frontend3) are located under `website/`.
- **Deployment Target**: **Google Cloud Run** (managed via Cloud Deploy pipeline `osv-website`).

### 3. Workers (`gcp/workers/`)
Expand Down
5 changes: 1 addition & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ lib-tests: ## Run core Python library tests
vanir-signatures-tests: ## Run Vanir signatures tests
cd gcp/workers/vanir_signatures && ./run_tests.sh

website-tests: ## Run legacy Python website tests
cd gcp/website && ./run_tests.sh

vulnfeed-tests: ## Run Go vulnfeeds tests
cd vulnfeeds && ./run_tests.sh

Expand Down Expand Up @@ -118,7 +115,7 @@ run-api-server-test:
@cd go && go build -o ./api-devserver ./cmd/api-devserver && (GOOGLE_CLOUD_PROJECT=oss-vdb-test OSV_VULNERABILITIES_BUCKET=osv-test-vulnerabilities ./api-devserver $(ARGS); EXIT_CODE=$$?; rm -f ./api-devserver; exit $$EXIT_CODE)

# TODO: API integration tests.
all-tests: lib-tests website-tests vulnfeed-tests bindings-tests go-tests ## Run all tests
all-tests: lib-tests vulnfeed-tests bindings-tests go-tests ## Run all tests

reimport-tui: ## Run the reimport TUI tool
test -f $(HOME)/.config/gcloud/application_default_credentials.json || (echo "GCP Application Default Credentials not set, try 'gcloud auth application-default login'"; exit 1)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,12 @@ consists of:
| `gcp/datastore` | The datastore index file (`index.yaml`) |
| `gcp/functions` | The Cloud Function for publishing PyPI vulnerabilities (maintained, but not developed) |
| `gcp/indexer` | The determine version `indexer` |
| `gcp/website` | Frontend assets for the osv.dev website (in `frontend3`) and blog posts (in `blog`) |
| `gcp/workers/` | Python workers (`vanir_signatures` and `oss_fuzz_worker`) |
| `go/` | Go module for shared libraries and commands (`cmd/api`, `cmd/website`, `cmd/importer`, `cmd/worker`, `cmd/exporter`, `cmd/recoverer`, `cmd/relations`, etc.) |
| `osv/` | The core OSV Python library, used in basically all Python services <br /> OSV ecosystem package versioning helpers in `ecosystems/` <br /> Datastore model definitions in `models.py` |
| `tools/` | Misc scripts/tools, mostly intended for development (datastore stuff, linting) <br /> The `indexer-api-caller` for indexer calling |
| `vulnfeeds/` | Go module for (mostly) the NVD CVE conversion <br /> The Alpine feed converter (`cmd/alpine`) <br /> The Debian feed converter (`tools/debian`, which is written in Python) |
| `website/` | Frontend assets for the osv.dev website (in `frontend3`) and blog posts (in `blog`) |


You'll need to check out submodules as well for many local building steps to
Expand Down
5 changes: 0 additions & 5 deletions cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,6 @@ steps:
args: ['gcloud', 'builds', 'submit', '--region=${LOCATION}', '--config=gcp/workers/cloudbuild.yaml', '.']
waitFor: ['init']

- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk:slim'
id: 'website-tests'
args: ['gcloud', 'builds', 'submit', '--region=${LOCATION}', '--config=gcp/website/cloudbuild.yaml', '.']
waitFor: ['init']

- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk:slim'
id: 'vulnfeeds-tests'
args: ['gcloud', 'builds', 'submit', '--region=${LOCATION}', '--config=vulnfeeds/cloudbuild.yaml', '.']
Expand Down
6 changes: 3 additions & 3 deletions deployment/build-and-stage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -462,13 +462,13 @@ steps:
corepack enable pnpm
pnpm install --frozen-lockfile --ignore-scripts
pnpm run build:prod
dir: 'gcp/website/frontend3'
dir: 'website/frontend3'
id: 'build-frontend3'
waitFor: ['setup']

- name: 'gcr.io/oss-vdb/ci'
args: ['hugo', '--buildFuture', '-d', '../dist/static/blog']
dir: 'gcp/website/blog'
dir: 'website/blog'
id: 'build-hugo'
waitFor: ['setup']

Expand All @@ -483,7 +483,7 @@ steps:
'-t', 'gcr.io/oss-vdb/osv-website:latest', '-t', 'gcr.io/oss-vdb/osv-website:$COMMIT_SHA',
'--target', 'website',
'--build-context', 'bindings=../bindings',
'--build-context', 'website-dist=../gcp/website/dist',
'--build-context', 'website-dist=../website/dist',
'--build-context', 'docs=../docs',
'-f', 'Dockerfile', '--cache-from', 'gcr.io/oss-vdb/osv-website:latest', '--pull', '.']
dir: 'go'
Expand Down
3 changes: 1 addition & 2 deletions docs/contributing/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ The [API server](../api/index.md) (hosted at `api.osv.dev`, source code in [`go/

## Website

The [main web UI](https://osv.dev) (source code in [`gcp/website`](../../gcp/website)) also runs on [Cloud Run], and is served through [Cloud Load Balancing].
The [main web UI](https://osv.dev) (server source code in [`go/cmd/website`](../../go/cmd/website) and frontend assets in [`website`](../../website)) also runs on [Cloud Run], and is served through [Cloud Load Balancing].

[Cloud Run]: https://cloud.google.com/run
[Cloud Load Balancing]: https://cloud.google.com/load-balancing
5 changes: 0 additions & 5 deletions gcp/website/.gitignore

This file was deleted.

69 changes: 0 additions & 69 deletions gcp/website/Dockerfile

This file was deleted.

138 changes: 0 additions & 138 deletions gcp/website/auth.py

This file was deleted.

17 changes: 0 additions & 17 deletions gcp/website/build.sh

This file was deleted.

Loading
Loading