diff --git a/reference/cli/authentication.md b/reference/cli/authentication.md index c2d9487ac..f24f54bc4 100644 --- a/reference/cli/authentication.md +++ b/reference/cli/authentication.md @@ -71,6 +71,10 @@ The CLI supports loading environment variables from your shell environment (or o - `HARPER_CLI_TARGET` (or `CLI_TARGET`) - Sets the default `target` for CLI commands. - `HARPER_CLI_USERNAME` (or `CLI_TARGET_USERNAME`) - Harper admin username for the target. - `HARPER_CLI_PASSWORD` (or `CLI_TARGET_PASSWORD`) - Harper admin password for the target. +- `HARPER_CLI_REFRESH_TOKEN` (or `CLI_TARGET_REFRESH_TOKEN`) - Long-lived token the CLI exchanges for an operation token on each run. Available since v5.2.0. +- `HARPER_CLI_OPERATION_TOKEN` (or `CLI_TARGET_OPERATION_TOKEN`) - Short-lived operation token, if you would rather supply one directly. Available since v5.2.0. + +**Prefer tokens over a password in CI.** A refresh token is scoped to authentication, can be revoked without changing the account password, and — unlike `HARPER_CLI_PASSWORD` — cannot be reused to log in interactively. See [Token credentials for CI/CD](#token-credentials-for-cicd) below. **Example `.env` file**: @@ -129,6 +133,51 @@ harper deploy target=https://prod-server.com:9925 replicated=true harper restart target=https://prod-server.com:9925 replicated=true ``` +##### Token credentials for CI/CD + +Available since: v5.2.0 + +Rather than storing an admin password in your CI provider, log in once locally and hand CI a **refresh token**. The CLI mints a fresh, short-lived operation token from it on every run, so the only durable secret CI holds is a revocable token. + +`harper login --for-ci` writes the two variables to **stdout** in `.env` format — and nothing else, so it pipes cleanly. Everything a human reads (the banner, prompts, status) goes to stderr: + +```bash +# Set both GitHub Actions secrets in one command — the token is never displayed +harper login --for-ci | gh secret set --env-file - + +# Or copy them to the clipboard to paste in by hand +harper login --for-ci | pbcopy +``` + +The block it emits: + +```bash +HARPER_CLI_TARGET=https://example.com:9925/ +HARPER_CLI_REFRESH_TOKEN=eyJhbGciOi... +``` + +Because stdout carries only these lines, the token never appears on screen or in your shell history — which is not true of copying it out of terminal output by hand. + +Expose the two values to the deploy step and no other credentials are needed: + +```yaml +- name: Deploy + run: harper deploy project=my-app restart=true replicated=true + env: + HARPER_CLI_TARGET: ${{ secrets.HARPER_CLI_TARGET }} + HARPER_CLI_REFRESH_TOKEN: ${{ secrets.HARPER_CLI_REFRESH_TOKEN }} +``` + +**Precedence**: an explicitly supplied username (arguments or `HARPER_CLI_USERNAME`/`HARPER_CLI_PASSWORD`) wins; otherwise env-var tokens are used; otherwise the token saved by `harper login`. Env-var tokens deliberately outrank the saved credentials file so a runner that has both behaves predictably. A token refreshed from an env var is held in memory for that invocation only — nothing is written to `~/.harperdb/credentials.json`. + +**Lifetimes**: operation tokens expire after `authentication.operationTokenTimeout` (default `1d`) and refresh tokens after `authentication.refreshTokenTimeout` (default `30d`). CI needs re-provisioning when the refresh token expires. + +:::warning +**Each user has only one valid refresh token at a time.** Logging in again as that user issues a new one and invalidates the previous one, so a routine local `harper login` will break a pipeline holding an older token for the same account. + +Create a **dedicated CI user** and run `harper login --for-ci` as that user. This also scopes the pipeline's permissions and lets you revoke its access — by logging in again as that user, or by changing its password — without disturbing anyone else. +::: + #### Method 3: Command Parameters Provide credentials directly as command parameters: diff --git a/reference/cli/commands.md b/reference/cli/commands.md index 24cfbc57b..d216a125a 100644 --- a/reference/cli/commands.md +++ b/reference/cli/commands.md @@ -155,6 +155,7 @@ harper login **Optional Parameters**: - `` - The URL of the Harper instance to log in to. +- `--for-ci` - Print CI/CD credentials to stdout after logging in. Available since v5.2.0. **Prompts**: @@ -164,6 +165,20 @@ You'll be asked to type in the following information: - `` - Harper admin username. - `` - Harper admin password. +**`--for-ci`**: + +Prints `HARPER_CLI_TARGET` and `HARPER_CLI_REFRESH_TOKEN` to **stdout** in `.env` format — and nothing else, so the output pipes directly into a secret store without the token being displayed. Everything else (banner, prompts, status) goes to stderr: + +```bash +# Set both GitHub Actions secrets in one command +harper login --for-ci | gh secret set --env-file - + +# Or copy them to paste in by hand +harper login --for-ci | pbcopy +``` + +See [Token credentials for CI/CD](authentication.md#token-credentials-for-cicd) for how the CLI consumes these variables, and for why a pipeline should use a dedicated user. + ### `harper logout` Available since: v5.0.0 diff --git a/reference/cli/operations-api-commands.md b/reference/cli/operations-api-commands.md index e893f42ec..27fa04fef 100644 --- a/reference/cli/operations-api-commands.md +++ b/reference/cli/operations-api-commands.md @@ -34,80 +34,84 @@ The following operations are available through the CLI. Operations that require This is just a brief overview of all operations available as CLI commands. Review the respective operation documentation for more information on available arguments and expected behavior. Keep in mind that all operations options are converted to CLI arguments in the same way (using `snake_case`). ::: -| Operation | Description | Category | Available Since | -| -------------------------------- | ------------------------------------- | ---------------------------------------------------------------------- | --------------- | -| `describe_table` | Describe table structure and metadata | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `describe_all` | Describe all databases and tables | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `describe_database` | Describe database structure | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_database` | Create a new database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_database` | Delete a database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_table` | Create a new table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_table` | Delete a table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_attribute` | Create a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_attribute` | Delete a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `search_by_id` | Search records by ID | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `search_by_value` | Search records by attribute value | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `insert` | Insert new records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `update` | Update existing records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `upsert` | Insert or update records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `delete` | Delete records | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `list_users` | List all users | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `add_user` | Create a new user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `alter_user` | Modify user properties | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `drop_user` | Delete a user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `list_roles` | List all roles | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `drop_role` | Delete a role | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_ssh_key` | Add SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `get_ssh_key` | Get SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.7.2 | -| `update_ssh_key` | Update SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `delete_ssh_key` | Delete SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `list_ssh_keys` | List all SSH keys | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `set_ssh_known_hosts` | Set SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `get_ssh_known_hosts` | Get SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `remove_node` | Remove node from cluster | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `get_components` | List all components | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `get_component_file` | Get component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `set_component_file` | Set component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `install_node_modules` | Install Node.js dependencies | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `set_configuration` | Update configuration settings | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | -| `get_configuration` | Get current configuration | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | -| `create_authentication_tokens` | Create authentication tokens | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | -| `refresh_operation_token` | Refresh operation token | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | -| `restart_service` | Restart Harper service | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `restart` | Restart Harper instance | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `system_information` | Get system information | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `registration_info` | Get registration information | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `get_fingerprint` | Get instance fingerprint | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `set_license` | Set license key | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `get_usage_licenses` | Get usage and license info | [Licensing](../operations-api/operations.md#registration--licensing) | v4.7.3 | -| `get_job` | Get job status | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | -| `search_jobs_by_start_date` | Search jobs by start date | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | -| `read_log` | Read application logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `read_transaction_log` | Read transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `read_audit_log` | Read audit logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `delete_transaction_logs_before` | Delete old transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `purge_stream` | Purge streaming data | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | -| `delete_records_before` | Delete old records | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | -| `get_status` | Get custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | -| `set_status` | Set custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | -| `clear_status` | Clear custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| Operation | Description | Category | Available Since | +| -------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------- | --------------- | +| `describe_table` | Describe table structure and metadata | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `describe_all` | Describe all databases and tables | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `describe_database` | Describe database structure | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_database` | Create a new database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_database` | Delete a database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_table` | Create a new table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_table` | Delete a table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_attribute` | Create a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_attribute` | Delete a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `search_by_id` | Search records by ID | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `search_by_value` | Search records by attribute value | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `insert` | Insert new records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `update` | Update existing records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `upsert` | Insert or update records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `delete` | Delete records | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `list_users` | List all users | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `add_user` | Create a new user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `alter_user` | Modify user properties | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `drop_user` | Delete a user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `list_roles` | List all roles | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `drop_role` | Delete a role | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_ssh_key` | Add SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `get_ssh_key` | Get SSH key | [Components](../operations-api/operations.md#components) | v4.7.2 | +| `update_ssh_key` | Update SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `delete_ssh_key` | Delete SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `list_ssh_keys` | List all SSH keys | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `set_ssh_known_hosts` | Set SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `get_ssh_known_hosts` | Get SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `remove_node` | Remove node from cluster | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `stage` (alias) | Stage a component without going live | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `activate` (alias) | Activate a previously-staged deploy | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `revert_component` | Roll a component back to its previous version | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `revert` (alias) | Alias for `revert_component` | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `get_components` | List all components | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `get_component_file` | Get component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `set_component_file` | Set component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `install_node_modules` | Install Node.js dependencies | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `set_configuration` | Update configuration settings | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | +| `get_configuration` | Get current configuration | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | +| `create_authentication_tokens` | Create authentication tokens | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | +| `refresh_operation_token` | Refresh operation token | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | +| `restart_service` | Restart Harper service | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `restart` | Restart Harper instance | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `system_information` | Get system information | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `registration_info` | Get registration information | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `get_fingerprint` | Get instance fingerprint | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `set_license` | Set license key | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `get_usage_licenses` | Get usage and license info | [Licensing](../operations-api/operations.md#registration--licensing) | v4.7.3 | +| `get_job` | Get job status | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | +| `search_jobs_by_start_date` | Search jobs by start date | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | +| `read_log` | Read application logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `read_transaction_log` | Read transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `read_audit_log` | Read audit logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `delete_transaction_logs_before` | Delete old transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `purge_stream` | Purge streaming data | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | +| `delete_records_before` | Delete old records | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | +| `get_status` | Get custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| `set_status` | Set custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| `clear_status` | Clear custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | ### Command Aliases @@ -115,6 +119,9 @@ The following aliases are available for convenience: - `deploy` → `deploy_component` - `package` → `package_component` +- `stage` → `deploy_component` with `activate=false` (build + upload the component cluster-wide without going live; prints a `deployment_id`) +- `activate` → `deploy_component` with a `deployment_id` (take a previously staged deployment live; requires `deployment_id`) +- `revert` → `revert_component` For detailed parameter information for each operation, see the [Operations API documentation](../operations-api/operations.md). @@ -212,6 +219,30 @@ harper get_components harper deploy project=my-app package=https://github.com/user/repo ``` +**Stage a component without going live** (builds + uploads across the cluster and prints a `deployment_id`): + +```bash +harper stage project=my-app +``` + +**Activate a previously-staged deployment** (takes it live; requires the `deployment_id` from `stage`): + +```bash +harper activate project=my-app deployment_id=a3f8c2d1... +``` + +**Roll a component back to its previous version**: + +```bash +harper revert project=my-app +``` + +**Mint a deploy key for a private repository** (Harper generates the ed25519 keypair server-side and returns only the public half — register it with your git host): + +```bash +harper add_ssh_key name=my-key generate=true host=my-key.github.com hostname=github.com +``` + :::tip For more information on components and applications, see the [Components Reference](../components/overview.md). ::: diff --git a/reference/components/applications.md b/reference/components/applications.md index dfc70cd33..95882a494 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -130,6 +130,79 @@ Harper generates a `package.json` from component configurations and uses a form For SSH-based private repos, use the [Add SSH Key](#add_ssh_key) operation to register keys first. +### Deploying by Reference + +Available since: v5.2.0 + +Omitting `package` uploads a snapshot of your working directory. The result is an anonymous artifact: nothing records _which_ commit it came from, so reproducing it later — or stepping back to a previous release — means finding those exact files again. + +Deploying by **reference** sends a pinned git reference instead, and the cluster fetches that exact commit. Redeploying the same reference is an exact redeploy, and rolling back is deploying an older one. + +`harper deploy by_ref=true` builds that reference from the local git repository, so you don't assemble the URL yourself: + +```sh +harper deploy by_ref=true restart=true replicated=true +``` + +This resolves the repository's `origin` remote and the current commit, then deploys `package=git+https://github.com//.git#`. + +**Parameters**: + +- `by_ref` - Build the package reference from the local repository. +- `ref` _(optional)_ - Deploy a specific commit, tag, or branch instead of `HEAD`. Implies `by_ref`. +- `credential` _(optional)_ - Git host whose stored credential authenticates the clone, e.g. `github.com`. Omit for public repositories. + +```sh +# Deploy a specific tag +harper deploy ref=v1.2.0 restart=true replicated=true + +# Roll back by deploying an older commit +harper deploy ref=9f8c2a1 restart=true replicated=true +``` + +**A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved locally and the full commit SHA is what ships. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. + +**Commit and push first.** The cluster clones from the remote, so it only sees commits that have been pushed. `by_ref` warns when the working tree is dirty, since uncommitted changes won't be part of the deploy. + +#### Private repositories + +Pass `credential=` for a private repository. The CLI attaches a `credentials` reference naming a secret that the cluster resolves in memory at clone time, so no token travels in the operation body or lands on disk: + +```sh +harper deploy by_ref=true credential=github.com restart=true replicated=true +``` + +Provision that credential once with [`harper deploy setup=true`](#provisioning-a-deploy-credential). See [Private-source deploy credentials](../security/secrets.md#private-source-deploy-credentials) for how the secret is named and resolved, and [`add_ssh_key`](#add_ssh_key) for the SSH-key alternative. + +:::note +Deploying by reference means the **cluster** installs and builds the component from source. If your application needs a build step that can't run on the node, keep shipping the built output as a payload deploy instead. +::: + +### Provisioning a Deploy Credential + +Available since: v5.2.0 + +`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source: + +```sh +harper deploy setup=true +``` + +It asks which private source needs a credential (a GitHub repository or an npm registry) and sources a token — from your `gh` CLI session, or one you paste — then: + +1. Fetches the cluster's public key with `get_secrets_public_key`. +2. **Encrypts the token locally** into an `enc:v1:` envelope. +3. Stores only the ciphertext via `set_secret`, granted to the component. +4. Prints the `credentials` reference for the deploy to use. + +The plaintext never leaves your machine: the operations API, its logs, and replication only ever carry the envelope, and the cluster decrypts it in memory at deploy time. This requires a cluster with secrets custody (Harper Pro / Fabric) — see [Client-side encryption](../security/secrets.md#client-side-encryption-encrypt-before-it-leaves-the-client). + +Because the stored token is durable, later deploys — including re-fetching an older reference — reuse it without re-entering anything. + +:::note +Rolling back to the **immediately previous** version needs no credential at all: [`revert_component`](../operations-api/operations.md#revert_component) swaps in the retained previous build without re-fetching from the source. +::: + ## Dependency Management Harper uses `npm` and `package.json` for dependency management. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 7a52b122a..72ff9e415 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -602,10 +602,27 @@ Detailed documentation: [Components Overview](../components/overview.md) Deploys a component. The `package` option accepts any valid NPM reference including GitHub repos (`HarperDB/app#semver:v1.0.0`), tarballs, or NPM packages. The `payload` option accepts a base64-encoded tar string from `package_component`. Supports `"replicated": true` and `"restart": true` or `"restart": "rolling"`. +Across a cluster, `deploy_component` runs in two phases so a deploy is all-or-nothing at go-live: + +1. **Stage** — the incoming version is downloaded/packed, extracted, and `npm install`ed into a hidden staging directory on **every** node, without touching the live component. +2. **Activate** — only after every node reports a successful stage does any node atomically swap the staged copy into the live path and restart. + +If a node can't fetch the package or fails `npm install`, it fails during staging and the live component is left untouched on every node, rather than leaving part of the cluster half-updated. The request and response shape are unchanged; the two phases are internal. + +:::note +Deploying a **brand-new** component without a restart (`"restart": false`, or omitting `restart`) marks a restart as required — `get_status` reports `restartRequired: true`, and requests to the new component's routes return an actionable 404 explaining that a restart is needed. A never-loaded component can't serve its routes until Harper restarts, so this makes that state visible instead of silent. Each node reports this for itself, since whether the component was already active can differ per node. Redeploying a component that is **already** live does not set the flag: that component's own file watcher requests a restart only if the update actually needs one. +::: + Additional parameters: - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) +- `activate` — set to `false` to **stage only** and stop before go-live. The build is prepared and verified on every node and the response returns a `deployment_id` in a `staged` state; nothing goes live. Activate it later by calling `deploy_component` again with that `deployment_id` (see below). Useful for pre-staging a release and flipping it live in a separate, fast step. +- `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. For a `package` deploy you do not need to repeat `package` here: the identifier and credential references recorded when it was staged are recovered from the deployment and persisted to root config at activation, on every node — so a later restart or a newly joined peer reinstalls the version you activated. +- `revert_on_failure` — if the activate phase fails on some nodes (leaving the cluster split across versions), automatically swap the nodes that did activate back to their previous version so the cluster reconverges. Off by default. Applies to both a full deploy and a `deployment_id` activate. +- `ignore_replication_errors` — treat replication/peer failures as non-fatal (best-effort deploy to a partially-available cluster). This also opts out of the stage barrier. Applies to both a full deploy and a `deployment_id` activate. +- `deployment_timeout` — per-deploy budget (ms) for peers to receive the replicated deployment row; defaults to 120000. +- `two_phase` — set to `false` to force the legacy single-phase (in-place) deploy instead of stage-then-activate. - `credentials` — credentials for installing a component from a private npm registry or private git repository (see below) #### Deploy credentials (`credentials`) @@ -656,7 +673,7 @@ Private git repository (token resolved from an existing secret): `credentials` replaces the earlier `registryAuth` field (renamed while the feature was in alpha, before it grew to carry git-host credentials). `registryAuth` is now rejected with an error directing you to `credentials`. ::: -The response includes a `deployment_id` that can be used to query the deployment record: +A normal deploy (stage + activate): ```json { @@ -668,31 +685,89 @@ The response includes a `deployment_id` that can be used to query the deployment } ``` -Response: +Response — a rolling restart is driven by a separate replicated job, so its id comes back as `restartJobId`: ```json { "deployment_id": "a3f8c2d1...", - "message": "Component deployed successfully" + "restartJobId": "b7d41e09...", + "message": "Successfully deployed: my-app, restarting Harper" +} +``` + +Without a restart (`"restart": false`, or omitted) the response carries no `restartJobId` and the message is just `Successfully deployed: my-app`. + +Stage now, activate later: + +**Stage request** (`activate: false`): + +```json +{ "operation": "deploy_component", "project": "my-app", "package": "my-org/my-app#semver:v1.2.3", "activate": false } +``` + +**Stage response** (nothing is live yet; note the `staged` marker and the `deployment_id`): + +```json +{ "deployment_id": "a3f8c2d1...", "project": "my-app", "staged": true, "message": "Staged component: my-app" } +``` + +**Activation request** (take the staged build live by passing its `deployment_id`): + +```json +{ "operation": "deploy_component", "project": "my-app", "deployment_id": "a3f8c2d1...", "restart": "rolling" } +``` + +### `revert_component` + +Swaps a component's live version back to its **retained previous version** across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced (one previous version is kept per component), so `revert_component` is a fast rollback that does not re-fetch or re-install. The swap is bidirectional — reverting a revert rolls forward again. + +This supports customer-driven rollback: deploy a new version, run your own health checks against it, and revert if you are not happy — even when the cluster otherwise looks healthy. + +```json +{ + "operation": "revert_component", + "project": "my-app", + "restart": "rolling" } ``` +`revert_component` fails with "no previous version is retained" for a component that has only ever been deployed once (nothing to revert to). + +:::caution +Reverting swaps the **live directories** on the nodes currently in the cluster; it does not rewrite the component's stored `package:` reference in `harperdb-config.yaml`. For a `package` deploy that means a revert is not a config-level rollback: a node provisioned _after_ the revert — a newly joined peer, or an existing node whose components directory is rebuilt — installs from the stored package reference at boot, so it comes up on the version you reverted away from rather than the one every other node is running. + +To roll back durably for a `package` deploy, deploy the older version explicitly (`deploy_component` with the previous `package` reference) instead of, or after, reverting. `revert_component` is the fast live-instance swap; an explicit deploy is what changes what a future node will install. +::: + ### Deployment Operations -Harper records every `deploy_component` call in the `system.hdb_deployment` table, capturing the full lifecycle of a deployment including phase transitions (prepare → load → replicate → restart → success/failed), per-node outcomes, and a bounded event log of install output. +Harper records every `deploy_component` call in the `system.hdb_deployment` table, capturing the full lifecycle of a deployment including phase transitions (`stage` → `activate` → `restart` → `success`/`failed`, or `prepare` → `replicate` → `restart` on the legacy single-phase path), per-node outcomes, and a bounded event log of install output. + +**Staged-build retention.** Deployments staged with `activate: false` leave their built files on disk until they are activated. Harper keeps only the most recent staged builds per component (default 5, configurable via the `deployment_stagingRetention_maxCount` configuration option); older not-yet-activated staged builds are evicted automatically when a new stage lands. Activating a `deployment_id` that has aged out of this window fails with "no staged build found." + +**Payload retention.** The `hdb_deployment` records themselves are always retained as the audit trail — retention only ever reclaims the stored tarball (`payload_blob`), never the row. Two configuration options bound it, and they answer different questions: + +| Option | Default | Effect | +| -------------------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | +| `deployment_payloadRetention_maxSize` | 10 MiB | Reclaims **this** deploy's tarball right after it succeeds, if the tarball was larger than this. Bounds any single payload. | +| `deployment_payloadRetention_maxCount` | 1 | Keeps at most this many stored tarballs **per project**, newest first, dropping the rest after a successful deploy. Bounds total. | + +The default of `maxCount: 1` means only the current version's tarball is kept. It is deliberately conservative: retained payloads share the instance's disk with your own data, so several copies of a large application payload can quietly consume quota. Raise it if you want a wider window of deployments whose payload is still downloadable; set it to `0` to keep none. + +Pruning is automatic and best-effort — it never fails a deploy — and is skipped when a peer failed, since the older payloads are still the retry artifact in that case. A deployment whose payload has been reclaimed (automatically, or explicitly via [`delete_deployment_payload`](#delete_deployment_payload)) reports `payload_blob_present: false` and can no longer serve [`get_deployment_payload`](#get_deployment_payload); everything else about the record stays intact. ### `list_deployments` Returns a list of deployment records, newest first. All filter parameters are optional. -| Parameter | Type | Description | -| --------- | ------ | ------------------------------------------------ | -| `project` | string | Filter to a specific component project | -| `status` | string | Filter by status: `pending`, `success`, `failed` | -| `since` | number | Start of time range (Unix timestamp ms) | -| `until` | number | End of time range (Unix timestamp ms) | -| `limit` | number | Maximum number of results (default: 100) | -| `offset` | number | Pagination offset | +| Parameter | Type | Description | +| --------- | ------ | ---------------------------------------- | +| `project` | string | Filter to a specific component project | +| `status` | string | Filter by status (see below) | +| `since` | number | Start of time range (Unix timestamp ms) | +| `until` | number | End of time range (Unix timestamp ms) | +| `limit` | number | Maximum number of results (default: 100) | +| `offset` | number | Pagination offset | ```json { @@ -705,6 +780,12 @@ Returns a list of deployment records, newest first. All filter parameters are op Response includes a `deployments` array and a `total` count. The `payload_blob` field is stripped from list responses for size; use `get_deployment_payload` to retrieve the tarball. +Deployment statuses fall into three groups: + +- **Terminal** — `success`, `failed`, `rolled_back`. The deploy is over. Only these count as terminal internally, which is what gates `get_deployment_payload` and makes a payload eligible for retention pruning. +- **Resting** — `staged`. An `activate: false` stage-and-stop, waiting to be activated or to age out of the staging-retention window. It is finished but not terminal, so its payload is deliberately still held: it is the source the pending activation needs. +- **In flight** — `pending`, `extracting`, `installing`, `staging`, `loading`, `replicating`, `activating`, `reverting`, `restarting`. The phase the deployment is currently in; a record only stays in one of these while the deploy is running. + ### `get_deployment` Returns a single deployment record by `deployment_id`. When called on an in-progress deployment via a request that accepts `text/event-stream`, the response streams live phase events and install output as Server-Sent Events, replaying the buffered event log then tailing until the deployment reaches a terminal status. @@ -718,22 +799,22 @@ Returns a single deployment record by `deployment_id`. When called on an in-prog The deployment record includes: -| Field | Description | -| -------------------- | ----------------------------------------------------------------------- | -| `deployment_id` | Unique identifier (content hash) | -| `project` | Component project name | -| `package_identifier` | Package reference or `payload` for tar uploads | -| `status` | `pending`, `success`, `failed`, or `rolled_back` | -| `phase` | Current lifecycle phase: `prepare`, `load`, `replicate`, `restart` | -| `event_log` | Bounded log of install output and phase transitions (up to 200 entries) | -| `peer_results` | Per-node outcome map for replicated deployments | -| `payload_hash` | SHA-256 hash of the deployment tarball | -| `payload_size` | Byte size of the deployment tarball | -| `started_at` | Timestamp when deployment began | -| `completed_at` | Timestamp when deployment finished | -| `user` | User who initiated the deployment | -| `rollback_of` | `deployment_id` of the deployment this rolls back, if applicable | -| `error` | Error message for failed deployments | +| Field | Description | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `deployment_id` | Unique identifier (content hash) | +| `project` | Component project name | +| `package_identifier` | Package reference or `payload` for tar uploads | +| `status` | Any of the values listed under [`list_deployments`](#list_deployments) — `pending`, `extracting`, `installing`, `staging`, `staged`, `loading`, `replicating`, `activating`, `reverting`, `restarting`, `success`, `failed`, or `rolled_back` | +| `phase` | Current lifecycle phase: `stage`, `load`, `activate`, `restart` (or legacy `prepare`, `replicate`) | +| `event_log` | Bounded log of install output and phase transitions (up to 200 entries) | +| `peer_results` | Per-node outcome map for replicated deployments | +| `payload_hash` | SHA-256 hash of the deployment tarball | +| `payload_size` | Byte size of the deployment tarball | +| `started_at` | Timestamp when deployment began | +| `completed_at` | Timestamp when deployment finished | +| `user` | User who initiated the deployment | +| `rollback_of` | `deployment_id` of the deployment this rolls back, if applicable | +| `error` | Error message for failed deployments | ### `get_deployment_payload` @@ -775,7 +856,11 @@ The deployment must be in a terminal status (`success`, `failed`, or `rolled_bac ### `add_ssh_key` -Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. +Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. Supply the private key with `key`, or omit it and pass `generate: true` to have Harper mint the keypair itself. + +The stored private key is encrypted at rest and only ever crosses the cluster as ciphertext; `list_ssh_keys` and the logs never return key material. + +Adding an existing key: ```json { @@ -787,6 +872,37 @@ Adds an SSH key (must be ed25519) for authenticating deployments from private re } ``` +#### Server-side key generation (`generate`) + +With `generate: true`, Harper mints an ed25519 keypair on the node handling the request and returns only the **public** half. The private key is created inside the cluster and never travels from a client, so it can't be captured in a shell history, CI log, or request body on the way in: + +```json +{ + "operation": "add_ssh_key", + "name": "my-key", + "generate": true, + "host": "my-key.github.com", + "hostname": "github.com" +} +``` + +Response: + +```json +{ + "message": "Added ssh key: my-key", + "public_key": "ssh-ed25519 AAAAC3Nza... harper:my-key" +} +``` + +Register that `public_key` with your git host (e.g. as a GitHub deploy key) to authorize the deploy. The generated key is commented `harper:` so it's identifiable in the host's key list. + +`key` and `generate` are mutually exclusive — sending both is rejected. Generation runs `ssh-keygen` on the node; if it isn't available on the PATH the operation fails with an actionable error rather than storing a partial key. + +:::note +`public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the sealed private key and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. +::: + --- ## Secrets diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 2e2a0a842..9b0ac045d 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -289,6 +289,8 @@ function encryptSecret(plaintext, publicKeyPem, kid) { `deploy_component` accepts a `credentials` array so a component installed from a private **npm registry** or private **git repository** can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body, persisting as a plaintext `.npmrc`, or being written to disk for git — so package-reference deploys survive rollback, reboot, and new peers joining. Ingested tokens are stored under a derived name (`deploy..` or `deploy..git.`) granted to the component. See [`deploy_component`](../operations-api/operations.md#deploy_component). +To provision one of these without handing the cluster a plaintext token at all, `harper deploy setup=true` (v5.2.0+) runs the [client-side encryption](#client-side-encryption-encrypt-before-it-leaves-the-client) flow above for you: it fetches the public key, seals the token locally into an `enc:v1:` envelope, stores only the ciphertext under that same derived name, and prints the `credentials` reference for the deploy to use. See [Provisioning a Deploy Credential](../components/applications.md#provisioning-a-deploy-credential). + ## Threat model **Protects against:** theft of on-disk config/`.env` files, the editor/operations read surface, secrets appearing in operations logs and replication payloads, and an operator observing traffic at the TLS-terminating layer. Client-side encryption additionally keeps plaintext off the operations API entirely.