diff --git a/docs/install-siteops.md b/docs/install-siteops.md index fdbeffb4..e9e424eb 100644 --- a/docs/install-siteops.md +++ b/docs/install-siteops.md @@ -427,7 +427,7 @@ an existing cluster connected to Azure Arc. The script verifies `siteops` inside its child shell and prints its `Command directory:` on success. If `siteops` is not on the parent shell's -current PATH, open a new shell or add the printed directory for this session: +current PATH, add the printed directory for this session: ```bash export PATH=":$PATH" @@ -437,6 +437,9 @@ export PATH=":$PATH" $env:PATH = ";" + $env:PATH ``` +For new terminals, run `uv tool update-shell`, then open a new terminal. If +`uv` is not on `PATH`, run the `uv` in the printed directory. + The printed directory can differ when a native manager location was explicitly selected. Do not assume a fixed command directory. diff --git a/docs/migrating.md b/docs/migrating.md index 96291858..1ab2aa93 100644 --- a/docs/migrating.md +++ b/docs/migrating.md @@ -51,7 +51,7 @@ Start with the changes that affect your workflow: | If you... | What to change | |---|---| -| Installed Site Ops with `pip install -e .` | [Reinstall, or install an identified release](#site-ops-installation). | +| Installed Site Ops with `pip install -e .` | [Run it again](#site-ops-installation) after updating your clone. | | Check the installed release with `siteops --version` | Expect the [engine version](#reported-version), `1.0.0b1`, with a build suffix for an installed release. | | Deploy with Azure CLI older than 2.70.0 | Run `az upgrade`. See [Azure CLI version](#azure-cli-version). | | Run `plan` or `deploy` for Bicep or kubectl steps | Install the [local tools](#local-tool-checks) those steps use. | @@ -72,6 +72,7 @@ Start with the changes that affect your workflow: | Use the Azure Pipelines templates | Choose an [engine selection](#azure-pipelines-templates). | | Reference the Azure Pipelines templates from another repository | Set [`templateRepository`](#template-repositories-and-copied-templates) to the repository resource alias. | | Copy the Azure Pipelines templates into your repository | Copy one [matching set](#template-repositories-and-copied-templates) from the same release. | +| Set `siteopsSource` in your pipeline | Keep the template `ref` on [the same release](#template-repositories-and-copied-templates). | | Use `SITE_OVERRIDES` in Azure Pipelines | Define the variable and review [override validation](#site-overrides). | | Pass absolute paths to the deployment pipelines | Use [relative paths](#pipeline-checkout-paths). | | Queue `.pipelines/deploy.yaml` with `siteopsSource` | Set `release` and `sourceCommit` instead. See [Azure Pipelines templates](#azure-pipelines-templates). | @@ -83,16 +84,17 @@ Start with the changes that affect your workflow: ### Site Ops installation -The v1.0.0b6 guide installed Site Ops with `pip install -e .` in a clone. The -engine now also depends on `packaging`, so run `pip install -e .` again after -updating a clone. +After updating a clone, run `pip install -e .` again, because the engine now +also depends on `packaging`. In Azure Pipelines, `siteopsSource` installs Site +Ops with pip from the source you name, such as a tagged Git source. See +[Azure Pipelines templates](#azure-pipelines-templates). -The editable installation is now a contributor workflow. To run an identified -engine instead, use the command from the selected release: +You can also install the engine built for a release, using the command from +that release's notes: [native `uv tool install`](install-siteops.md#install-the-release-wheel) of the release wheel, or the [verified bootstrap](install-siteops.md#bootstrap-from-https). -Deactivate or uninstall the editable installation first, then confirm the -version with `siteops --version`. +Deactivate or uninstall the pip installation first, then confirm the version +with `siteops --version`. ### Reported version @@ -541,6 +543,11 @@ the same release and replace the whole set when you upgrade. See [templates your pipeline uses](ci-cd-setup.md#templates-your-pipeline-uses) for the files each route needs. +When you set `siteopsSource`, use the same release as the template `ref`. A +template older than its engine can report success for a command the engine +rejected. With v1.0.0b2 or v1.0.0b3 templates, for example, the validation +step succeeds even though validation never ran. + ### Site overrides Define `SITE_OVERRIDES` in Azure Pipelines, empty when you do not use @@ -814,6 +821,8 @@ Define `SITE_OVERRIDES` in Azure Pipelines, empty when you do not use overrides. Invalid JSON, including the literal `$(SITE_OVERRIDES)` of an undefined variable, fails the override step. +Validation errors now fail the validation step, so Deploy does not run. + The `manifest` input of `.pipelines/deploy.yaml` and `.github/workflows/deploy.yaml` is a path relative to the workspace, such as `manifests/aio-install.yaml` instead of `aio-install`. diff --git a/releases/v1.0.0b7/notes.md b/releases/v1.0.0b7/notes.md new file mode 100644 index 00000000..30e70b7e --- /dev/null +++ b/releases/v1.0.0b7/notes.md @@ -0,0 +1,304 @@ +## Highlights + +### 1. Install Site Ops from a release + +Install Site Ops and use published Scale Kit content with your own Site +inputs, without a repository clone. The bootstrap verifies the engine +archive's provenance and payload, provisions native uv and managed Python +with your consent, and installs the recorded dependency wheels. +It runs as your ordinary user on Windows x64 or Linux x64 based on glibc. + +Use the generated **Install Site Ops** section below for the exact package +version, Windows and Linux commands, verification choices and source +enrollment. The [README](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/README.md#quick-start) +also provides a short command that selects this release by tag and can +enroll the `official` source in the same invocation. +That route trusts HTTPS for the initial script. Choose the attestation +verification route when your policy requires script provenance before it runs. + +Source enrollment records the publisher you trust for 30 days. Azure login +and deployment confirmation remain separate choices. + +### 2. Deploy AIO from one cluster ID, with optional Secret Sync + +The Arc cluster resource ID is the only required answer for guided AIO +installation. Site Ops reads the cluster to derive the subscription, resource +group, region and cluster name, and names the Site after the cluster in +lowercase. To choose another name, or when the cluster's name is not a valid +Site name, add `--input siteName=`. +Environment and country labels are optional. + +Use an existing Arc cluster in its own resource group, Azure CLI 2.70.0 or +newer signed in with deployment and role assignment permissions, and +GitHub CLI 2.95 or a later 2.x release. Enroll `official` using the +instructions below, then replace the cluster placeholder. This command +installs AIO and enables Secret Sync together: + +```text +siteops deploy aio-install --source "official@v1.0.0b7" --input "cluster=" --input enableSecretSync=true +``` + +Secret Sync requires the existing cluster to report an OIDC issuer and +enabled workload identity. Omit `--input enableSecretSync=true` for AIO +alone. Secret Sync is disabled by default. + +For an existing AIO 2607, 2608 or 2609 instance, replace the instance +placeholder and enable Secret Sync directly: + +```text +siteops deploy secretsync --source "official@v1.0.0b7" --input "instance=" +``` + +This route resolves the instance and its associated cluster, uses the actual +instance name, and performs Secret Sync enablement without reinstalling or +upgrading AIO. Add `existingVault` to use an existing Key Vault. +Answer files and inline answers use the same typed input contract. A Site +saved from those answers is an ordinary Site file, and later deployments +from it use the saved values without reading the cluster again. +See [guided inputs](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/guided-inputs.md). + +### 3. Review a prepared plan and read deployment outcomes + +Prepare an executable plan before deployment: + +```text +siteops plan aio-install --source "official@v1.0.0b7" --input "cluster=" +``` + +`plan` validates inputs, resolves operations, compiles templates and checks +the local tools those operations need. It performs no deployment writes. +Supplying a resource ID authorizes bounded Azure reads to resolve the Site. +Use `--describe` for the plan shape without compilation or tool checks. + +In an interactive terminal, `deploy` shows its prepared plan, asks you to +confirm, and executes that same plan. A separate `plan` invocation is a +preview, not a saved execution artifact. Add `--yes` for CI, JSON output or +other unattended deployment. + +Verified packages deploy the templates their publisher compiled. A local +workspace submits its Bicep source, and the plan records the compilation +checked during preparation. Progress uses stderr, and final JSON output is +one document on stdout. Results retain each Site's and operation's outcome, +including partial failure, skipped work, cancellation and unconfirmed +completion. + +A successful deployment reports completed operations. Check AIO readiness +and the workload's functional result separately before proceeding. +See [plan output](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/plan-output.md), +[run output](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/run-output.md) +and [check the result](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/guided-inputs.md#check-the-result). + +### 4. Browse, acquire and publish complete workspace packages + +Keep your Site configuration separate from the content you consume. +After enrolling `official`, acquire this release's workspace and inspect +its deployment choices: + +```powershell +siteops --approved-source official project pin .\factory --release v1.0.0b7 +siteops --approved-source official --project .\factory browse aio-install +``` + +The project pin selects an exact package and leaves your Sites unchanged. +Package use verifies provenance, byte integrity, compatibility and current +source policy. Add `--offline-content` to content commands in a pinned +project to use cached package and proof bytes. This restricts content +acquisition, not Azure reads or deployment. + +`browse` shows requirements, effects, guides and typed input availability +without loading Site values or invoking deployment tools. Remote +`browse --source` reads published descriptions rather than deployable +content. + +Content authors generate a public content index with +`siteops -w workspaces\iot-operations index --public` in a reviewed local +checkout, then commit the generated files. Reviewed release declarations +build and sign complete workspace packages, including compiled templates +and their documentation. +A signed engine reference identifies the exact engine selected for a +content release. Source enrollment remains the consumer's decision. +See [projects](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/projects.md), +[browsing](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/browse-content.md) +and [workspace packages](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/workspace-packages.md). + +### 5. Qualify the exact release candidate before publication + +Publication requires passing acceptance for the exact frozen engine and +workspace assets. Acceptance covers native installation, guided AIO alone, +combined AIO and Secret Sync, Secret Sync on an existing instance, and +one deployment across a fleet. Deployment cases check AIO pod readiness +and confirm removal of the resources they created. + +The publication approval remains separate, and the publisher verifies the +bound acceptance receipt before creating a tag or release. Official +releases require immutable tags and assets. Windows installation includes +a standard user path, and installation checks include Ubuntu 26.04. + +Deployment, infrastructure readiness, workload data delivery and Secret Sync +materialization remain distinct outcomes. Acceptance checks deployment +and readiness. Use the workload and Secret Sync guides for functional checks. +See [release acceptance](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/releasing.md#accept-the-candidate). + +### 6. Diagnose failures without publishing private values + +Content errors carry stable codes such as `content.duplicate-key` and +`content.unknown-field`, with fixed guidance and documentation links. +Validation also emits structured JSON. From a matching local checkout: + +```powershell +siteops -w workspaces\iot-operations validate aio-install --output json --projection publishable +``` + +Deployment results classify failed and unconfirmed logical operations, +and include failure diagnostics beside the existing operation counts. +Publishable output retains causes and allowlisted Azure codes while keeping +Site values, paths and raw provider messages private. +Azure Pipelines and GitHub Actions render these results with the same helper +and preserve the operation's exit status through reporting and cleanup. +See the [diagnostic index](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/diagnostics.md). + +### 7. Choose AIO 2609 or keep an explicit AIO release + +AIO 2608 and 2609 configurations join the workspace. AIO 2609, which pins +AIO 1.4.112, is the default for inherited Site settings and guided +installation. AIO 2607 and 2608 remain selectable through +`properties.aioRelease` or the guided `aioRelease` input. +For example, add `--input aioRelease=2608` to the installation command to +select 2608 deliberately. + +The 2608 and 2609 configurations supply WebAssembly graph controller MQTT +trust settings and the OPC UA connector template. Install and upgrade +create that template when OPC UA is enabled. With feature mode `Disabled`, +they skip it. The legacy `connectors` feature supplies the mode when +`opcua` is absent, and `opcua` takes precedence when both are present. +Upgrade uses the existing instance's resolved features. + +Use `aio-upgrade` to change an existing installation's AIO release. +See [AIO releases](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/aio-releases.md) +for pins, supported transitions and the AIO 2607 connector template limitation. + +### 8. Compose catalog resource sets across a fleet + +Select ordered resource sets for devices, assets and dataflows on each Site. +Definitions compose by resource identity across the selected files. +Assets can reference independently selected devices, and dataflows can +reference endpoints and profiles from other sets. Duplicate writers and +unresolved references are checked before deployment. + +In a matching local checkout, a configured Site can select the shipped sets: + +```yaml +properties: + resourceSets: + devices: + - site-devices + assets: + - site-assets + dataflows: + - shared-mqtt-endpoint + - shared-dataflow-profile + - basic-routing +``` + +For a configured Site named `plant-one` that already runs AIO, review the +effective composition: + +```powershell +siteops -w workspaces\iot-operations plan aio-resources -l name=plant-one --describe +``` + +The example device points at the OPC PLC simulator used by the samples. +Use the sample setup or change its endpoint to your reachable server before +applying it. + +Local plans show source provenance, applied resources and resolved +references. Explicit external assertions let a set describe a dependency +managed elsewhere, with its reason retained locally. Those assertions +describe expected resources rather than reading Azure to prove existence. + +Catalog deployment creates or updates selected definitions. Deselecting a +set leaves existing resources in place for deliberate removal. +The [resource catalog](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/resource-catalog.md) +and the [basic](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/workspaces/iot-operations/samples/resource-set-basic/README.md) +and [composition](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/workspaces/iot-operations/samples/resource-set-composition/README.md) +samples show the full path from selection to workload checks. + +## Automation and operator controls + +- Azure Pipelines can select an attested engine from a release tag or explicit + release and source commit, while keeping caller content in its own checkout. + Environment service connections and variable groups live in reviewed YAML. +- GitHub Actions uses full commit SHA references for external actions. +- Plain output uses ASCII markers, wraps to the terminal width and shows + bounded progress while preparation or provider work continues. +- `sites --output yaml`, `sites --output json` and `sites --show-sources` + support private inspection of resolved configuration and provenance. +- AIO extension configuration overrides belong under the saved Site's + `parameters` map. See [configure the Site](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/workspaces/iot-operations/manifests/aio-install/README.md#configure-the-site). + +See [CI/CD setup](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/ci-cd-setup.md) +for engine selection, fleet targeting and diagnostic rendering in automation. + +## Upgrading + +Read the [v1.0.0b6 to v1.0.0b7 migration guide](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/migrating.md#to-v100b7) +before updating existing workspaces, scripts or pipelines. It compares this +release with v1.0.0b6. From an earlier release, start with the guide's +section for the release after yours. + +The breaking changes to check first are Site selected parameter paths and +catalog resource set selections. Keep dynamic parameter paths relative +and inside the workspace, with no `..` segments. Replace scalar resource +set selections with ordered lists, omit an area for no selection, and +use `[]` to clear an inherited list. Remove the `none` selection, and give +a directly attached dataflow parameter file the `path` and `collections` +form shown in the migration guide. + +The inherited AIO default changes from 2607 to 2609. Pin +`properties.aioRelease: "2607"` to retain the v1.0.0b6 AIO release, or +`"2608"` to select 2608. Use the upgrade operation for an existing instance. + +Update automation to use `plan` in place of `deploy --dry-run` and +`plan --describe` in place of `validate --plan`. Unattended deployment +requires `--yes`. Review moved Manifest and resource set paths, the +Azure CLI minimum, and matching pipeline templates and result helpers +in the migration guide. + +An Azure Pipelines pipeline that references the templates from another +repository, for example with `@scalekit`, sets `templateRepository` to that +repository resource's alias. Copied templates need the matching helper +scripts from this release. See +[template repositories and copied templates](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/migrating.md#template-repositories-and-copied-templates). + +## Operational notes + +- This is a Scale Kit beta release. The Site Ops source version remains + `1.0.0b1`, and the included engine artifact has a distinct build suffix. + The generated installation section reports its exact package version. +- If `siteops` is not found in a new terminal after installation, as is + common on Windows, run `uv tool update-shell` and open a new terminal. If + `uv` is not on `PATH`, run the `uv` in the command directory the installer + prints. Before installing, remove an older `siteops`, such as a pip + installation, so it does not take precedence. +- Installation and source enrollment do not sign in to Azure or authorize + resource changes. Planning with a resource ID performs bounded reads. + Deployment requires its own confirmation and can incur Azure charges. +- Reapplying `aio-install` can overwrite instance settings. Select new + clusters explicitly for installation, use `aio-upgrade` for upgrades, + and use `secretsync` for enablement on an existing instance. +- Secret Sync enablement grants Key Vault roles to its managed identity. To + manage the vault's secrets yourself, for example in the IoT Operations + portal, assign your account a Key Vault data role such as Key Vault + Secrets Officer. +- A Site name becomes part of its schema registry namespace, which must be + unique in the Microsoft Entra tenant. Sites built from typed inputs scope + that namespace to their resource group. Give a Site file you write + yourself a name that no other Site in the tenant uses. See + [Site identity](https://github.com/Azure/digital-ops-scale-kit/blob/v1.0.0b7/docs/site-configuration.md#site-identity). +- Run the OPC UA solution sample from a local checkout of this release, with + `siteops -w workspaces/iot-operations`. The sample applies an OPC PLC + simulator from a pinned public URL, and verified packages apply only the + files they carry. +- Failure or interruption can leave partial changes. Review the deployment + result and provider state before retrying. Resource removal remains an + explicit operator action. diff --git a/releases/v1.0.0b7/release.json b/releases/v1.0.0b7/release.json new file mode 100644 index 00000000..ba3ede56 --- /dev/null +++ b/releases/v1.0.0b7/release.json @@ -0,0 +1,31 @@ +{ + "tag": "v1.0.0b7", + "headline": "Verified installation, guided AIO deployment and AIO 2608/2609", + "siteops": { + "build": true + }, + "workspaces": [ + { + "workspace": "workspaces/iot-operations", + "id": "azure.iot-operations", + "package": "iot-operations.zip", + "compatibility": { + "siteops": ">=1.0.0b1,<2", + "requiredFeatures": [ + "manifest/v1", + "composition/v1", + "manifest-selection/v1" + ] + }, + "include": [ + "docs", + "README.md" + ], + "licenses": [ + "LICENSE", + "ThirdPartyNotices.txt" + ] + } + ], + "latest": false +}