Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/electron-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:
- name: Build the unpacked Windows application including beforePack
env:
CSC_IDENTITY_AUTO_DISCOVERY: 'false'
run: npx electron-builder --win --x64 --dir --publish never
run: npx electron-builder --config electron-builder-code-only.cjs --win --x64 --dir --publish never

- name: Smoke packaged wallpaper startup resources and native tray
shell: pwsh
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/python-local-models.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ jobs:
tests/test_japanese_base_portability.py
tests/test_japanese_aliases.py
tests/test_japanese_full_resource.py
tests/test_bundled_japanese_dictionary.py
- name: Switch the same environment to T3 vad
run: |
uv sync --locked --extra voice --extra vad --extra torch-cpu
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ jobs:
working-directory: electron
env:
CSC_IDENTITY_AUTO_DISCOVERY: 'false'
run: npx electron-builder --win --x64 --dir --publish never
run: npx electron-builder --config electron-builder-code-only.cjs --win --x64 --dir --publish never

- name: Launch packaged frontend with supplied Python runtime
run: >-
Expand Down
83 changes: 73 additions & 10 deletions .github/workflows/source-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,25 @@ on:
- electron/package-lock.json
- scripts/**
- tools/build_source_release.py
- tools/stage_japanese_dictionary.py
- tools/probes/verify_bundled_dictionary.py
- electron/scripts/build-runtime-resources.cjs
- electron/electron-builder-code-only.cjs
- config/japanese_dictionary.py
- tests/test_bundled_japanese_dictionary.py
- tools/check_third_party_provenance.py
- tests/test_release_tooling.py
- .github/workflows/source-release.yml
workflow_dispatch:
inputs:
dictionary_url:
description: "HTTPS URL of the exact pinned resource (required for full application archives)"
type: string
required: false
code_only:
description: "Build an explicitly labelled developer archive without runtime resources"
type: boolean
default: false
push:
tags:
- "v*"
Expand All @@ -41,17 +56,43 @@ jobs:
- name: Verify included third-party provenance
run: python tools/check_third_party_provenance.py --release-ready

- name: Select distribution inputs
shell: pwsh
env:
EVENT_NAME: ${{ github.event_name }}
CODE_ONLY: ${{ inputs.code_only }}
DICTIONARY_URL: ${{ inputs.dictionary_url }}
run: |
$codeOnly = $env:EVENT_NAME -eq 'pull_request' -or $env:CODE_ONLY -eq 'true'
if ($codeOnly) {
"SOURCE_ARCHIVE=amadeus-code-only.zip" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
"CODE_ONLY_DISTRIBUTION=true" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
} else {
$pin = Get-Content release/japanese_dictionary.json -Raw | ConvertFrom-Json
$url = $env:DICTIONARY_URL
if (!$url) { $url = $pin.download_url }
if (!$url -or ([uri]$url).Scheme -ne 'https') {
throw 'Full application archives require the pinned dictionary URL; resources must be accepted and supplied before release.'
}
New-Item -ItemType Directory -Force build/release-inputs | Out-Null
Invoke-WebRequest -Uri $url -OutFile build/release-inputs/japanese-dictionary.zip
"SOURCE_ARCHIVE=amadeus-source.zip" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
"CODE_ONLY_DISTRIBUTION=false" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
}

- name: Build deterministic source archive
run: >-
python tools/build_source_release.py
--manifest-output build/release/source-manifest.json
--output dist/amadeus-source.zip
shell: pwsh
run: |
$resourceArgs = @('--dictionary-bundle', 'build/release-inputs/japanese-dictionary.zip')
if ($env:CODE_ONLY_DISTRIBUTION -eq 'true') { $resourceArgs = @('--code-only') }
python tools/build_source_release.py --manifest-output build/release/source-manifest.json --output "dist/$env:SOURCE_ARCHIVE" @resourceArgs
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

- name: Write SHA-256 checksum
shell: pwsh
run: |
$hash = (Get-FileHash -Algorithm SHA256 dist/amadeus-source.zip).Hash.ToLowerInvariant()
"$hash amadeus-source.zip" | Set-Content -Encoding ascii dist/SHA256SUMS.txt
$hash = (Get-FileHash -Algorithm SHA256 "dist/$env:SOURCE_ARCHIVE").Hash.ToLowerInvariant()
"$hash $env:SOURCE_ARCHIVE" | Set-Content -Encoding ascii dist/SHA256SUMS.txt

# Repository builds cannot detect files missing from the allowlisted ZIP.
# Install and launch the packaged frontend using only extracted sources.
Expand All @@ -69,7 +110,7 @@ jobs:
- name: Extract the release archive
shell: pwsh
run: |
Expand-Archive -LiteralPath dist/amadeus-source.zip -DestinationPath build/source-smoke
Expand-Archive -LiteralPath "dist/$env:SOURCE_ARCHIVE" -DestinationPath build/source-smoke
$manifest = Get-Content build/release/source-manifest.json -Raw | ConvertFrom-Json
$sourceRoot = Join-Path $PWD "build/source-smoke/amadeus-$($manifest.version)"
"SOURCE_ROOT=$sourceRoot" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
Expand All @@ -82,6 +123,14 @@ jobs:
working-directory: ${{ env.SOURCE_ROOT }}
run: uv run --locked --no-sync python tools/verify_python_environment.py --profile ci

- name: Verify the extracted bundled dictionary with the native frontend
if: env.CODE_ONLY_DISTRIBUTION == 'false'
working-directory: ${{ env.SOURCE_ROOT }}
run: >-
uv run --locked --no-sync --with pyopenjtalk-plus==0.4.1.post9
python -X utf8 tools/probes/verify_bundled_dictionary.py
--output build/source-release-smoke/dictionary-source.json

- name: Install Electron and Pi from extracted sources
working-directory: ${{ env.SOURCE_ROOT }}/electron
run: npm ci
Expand All @@ -101,7 +150,21 @@ jobs:
working-directory: ${{ env.SOURCE_ROOT }}/electron
env:
CSC_IDENTITY_AUTO_DISCOVERY: "false"
run: npx electron-builder --win --x64 --dir --publish never
shell: pwsh
run: |
$configArgs = @()
if ($env:CODE_ONLY_DISTRIBUTION -eq 'true') { $configArgs = @('--config', 'electron-builder-code-only.cjs') }
npx electron-builder --win --x64 --dir --publish never @configArgs
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

- name: Verify the dictionary carried by the Electron package
if: env.CODE_ONLY_DISTRIBUTION == 'false'
working-directory: ${{ env.SOURCE_ROOT }}
shell: pwsh
run: |
$env:AMADEUS_BUNDLED_JAPANESE_DICTIONARY = Join-Path $PWD 'electron/build/win-unpacked/resources/assets/bundled/japanese-pronunciation'
uv run --locked --no-sync --with pyopenjtalk-plus==0.4.1.post9 python -X utf8 tools/probes/verify_bundled_dictionary.py --output build/source-release-smoke/dictionary-electron.json
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

- name: Launch packaged application from the source archive
working-directory: ${{ env.SOURCE_ROOT }}
Expand All @@ -120,9 +183,9 @@ jobs:

- uses: actions/upload-artifact@v6
with:
name: amadeus-source-${{ github.sha }}
name: ${{ env.SOURCE_ARCHIVE }}-${{ github.sha }}
path: |
dist/amadeus-source.zip
dist/*.zip
dist/SHA256SUMS.txt
build/release/source-manifest.json
if-no-files-found: error
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -146,3 +146,6 @@ assets/audio/sfx/*.wav

# 口型调试用参考图(本地生成)
tools/debug_mouth/*.png

# Generated, independently versioned defaults are injected only into distributions.
/assets/bundled/
13 changes: 10 additions & 3 deletions config/asset_packages.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,13 @@ def external_asset_pack_status(
except KeyError as exc:
raise AssetPackageError(f"unknown external asset pack: {pack_id}") from exc

present = [path for path in spec.required if asset_path(asset_root, path).is_file()]
dictionary_root = None
if spec.validator == "japanese_dictionary":
from config.japanese_dictionary import dictionary_pack_root
dictionary_root = dictionary_pack_root(asset_root)
present = [path for path in spec.required
if (dictionary_root / path.relative_to(spec.trees[0]) if dictionary_root is not None
else asset_path(asset_root, path)).is_file()]
missing = [path for path in spec.required if path not in present]
message = ""
version = ""
Expand All @@ -156,7 +162,7 @@ def external_asset_pack_status(
if state == "installed" and spec.validator == "japanese_dictionary":
from config.japanese_dictionary import JapaneseDictionaryError, load_dictionary_pack
try:
pack = load_dictionary_pack(asset_path(asset_root, spec.trees[0]))
pack = load_dictionary_pack(dictionary_root)
version = pack.version
if pack.format_version == 2:
message = (f"{pack.entry_count} source rows preserved; {pack.reviewed_rows} reviewed reading corrections; "
Expand Down Expand Up @@ -197,6 +203,7 @@ def external_asset_pack_status(
"required_count": len(spec.required),
"missing": [path.as_posix() for path in missing],
"message": message,
**({"version": version} if spec.validator == "japanese_dictionary" else {}),
**({"version": version, "source": "installed" if dictionary_root == asset_path(asset_root, spec.trees[0])
else "bundled"} if spec.validator == "japanese_dictionary" else {}),
"install_command": "python tools/external_assets.py install <asset-bundle.zip>",
}
8 changes: 4 additions & 4 deletions config/catalog/voice/tts_japanese_dictionary.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
"id": "tts_japanese_dictionary",
"title": {"en-US": "Japanese pronunciation dictionary", "zh-CN": "日语发音词典"},
"description": {
"en-US": "Use an optionally installed dictionary for English words and names in local Japanese speech.",
"zh-CN": "为本地日语语音使用可选安装的英文单词与人名读音词典。"
"en-US": "Use the bundled Japanese pronunciation dictionary or an explicitly installed independent version.",
"zh-CN": "使用应用随附的日语读音词典,或显式安装的独立版本。"
},
"desktop": true,
"apply": "backend_restart",
Expand All @@ -12,8 +12,8 @@
"type": "boolean",
"title": {"en-US": "Use installed Japanese dictionaries", "zh-CN": "使用已安装的日语词典"},
"description": {
"en-US": "Does not download a dictionary. Install japanese-pronunciation separately. Turn off to use the base dictionary; restart the backend after changes.",
"zh-CN": "不会下载词典,请单独安装 japanese-pronunciation。关闭后使用基础词典;更改后需重启后端。"
"en-US": "Application distributions include a default dictionary. Explicit dictionary updates or rollbacks take precedence; personal entries are preserved. Turn off to use the base dictionary and restart the backend.",
"zh-CN": "应用发行包已包含默认词典。显式安装的更新或回退版本优先生效,个人词条保留。关闭后使用基础词典;更改后需重启后端。"
},
"default": true,
"accepted_values": ["true", "false", "1", "0", "yes", "no"],
Expand Down
17 changes: 17 additions & 0 deletions config/japanese_dictionary.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import hashlib
import io
import json
import os
import re
from functools import lru_cache
from pathlib import Path
Expand All @@ -14,6 +15,8 @@

PACK_ID = "japanese-pronunciation"
PACK_TREE = "text/ja/english"
BUNDLED_TREE = "bundled/japanese-pronunciation"
BUNDLED_ENV = "AMADEUS_BUNDLED_JAPANESE_DICTIONARY"
LATIN_WORD_BOUNDARY = "\u3000"
KANJI_COMPOUND = re.compile(r"[\u3400-\u9fff々]{2,}")
KATAKANA_READING = re.compile(r"[\u30a1-\u30fa\u30fc]+")
Expand All @@ -24,6 +27,20 @@
FULL_SOURCE_FILES = ("source.csv", "overlay.csv", "OpenJTalk-COPYING.txt")


def dictionary_pack_root(asset_root: Path) -> Path:
"""Explicit independent installs own selection; app updates replace only the default.

An incomplete explicit installation remains visible as an error instead of
silently reverting a user's update or rollback to the bundled version.
"""
installed = asset_root / PACK_TREE
if installed.exists():
return installed
supplied = os.environ.get(BUNDLED_ENV, "")
bundled = Path(supplied) if supplied else asset_root / BUNDLED_TREE
return bundled if bundled.exists() else installed


def latin_word_key(value: str) -> str:
return unicodedata.normalize("NFKC", value).translate(str.maketrans("’−", "'-")).casefold().strip(" ")

Expand Down
102 changes: 102 additions & 0 deletions docs/evidence/japanese_bundled_delivery_2026-10-22.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
{
"implementation_commit": "92d45b8a676689dbda6d01eb8a4575a6b7158ab4",
"resource": {
"version": "2026.10.22.1-rc2",
"sha256": "a935b3adfded9022afe3b2384cebc2ef1e4023f54f48df1f9ded9ff45b896989",
"resource_identity": "298576bfaf6e553f55631b2bbbea8099deb9b3919903bce8f646ca2ecd791c46"
},
"application_archive": {
"sha256": "c1761b41baa005cc96c8556c1fe1511ffa845a70cc24bd944ee35418fbcd971c",
"bytes": 27620745
},
"cases": {
"fresh-native": {
"source": "bundled",
"version": "2026.10.22.1-rc2",
"source_rows": 134201,
"case_matches": 4
},
"before-app-upgrade": {
"source": "bundled",
"version": "2026.10.20.3",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"after-app-upgrade": {
"source": "bundled",
"version": "2026.10.22.1-rc2",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"independent-install": {
"source": "installed",
"version": "2026.10.20.3",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"independent-update": {
"source": "installed",
"version": "2026.10.22.1-rc2",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"independent-rollback": {
"source": "installed",
"version": "2026.10.20.3",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"rollback-after-app-upgrade": {
"source": "installed",
"version": "2026.10.20.3",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"return-to-bundled": {
"source": "bundled",
"version": "2026.10.22.1-rc2",
"personal_reading": "メヌー",
"personal_reading_match": true
},
"electron-native": {
"source": "bundled",
"version": "2026.10.22.1-rc2",
"source_rows": 134201,
"case_matches": 4
}
},
"bundled_member_count": 9,
"personal_and_settings_preserved": true,
"live_install_unchanged": true,
"electron_resource_members_verified": 9,
"audio_rendered": false,
"merged": false,
"published": false,
"scope": "Bundled application delivery; resource bytes and pronunciation rules are unchanged from the accepted rc2.",
"tests": {
"python_combined": {
"passed": 166
},
"final_archive_targeted": {
"passed": 19
},
"electron_unit": {
"passed": 537,
"failed": 1,
"limitation": "Existing localhost:5173 listener caused EADDRINUSE in the Vite development-server integration test; it was left running."
},
"electron_build": "passed",
"electron_full_unpacked_package": "passed",
"electron_code_only_unpacked_package": "passed",
"core_plus_pinned_japanese_frontend": "passed",
"extracted_source_electron_staging": "passed"
},
"limitations": [
"Windows artifact lifecycle qualification; native contract CI is configured on Windows, Linux and macOS.",
"Actual native Japanese frontend output was checked, not newly synthesized audio; dictionary bytes are unchanged from prior listening acceptance.",
"Source extraction and Electron resource handoff were verified. This is not a claim that the Electron installer independently installs the Python environment.",
"Existing independent dictionary installs remain selected across application upgrades, including older format-1 versions.",
"The resource download URL remains unset; full cloud release packaging requires an approved HTTPS input matching the committed hash.",
"No application version bump, release tag, merge or publication is authorized by this receipt. Wait for the console UI/UX work and combined acceptance."
]
}
20 changes: 10 additions & 10 deletions docs/external_asset_bundles.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,16 +137,16 @@ reviewed before publishing either voice archive.

## Archive contract

The independent optional `japanese-pronunciation` pack contains a pinned,
corrected Japanese word/name CSV and its provenance/license notices. Build,
verify and install use the existing commands; compiled and personal dictionaries
are excluded. This requires an application version with the new catalog entry.
See [Japanese pronunciation dictionary](japanese_pronunciation_dictionary.md).
Publish it alongside the compatible application source ZIP in the same GitHub
Release, with checksums and the tested version pair. It remains optional and
does not require rebuilding other runtime archives. Later format changes ship
updated resources with the new application; old pack contracts need not be
retained indefinitely. Keep published filenames and bytes unchanged.
Application releases include the independently versioned `japanese-pronunciation`
resource as a bundled default; consumers do not install it separately. Maintainers
pin and validate its archive and all nine files before building either source or
Electron application distributions. Explicit independent installs under
`assets/text/ja/english` take precedence, preserving data-only updates and rollback
across application upgrades. Personal `assets/text/ja/user.csv` stays separate.
See [Japanese pronunciation dictionary](japanese_pronunciation_dictionary.md)
for staging, release gates, actual loading checks and returning to the default.
The same immutable independent ZIP may be supplied with the application Release
for data-only updates; other voice/model/media archives remain unchanged.

The optional `voice-kurisu-emotions` pack adds paired Japanese semantic references
to an existing Windows CUDA V3 installation. It uses the same bundle contract
Expand Down
Loading
Loading