Skip to content

Fix DiffSinger rendering (Phonemizer & Multi-Speaker) & Misc. - #287

Open
KahazaTester wants to merge 1 commit into
vocoder712:devfrom
KahazaTester:fix/ds-dev
Open

Fix DiffSinger rendering (Phonemizer & Multi-Speaker) & Misc.#287
KahazaTester wants to merge 1 commit into
vocoder712:devfrom
KahazaTester:fix/ds-dev

Conversation

@KahazaTester

Copy link
Copy Markdown

Pull Request: DiffSinger Engine Fixes, Singer Import Auto-Detection & L10n Fallback

This should fix the following issues: #277 #280

Summary of Changes

1. DiffSinger Renderer & Phonemizer Fixes

  • Eliminate np.dot Crashes (DiffSingerSpeakerEmbedManager.cs):
    • Replaced np.dot matrix multiplication in both PhraseSpeakerEmbedByFrame (rendering) and PhraseSpeakerEmbedByPhone (duration model phonemization) with managed per-frame normalized weighting and direct float buffer copies, fixing NullReferenceException crashes on mobile runtimes.
    • Added path-segment suffix matching and fallback handling for speaker embeddings.
  • Speaker Subbank Fallback (DiffSingerBasePhonemizer.cs):
    • Added fallback hierarchy in GetSpeakerAtIndex (tone+color match -> color match -> first defined subbank) to correctly resolve singers with named voice colors on default notes.
  • Multi-Dict G2P Language Prefixing (DiffSingerG2pPhonemizer.cs):
    • Fixed multi-dictionary phoneme prefixing by checking !string.IsNullOrEmpty(langCode) instead of useLangId, ensuring voicebanks with use_lang_id: false correctly map G2P phonemes to <langCode>/<phoneme>.

2. Singer Import Auto-Detection (ClassicSingerSetupViewModel.cs)

  • Added archive inspection heuristic when importing singer archives without explicit singer_type in character.yaml:
    • Detects dsconfig.yaml -> sets SingerType = "diffsinger".
    • Detects enuconfig.yaml -> sets SingerType = "enunu".
    • Defaults to "utau" otherwise.

3. Localization Default (LocalizationManager.cs)

  • Changed SystemLanguageFallbackCode from zh-Hans to en so unrecognized system languages default to English on first start.

Verification

  • Tested on Android 16 (NNAPI & CPU).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant