iree-android: IreeMoonshineStream — the streaming Moonshine v2 speech-to-text runtime - #450
Merged
Merged
Conversation
…h-to-text runtime libskainet_moonshine_stream.so (arm64-v8a + armeabi-v7a, Vulkan + local-task) drives the five graphs of MoonshineV2ExportCli -- frontend, encoder, adapter, masked prefill, dynamic with-past step -- plus the shared decoder parameter archive as a real streaming loop on the device: PCM in (64-frame windows, hop 0.88 s, encode stages on the device, incremental greedy decode), cumulative partial transcripts out, one exact full re-decode on finish(). IreeMoonshineStream is the Kotlin binding, shaped like IreeKvSession: files by absolute path, no Context, every native failure an IllegalStateException. Built by native/build-moonshine-stream.sh with the same image and cache as the other two libraries; the tiny geometry (320 / 6 / 8) is compiled in, GEOMETRY="-D…" selects the split-width small variant. This is the last piece a Moonshine cartridge needed from a released artifact: the exporter (0.56.1) and now the runtime.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
libskainet_moonshine_stream.so+IreeMoonshineStream: the streaming Moonshine v2 speech-to-text runtime, next toIreeKvSessionandIreeRedecodeSessioninllm-runtime:iree-android.It drives the five graphs of
MoonshineV2ExportCli(0.56.1) — frontend, encoder, adapter, masked prefill, dynamic with-past step — plus the shared decoder parameter archive as a real streaming loop on the device:feedPcm(FloatArray)with mono 16 kHz PCM returns the updated cumulative partial transcript,finish()runs one exact full re-decode and returns the final transcript,reset()aborts. Files by absolute path (IreeMoonshineStream.Files/filesIn(dir)), no Context, native failures asIllegalStateException.Build
native/build-moonshine-stream.sh <abi> --vulkan— sameskainet/iree-android:3.11.0image, cache and--linkset as the other two libraries. Both ABIs'.sos are committed underjniLibslike the existing ones. Geometry is compiled in (tiny 320/6/8, which the English and German tiny streaming checkpoints share);GEOMETRY="-DL=10 -DHD=64 -DDIM=512 -DENC_DIM=620"builds the split-widthsmallvariant.Verification
:llm-runtime:iree-android:assembleReleasepackages all three libraries for both ABIs;nmshows the fiveJava_sk_ainet_transformers_iree_android_IreeMoonshineStream_*symbols.Why
With the exporter released in 0.56.1, the runtime was the last piece a Moonshine cartridge needed from a released artifact rather than from C source in the cartridge repo.