Skip to content

Android: run the tests on an x86_64 emulator, in CI and before a release - #823

Merged
ASDAlexander77 merged 1 commit into
mainfrom
android-emulator-tests
Nov 12, 2026
Merged

ASDAlexander77 merged 1 commit into
mainfrom
android-emulator-tests

Conversation

@ASDAlexander77

Copy link
Copy Markdown
Owner

Android programs now run on an emulator, not just link. This PR runs the tester's tests and the default library's tests, built for x86_64 Android (API 29) under gc, rc, none and own, on an x86_64 emulator. It does this in CI and before each release.

tslang/test/android/run_tests.mjs

  • --build compiles each test (-mtriple=x86_64-linux-android29 --opt) and links it through the NDK.

    • The libraries come from a Windows package or the repo's own Android builds: the default library, the collector and the async runtime (--android-lib=<package>\android\x86_64\lib).
    • Each program is stripped with the NDK's llvm-strip. Most of a program's size is debug info from the collector, libc++ and the default library's C++ wrappers: unstripped, the build is 4.6 GB; stripped, 832 MB (320 MB as a tar.gz).
    • It writes the programs and manifest.json to out/.
  • --run needs only adb and out/.

    • It runs one leg (a suite under a model) at a time: it pushes the leg, runs its programs four at a time on the device, pulls the results and deletes the leg. All legs together would not fit on an emulator's data partition.
    • Each program runs under timeout and ulimit -v. An out-of-memory kill of an adb-shell process can take the emulator down.
    • What comes back for each program: its exit code, whether it printed done. / ALL DONE, how much it wrote to stderr, the last lines of its output, and how long it ran.
    • On API 29, toybox timeout returns 124 for a program that died from a signal as well as for a timeout. The runner tells them apart by the run time: a crash is a FAIL, not a TIMEOUT.
  • Shared code: the test list, skips and pass rules move into tslang/test/suites.mjs, which the wasm runner now uses too, unchanged otherwise.

  • android_skips.txt, each with its reason:

    • 00cppvtable_hidden_return.ts: Windows-only by design, as in ctest.
    • fetch.ts: Android's default library has no HTTP.
    • json.ts under none: about 5 GB never freed.
    • 00global_interface_initializer.ts under own: the same assert fails natively.

    The last three are the same as in the wasm skips.

CI (.github/workflows/test-android.yml)

  • Only the Windows package ships Android's libraries, so the programs are built on Windows: the job unpacks the package and builds with its tslang.exe, the runner's NDK and android/x86_64/lib. It uploads out/ as an artifact.
  • They run on Linux, which has KVM: the job turns KVM on and runs them on an API 29 x86_64 emulator (reactivecircus/android-emulator-runner).
  • When it runs:
    • A release calls it with the package it has just built (test_android), and create_release waits for it.
    • A pull request that changes the runner or the workflow runs it against the latest published release's Windows package. That is how this PR tests it before it can block a release.

arm64-v8a and armeabi-v7a are still only linked (test-compile-android-link, #822). An x86_64 emulator runs them only through a binary translator, which isn't reliable enough to test with.

Testing

Locally, on an API 29 x86_64 emulator, with NDK r30:

Leg gc rc none own
tester 653 653 653 488 (+164 that own cannot compile natively either)
default library 211 208 207 207
  • No failures. Building took 81 s; running all eight legs took about 100 s.
  • Taking the skips list away makes its crashes show up as FAIL ("ended by a signal"), not TIMEOUT.
  • The full wasm matrix (release and debug) is unchanged with the shared suites.mjs.

🤖 Generated with Claude Code

…4 emulator

tslang/test/android/run_tests.mjs builds the tests for x86_64 Android (API 29) under gc, rc, none
and own (--build: tslang, the NDK and the Android libraries; programs stripped) and runs them on a
device or emulator through adb (--run: a leg at a time, four programs at a time, each under
timeout and a memory limit; exit code, markers and output tails come back). The test list, skips
and pass rules are shared with the wasm runner (tslang/test/suites.mjs).

test-android.yml builds the programs on Windows with a Windows package (the only one that ships
Android's libraries) and runs them on an API 29 x86_64 emulator on Linux (KVM). create-release
runs it against the package it has just built (test_android); a pull request that changes the
runner runs it against the latest published release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ASDAlexander77
ASDAlexander77 merged commit 9568651 into main Nov 12, 2026
1 of 8 checks passed
@ASDAlexander77
ASDAlexander77 deleted the android-emulator-tests branch November 12, 2026 13:05
ASDAlexander77 added a commit that referenced this pull request Nov 12, 2026
…etplace actions are not allowed) (#825)

* Android emulator tests: boot the emulator with the runner's SDK, not a marketplace action

The repository allows GitHub's own actions only, so reactivecircus/android-emulator-runner was
refused and test-android.yml - and create-release.yml, which calls it - failed at startup (the
re-tagged v0.0-pre-alpha125 run, and #823's own run). The job now installs the emulator and an
API 29 x86_64 image with the runner's sdkmanager, creates the AVD, boots it headless on KVM, waits
for sys.boot_completed and runs the tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Android emulator tests: take the newest release, not GitHub's "latest"

Every release is a pre-release, which GitHub's "latest" leaves out, so `gh release download` with
no tag found none ("release not found") in #825's run. List the newest non-draft release and
download its Windows package by tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant