Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
cmake_minimum_required(VERSION 3.20)
# macOS: the oldest release the library loads on. Without it the build machine's SDK version is used.
set(CMAKE_OSX_DEPLOYMENT_TARGET "12.0" CACHE STRING "Minimum macOS version")
project(ModAVDExtras VERSION 0.2.0 LANGUAGES C)
project(ModAVDExtras VERSION 0.3.0 LANGUAGES C)

set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,15 @@
- 遇到新帧写到一半时重拷;
- 告诉调用方现在能不能用。
- 点击走模拟器控制台的 `event send`,发标准的多点触控事件(槽位、跟踪编号、坐标、压力),到达模拟器自己的虚拟触屏:安卓看到的是屏幕上的手指,和在模拟器窗口里点一样,不是应用注入的事件,客体里也不用装东西。最多 10 根手指。
- 按键和文字也走控制台(接口版本 3):
- 按键(`modavd_key`,安卓 KEYCODE)按的是模拟器键盘上的键,返回、主页、多任务这些也在内。要模拟器的控制台有 `event key`(比 ModAVD 0.40 新的模拟器,`MODAVD_CAP_KEY`);更旧的模拟器上只有能打出字符的键和编辑框里的编辑键可用。
- 文字(`modavd_input_text`,UTF-8):编辑框有焦点、ModAVD 的宿主输入法在用时,原样交给编辑框,任意语言;否则 ASCII 用键盘打。两样都不满足时返回 `MODAVD_E_UNSUPPORTED`,什么都不打。
- 库**不自己退回 adb**:出错时返回错误码,让调用方按自己的规则换别的办法。
- 只依赖系统库:Windows 上是 `KERNEL32`、`WS2_32`,C 运行库静态链接;POSIX 上是 `shm_open`、`mmap` 和套接字。

## 状态

- 截图(接口版本 1)和点击(接口版本 2,`MODAVD_CAP_TOUCH`)。按键、文字的函数已经声明,现在返回 `MODAVD_E_UNSUPPORTED`:控制台发的按键会进触屏设备、被丢掉,`event text` 会悄悄丢掉键盘映射里没有的字。
- 截图(接口版本 1)、点击(接口版本 2,`MODAVD_CAP_TOUCH`)、按键和文字(接口版本 3,`MODAVD_CAP_KEY`、`MODAVD_CAP_TEXT`)。
- Windows:截图这套逻辑在 ModAVD 上已经用了一个多月,形式是下面的兼容外壳。
- macOS(Apple Silicon):11-09 有贡献者用官方 Emulator 37.2.13、API 35 ARM64 实测过截图。
- Linux:能编,单元测试在 CI 上跑;还没有在模拟器上实测。
Expand Down Expand Up @@ -112,7 +115,7 @@ ModAVDExtras is a small C library that reads the screen of a [ModAVD](https://gi
- Frames come from the emulator's own shared memory: after `screenrecord webrtc start` (or `-share-vid`), the emulator writes every frame to a region named after its console port: `SHM_videmulator<port>` on Windows, `videmulator<port>` elsewhere. The layout is a 24-byte header (width, height, fps, frame number, timestamp) followed by BGRA pixels.
- The library finds the console port from an adb address, starts frame sharing through the console when it is off, copies a frame in the format the caller asks for (copying again if a new frame arrived meanwhile), and reports whether capture is usable. It never falls back to adb by itself.
- Touch (API version 2, `MODAVD_CAP_TOUCH`) goes through the console's `event send` as standard multi-touch events (slot, tracking id, position, pressure) to the emulator's own virtual touch screen: Android sees a finger on the screen, as with a click on the emulator window, not an injected event, and nothing is installed in the guest. Up to 10 contacts.
- Keys and text are declared and return `MODAVD_E_UNSUPPORTED`: keys sent through the console reach the touch screen device and are dropped, and `event text` silently drops characters missing from the keyboard map. See [include/modavd_extras.h](include/modavd_extras.h).
- Keys and text (API version 3) go through the console too. `modavd_key` presses keys of the emulator's keyboard by Android key code, BACK, HOME and APP_SWITCH included; it needs the console's `event key` (ModAVD emulators newer than 0.40, `MODAVD_CAP_KEY`), and on older emulators only keys that type a character and the editing keys of a focused text field work. `modavd_input_text` hands UTF-8 text in any language to the focused text field through ModAVD's host input method, or types ASCII on the keyboard when there is none. See [include/modavd_extras.h](include/modavd_extras.h).
- `external_renderer_ipc.dll` is a Windows-only compatibility shell. It exports the four capture functions that MaaCore looks up in MuMu's DLL, on top of the same code, so MaaCore versions that do not know ModAVD can still capture from it. AUTO-MAS uses it today. It will be retired once MaaCore can connect to ModAVD directly.
- Status: Windows has used this capture code for over a month through the shell below; on Apple Silicon macOS a contributor tested capture with the official Emulator 37.2.13 and API 35 ARM64; Linux builds and passes the unit tests in CI but has not been tested with an emulator. From v0.1.2 the [releases](https://github.com/ModAVD/ModAVDExtras/releases) carry `modavd_extras.dll` (Windows x64), `libmodavd_extras.dylib` (macOS arm64) and `libmodavd_extras.so` (Linux x64), plus the Windows-only shell.
- Build: `build.ps1` on Windows (Visual Studio 2022 Build Tools, CMake, Ninja); CMake on Linux and macOS. The unit tests need no emulator.
Expand Down
33 changes: 27 additions & 6 deletions include/modavd_extras.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ extern "C" {

/* Bumped whenever a function or field is added, or a declared one starts to work.
* Nothing is ever removed.
* 1: capture. 2: touch (modavd_touch_*). */
#define MODAVD_API_VERSION 2
* 1: capture. 2: touch (modavd_touch_*). 3: keys and text (modavd_key, modavd_input_text). */
#define MODAVD_API_VERSION 3

typedef struct modavd_instance* modavd_handle;

Expand All @@ -60,8 +60,8 @@ enum {
enum {
MODAVD_CAP_SCREENCAP = 1u << 0,
MODAVD_CAP_TOUCH = 1u << 1, /* API version 2 */
MODAVD_CAP_KEY = 1u << 2, /* not yet */
MODAVD_CAP_TEXT = 1u << 3, /* not yet */
MODAVD_CAP_KEY = 1u << 2, /* API version 3: this emulator takes every key of modavd_key() */
MODAVD_CAP_TEXT = 1u << 3, /* API version 3 */
};

/* Pixel formats for modavd_capture(). Rows are packed: stride = width * bytes per pixel. */
Expand Down Expand Up @@ -178,10 +178,31 @@ MODAVD_API int modavd_touch_up(modavd_handle h, int display, int contact);
/* Lifts every contact this handle pressed. modavd_close() does the same. */
MODAVD_API int modavd_touch_release_all(modavd_handle h);

/* ---- keys and text: declared, MODAVD_E_UNSUPPORTED for now ---- */
/* ---- keys and text (API version 3) ---- */

/* Android key code (KEYCODE_*); down = 1 press, 0 release. */
/* Presses (down = 1) or releases (down = 0) a key of the emulator's keyboard,
* given as an Android key code (KEYCODE_*): Android sees a hardware key, not an
* injected event. Only display 0.
*
* With MODAVD_CAP_KEY (ModAVD emulators newer than 0.40) every key of that
* keyboard works: letters, digits, punctuation, ENTER, TAB, SPACE, DEL, the
* arrows, SHIFT / CTRL / ALT, and BACK, HOME, APP_SWITCH, MENU, SEARCH, POWER,
* the volume and media keys. ESCAPE is its Esc key, which Android reads as BACK.
* Without it, and for keys the keyboard lacks, only these work, and the press
* sends both press and release:
* - keys that type a character (letters, digits, ENTER, TAB, SPACE, punctuation);
* - the editing keys (DEL, FORWARD_DEL, the arrows, MOVE_HOME, MOVE_END) while
* a text field has the focus and ModAVD's host input method is the current one.
* MODAVD_E_UNSUPPORTED: this emulator cannot send that key now; use another way. */
MODAVD_API int modavd_key(modavd_handle h, int display, int android_keycode, int down);

/* Types UTF-8 text into what has the focus. '\n' is ENTER; other control
* characters except '\t' are MODAVD_E_INVALID.
* - While a text field has the focus and ModAVD's host input method
* (org.modavd.ime, installed by ModAVD) is the current one, the text goes
* to the field as it is, in any language.
* - Otherwise printable ASCII is typed on the emulator's keyboard (a US layout).
* MODAVD_E_UNSUPPORTED: text beyond ASCII and no such text field; nothing was typed. */
MODAVD_API int modavd_input_text(modavd_handle h, const char* utf8);

#ifdef __cplusplus
Expand Down
4 changes: 4 additions & 0 deletions release/0.3.0.SHA256SUMS.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
ee0cbaff8e96f82c03e72b292e5e999afe4b4d0486c1494757798273bb1b4b61 external_renderer_ipc.dll
406813a35f504492b59a290a86612a0f235592cc57bbccefa1d8cfdcedf4ff10 libmodavd_extras.dylib
41bae8333146f59f1e5c08c579924a364ead38e74d149e8388cea04834fdac02 libmodavd_extras.so
4837c513ab4eb8a6393d2312163d4c1bffaf17fc1f7099208b1df0856d155b88 modavd_extras.dll
55 changes: 55 additions & 0 deletions release/0.3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# ModAVDExtras v0.3.0

- `modavd_extras.dll`:`4837c513ab4eb8a6393d2312163d4c1bffaf17fc1f7099208b1df0856d155b88`
- `external_renderer_ipc.dll`:`ee0cbaff8e96f82c03e72b292e5e999afe4b4d0486c1494757798273bb1b4b61`
- `libmodavd_extras.dylib`:`406813a35f504492b59a290a86612a0f235592cc57bbccefa1d8cfdcedf4ff10`
- `libmodavd_extras.so`:`41bae8333146f59f1e5c08c579924a364ead38e74d149e8388cea04834fdac02`

这一版加上按键和输入文字:`modavd_key`、`modavd_input_text` 能用了,接口版本升到 3。截图、点击和 MuMu 外壳的行为不变。

## 新增

- 输入文字(`MODAVD_CAP_TEXT`):`modavd_input_text`,UTF-8(#19)。
- 编辑框有焦点、ModAVD 的宿主输入法(`org.modavd.ime`)在用时,文字原样交给编辑框,任意语言;换行发成回车键。
- 否则,可打印的 ASCII 由模拟器的键盘打出来,约 1.2 ms 一个字符。
- 两样都不满足(不是 ASCII,又没有这样的编辑框)时返回 `MODAVD_E_UNSUPPORTED`,什么都不打。
- 按键(`MODAVD_CAP_KEY`):`modavd_key`,安卓 KEYCODE(#19)。
- 模拟器的控制台有 `event key` 时,按的是模拟器键盘上的真实按键,按下、抬起分开发,返回、主页、多任务也在内。这条命令要比 ModAVD 0.40 新的模拟器,那时才有 `MODAVD_CAP_KEY`。
- ModAVD 0.40 的模拟器上:能打出字符的键(字母、数字、回车、Tab、空格、标点)可用;编辑键(退格、方向键等)在编辑框有焦点、宿主输入法在用时可用;其余返回 `MODAVD_E_UNSUPPORTED`。这两条路按下时就把按下和抬起一起发了。
- Esc 键被模拟器键盘的键位表读作返回。
- `modavd_api_version()` 返回 3,`modavd_capabilities()` 多了 `MODAVD_CAP_TEXT`(接口版本 3 起恒有)和 `MODAVD_CAP_KEY`(控制台有 `event key` 时才有)。
- `modavd_version_string()` 返回 `"0.3.0"`。

## 说明

- `modavd_capabilities()` 第一次调用会连一次模拟器控制台,用来判断有没有 `event key`。
- 和点击一样不经 adb,客体里不用装东西;库不自己退回 adb,做不了时返回错误码,由调用方换别的办法。
- 只认接口版本 1、2 的程序不受影响:原有函数和行为都没变。
- `external_renderer_ipc.dll` 的源码没改,但它静态链接了库的代码,所以文件和 v0.2.0 不一样。
- Linux、macOS 上按键和文字只过了编译和单元测试,没有在模拟器上实测。

## 许可证与源码

- LGPL-3.0-or-later。正文是 tag 下的 [LICENSE](https://github.com/ModAVD/ModAVDExtras/blob/v0.3.0/LICENSE)(LGPL-3.0)和 [COPYING](https://github.com/ModAVD/ModAVDExtras/blob/v0.3.0/COPYING)(GPL-3.0);转发这些库时请一起带上。
- 源码:<https://github.com/ModAVD/ModAVDExtras/tree/v0.3.0>。

## 下载与核对

四个文件都由本仓库的 release workflow 在 GitHub 上编出,和 tag 下的 `release/0.3.0.SHA256SUMS.txt` 核对过才上传。编译环境和 v0.2.0 一样:

- Windows:`windows-2025`,`cmake -A x64`,`--config Release`。两个 DLL 都静态链接 C 运行库,只依赖 `KERNEL32`、`WS2_32`,编译带 `/Brepro`。
- macOS:`macos-15`(Apple Silicon),`cmake -DCMAKE_BUILD_TYPE=Release`。
- Linux:`ubuntu-24.04`(x64),`cmake -DCMAKE_BUILD_TYPE=Release`。

请用附件 `SHA256SUMS.txt` 核对:Windows 上 `Get-FileHash .\modavd_extras.dll -Algorithm SHA256`,macOS 上 `shasum -a 256 libmodavd_extras.dylib`,Linux 上 `sha256sum libmodavd_extras.so`。

## English

This release adds keys and text: `modavd_key` and `modavd_input_text` now work, and the API version is 3. Capture, touch and the MuMu shim behave as before.

- Text (`MODAVD_CAP_TEXT`): `modavd_input_text`, UTF-8 (#19). With a focused text field and ModAVD's host input method (`org.modavd.ime`) active, the text is handed to the field as is, in any language, and a newline is sent as ENTER. Otherwise printable ASCII is typed on the emulator's keyboard, about 1.2 ms per character. When neither applies it returns `MODAVD_E_UNSUPPORTED` and types nothing.
- Keys (`MODAVD_CAP_KEY`): `modavd_key`, Android key codes (#19). When the emulator console has `event key`, real keys of the emulator's keyboard are pressed, down and up sent separately, BACK, HOME and APP_SWITCH included; that needs an emulator newer than ModAVD 0.40, and only then is `MODAVD_CAP_KEY` set. On ModAVD 0.40 emulators, keys that type a character work, editing keys (DEL, arrows) work in a focused text field with the host input method active, and the rest return `MODAVD_E_UNSUPPORTED`; on those two paths the press sends down and up together. The emulator keyboard's layout reads Esc as BACK.
- `modavd_api_version()` returns 3; `modavd_capabilities()` adds `MODAVD_CAP_TEXT` (always, from API version 3) and `MODAVD_CAP_KEY` (when the console has `event key`), and its first call opens a console session once. `modavd_version_string()` returns `"0.3.0"`.
- Functions of API versions 1 and 2 are unchanged. `external_renderer_ipc.dll` has no source changes, but it links the library code statically, so the file differs from v0.2.0. On Linux and macOS, keys and text are only built and unit-tested, not tested with an emulator.
- License: LGPL-3.0-or-later; the texts are [LICENSE](https://github.com/ModAVD/ModAVDExtras/blob/v0.3.0/LICENSE) (LGPL-3.0) and [COPYING](https://github.com/ModAVD/ModAVDExtras/blob/v0.3.0/COPYING) (GPL-3.0) at the tag. Ship them with the libraries. Source: <https://github.com/ModAVD/ModAVDExtras/tree/v0.3.0>.
- All four files are built by this repository's release workflow on GitHub, the same way as v0.2.0, and checked against `release/0.3.0.SHA256SUMS.txt` at the tag before upload. Check them with the attached `SHA256SUMS.txt`.
12 changes: 10 additions & 2 deletions src/console.c
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ static int read_until_ok(mx_socket s, char* buf, int cap)

static int send_line(mx_socket s, const char* line)
{
char cmd[512];
char cmd[MX_CONSOLE_LINE_MAX + 3];
int n = snprintf(cmd, sizeof(cmd), "%s\r\n", line);
if (n <= 0 || (size_t)n >= sizeof(cmd)) {
return -1;
Expand Down Expand Up @@ -162,8 +162,11 @@ int mx_console_open(int port, int timeout_ms, mx_console* c)
return MODAVD_OK;
}

int mx_console_send(mx_console* c, const char* command)
int mx_console_query(mx_console* c, const char* command, char* reply, size_t cap)
{
if (cap) {
reply[0] = '\0';
}
if (c->sock == (intptr_t)MX_BAD_SOCKET) {
return MODAVD_E_NOT_FOUND;
}
Expand All @@ -179,9 +182,14 @@ int mx_console_send(mx_console* c, const char* command)
mx_console_close(c);
return MODAVD_E_NOT_FOUND;
}
if (cap) {
snprintf(reply, cap, "%s", buf);
}
return r == 1 ? MODAVD_OK : MODAVD_E_IO;
}

int mx_console_send(mx_console* c, const char* command) { return mx_console_query(c, command, NULL, 0); }

void mx_console_close(mx_console* c)
{
if (c->sock == (intptr_t)MX_BAD_SOCKET) {
Expand Down
8 changes: 7 additions & 1 deletion src/core.c
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,13 @@ void modavd_close(modavd_handle h)
free(h);
}

uint32_t modavd_capabilities(modavd_handle h) { return h ? MODAVD_CAP_SCREENCAP | MODAVD_CAP_TOUCH : 0; }
uint32_t modavd_capabilities(modavd_handle h)
{
if (!h) {
return 0;
}
return MODAVD_CAP_SCREENCAP | MODAVD_CAP_TOUCH | MODAVD_CAP_TEXT | mx_input_capabilities(h->input);
}

int modavd_console_port(modavd_handle h) { return h ? h->console_port : MODAVD_E_INVALID; }

Expand Down
Loading
Loading