Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ jobs:
- name: Check the fetch item with and without the old batch file, and that it keeps LOCALAPPDATA private
shell: pwsh
run: ./tests/fetch-item.ps1
- name: Check the dataverify item pattern, plan and mismatch reporting against a fake guest
shell: pwsh
run: ./tests/dataverify.ps1
- name: Scan tracked files for private identity and paths
run: python3 .modavd/scripts/privacy.py .

Expand Down
5 changes: 3 additions & 2 deletions README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ and the bundled scripts in `package/` and the bridge in `bridge/` of the ModAVD

## How to run

PowerShell 7 (`pwsh`); the helper scripts use Python 3.12.
PowerShell 7 (`pwsh`); the helper scripts use Python 3.12; the `dataverify` item also needs Go on the host (it builds the small writer that runs in the guest).

```powershell
pwsh -NoProfile -File regress.ps1 -Suite suites\release.psd1 -Scope emulator -Sdk <new SDK> -Baseline <SDK of the previous release>
Expand Down Expand Up @@ -69,7 +69,7 @@ The suite refers to keys with `{Local:<key>}`, and the item scripts take their d
| `PackageScripts` | The `package\` of the ModAVD repository (the bridge install script `modavd-bridge.ps1` is in it) | runner, bridge, upgrade, game-real |
| `ProductionRoots` | Root folders the tests never touch, in addition to `D:\ModAVD` (not instead of it): list the root you use yourself every day here too. A test root, output, work or instance folder under them stops the run | runner, all items |
| `TestRoots` | The only test roots the regression may use (optional): when set, only the listed root folders themselves are accepted, and a `TestRoot`, `PackageRoot` or `-Root` pointing elsewhere stops the run | runner, all items |
| `TestRoot` | The ModAVD root folder used for testing: the existing flow-a / flow-b under `E:\testbed`; a folder that is not an existing root (no `mas-avd.json`), or whose `mas-avd.json` does not set its own `adbServerPort` and `scriptAdbServerPort`, stops the run, none is created (`portBase` when the machine also has a production root folder; instance 0 has the bridge installed and is logged in with the *Onmyoji* test account; the other numbers are created temporarily and deleted after the run) | bridge, bridge-default, datadisk, yys |
| `TestRoot` | The ModAVD root folder used for testing: the existing flow-a / flow-b under `E:\testbed`; a folder that is not an existing root (no `mas-avd.json`), or whose `mas-avd.json` does not set its own `adbServerPort` and `scriptAdbServerPort`, stops the run, none is created (`portBase` when the machine also has a production root folder; instance 0 has the bridge installed and is logged in with the *Onmyoji* test account; the other numbers are created temporarily and deleted after the run) | bridge, bridge-default, datadisk, dataverify, yys |
| `PackageRoot` | The test root for the items that install a package (`TestRoot` when not set): it is turned into the package under test first and stays on the new package afterwards | upgrade, ime-pkg |
| `PrevPackage` | The package staging folder of the previous production version (a fixed copy) | upgrade |
| `Launcher` | The launcher `avd-p0.ps1` (starts an AVD headless or windowed, with parameters such as `-AdbPort` and `-VulkanFix`) | game-real, ime |
Expand Down Expand Up @@ -130,6 +130,7 @@ In the game UI, do only fixed operations such as "launch" and "tap Enter Game".
| ark / r1999 / sr / ef | `items\game-real.ps1` | A 15-minute real run of each game with its own automation scripts (the round with the bridge installed), with the frame rate compared with `baselines\fps.json` |
| bridge | `items\bridge.ps1` | The whole flow of installing / uninstalling the bridge (Windows PowerShell 5.1); it creates instance 6 and deletes it after the run |
| datadisk | `items\datadisk.ps1` | The inode cap on a new disk and growing with `set --data-gb`; it creates instance 3 and deletes it after the run |
| dataverify | `items\dataverify.ps1` | Disk read/write data check (ModAVD/ModAVD#138): creates instance 4 and, under `/data/unencrypted/` in the guest, writes a 1 GiB file whose pattern is computed from a seed (single writes of 4 KiB–1 MiB, ordinary and `O_DIRECT` writes mixed), then runs `sync`, drops the page cache and reads the hash back; after a normal shutdown and a cold boot it reads the hash again and compares it with the one the host computes from the seed, and on a mismatch pulls the file back and reports the first differing offset and the number of bad bytes; the file and the instance are deleted afterwards. `-SizeMiB` changes the total, `-Seed` and `-DirectPercent` the pattern and the share of direct writes, `-Keep` keeps the instance. The guest `dd` has no `oflag=direct`, so the writing is done by `items\dataverify-guest.go`, which needs Go on the host to build |
| upgrade | `items\upgrade.ps1` | Installs the previous release package into `PackageRoot` following the package manifests, creates a temporary instance and writes data, overlays the new package following "How to update", then checks boot, data, growing the disk, and that a normal boot installs the bridge automatically. When `-Next` is a package staging folder (with `components\modavd-ime.apk`), it also checks that the host IME came with the package, that the installed one is the officially signed one, and that it is set as the default (not checked when `-Next` is an SDK). Afterwards the temporary instance is deleted and the root folder stays on the new package |
| yys | `items\yys.ps1` | *Onmyoji* cold-boots into the game and the frame rate is measured (instance 0 of the test root, bridge installed, already logged in); then MaaYYs (`items\maayys_run.py`) really fights Mitama battles |
| ef-ab | `items\ef-ab.ps1` | *Arknights: Endfield* cold boots, alternating new and old builds; it measures only when the host is idle |
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ ModAVD 仓库里的随包脚本 `package/` 和执行桥 `bridge/`,所以单独

## 怎么跑

PowerShell 7(`pwsh`),辅助脚本用 Python 3.12。
PowerShell 7(`pwsh`),辅助脚本用 Python 3.12;`dataverify` 项还要宿主装有 Go(编客体里的写入小程序)。

```powershell
pwsh -NoProfile -File regress.ps1 -Suite suites\release.psd1 -Scope emulator -Sdk <新 SDK> -Baseline <上一个发布版的 SDK>
Expand Down Expand Up @@ -69,7 +69,7 @@ pwsh -NoProfile -File regress.ps1 -Suite suites\release.psd1 -Scope emulator -Sd
| `PackageScripts` | ModAVD 仓库的 `package\`(装桥脚本 `modavd-bridge.ps1` 在里面) | 运行器、bridge、upgrade、game-real |
| `ProductionRoots` | 测试不碰的根目录,在 `D:\ModAVD` 之外再加(不是替换它):你自己日常在用的根也写进来。测试根、输出、工作目录、实例目录在它们下面就停 | 运行器、各项 |
| `TestRoots` | 回归只许用的测试根(可以不写):写了就只认列出来的根目录本身,`TestRoot`、`PackageRoot`、`-Root` 指到别处就停 | 运行器、各项 |
| `TestRoot` | 测试用的 ModAVD 根目录:`E:\testbed` 下已有的 flow-a / flow-b(`mas-avd.json` 里要写自己的 `adbServerPort`、`scriptAdbServerPort` 和 `portBase`;0 号已装桥、登着阴阳师测试号,其余编号临时建、跑完删)。不是已有根目录(没有 `mas-avd.json`)、没写这两个 adb 端口就停,不新建 | bridge、bridge-default、datadisk、yys |
| `TestRoot` | 测试用的 ModAVD 根目录:`E:\testbed` 下已有的 flow-a / flow-b(`mas-avd.json` 里要写自己的 `adbServerPort`、`scriptAdbServerPort` 和 `portBase`;0 号已装桥、登着阴阳师测试号,其余编号临时建、跑完删)。不是已有根目录(没有 `mas-avd.json`)、没写这两个 adb 端口就停,不新建 | bridge、bridge-default、datadisk、dataverify、yys |
| `PackageRoot` | 装包类的项用的测试根(不写就用 `TestRoot`):先装成要测的包,测完根目录停在新包上 | upgrade、ime-pkg |
| `PrevPackage` | 上一个生产版的组包目录(固定副本) | upgrade |
| `ComponentCache` | 放着 `platform-tools_r37.0.1-win.zip`、`x86_64-34_r14.zip` 的目录(可以没有:没有就从网上下载一次) | fetch |
Expand Down Expand Up @@ -131,6 +131,7 @@ REGRESS-RESULT {"pass":true,"summary":"到登录页 41 秒;进游戏后 52.7 f
| ark / r1999 / sr / ef | `items\game-real.ps1` | 各游戏自己的自动化脚本实跑 15 分钟(装桥那轮),帧率和 `baselines\fps.json` 比 |
| bridge | `items\bridge.ps1` | 执行桥装 / 卸全流程(Windows PowerShell 5.1),新建 6 号实例跑完删掉 |
| datadisk | `items\datadisk.ps1` | 新盘 inode 封顶、`set --data-gb` 扩容,新建 3 号实例跑完删掉 |
| dataverify | `items\dataverify.ps1` | 磁盘读写数据核对(ModAVD/ModAVD#138):新建 4 号实例,在客体 `/data/unencrypted/` 里写一个由种子算出的 1 GiB 花样文件(单次 4 KiB–1 MiB,普通写和 `O_DIRECT` 混着),`sync`、清页缓存回读一次哈希;再正常关机、冷启动,回读哈希和宿主按种子算的比对,不一致就拉回来报第一个不同的偏移和错字节数;跑完删文件、删实例。`-SizeMiB` 改总量,`-Seed`、`-DirectPercent` 改花样和直写比例,`-Keep` 留着实例。客体的 `dd` 没有 `oflag=direct`,写入用 `items\dataverify-guest.go`,宿主上要有 Go 来编 |
| upgrade | `items\upgrade.ps1` | 在 `PackageRoot` 上先按包根清单装回上一版正式包、建临时实例写数据,照「怎么更新」覆盖新包后核对开机、数据、扩容、照常开机自动装桥;`-Next` 是组包目录(带 `components\modavd-ime.apk`)时还核对宿主输入法跟着包过来、装上的是正式签名那份并设为默认(`-Next` 是 SDK 时不查);跑完删临时实例,根目录停在新包上 |
| yys | `items\yys.ps1` | 阴阳师冷启动进游戏测帧(测试根的 0 号,已装桥、已登录),再用 MaaYYs(`items\maayys_run.py`)真打御魂 |
| ef-ab | `items\ef-ab.ps1` | 终末地冷启动新旧交替,等宿主空闲才测 |
Expand Down
80 changes: 80 additions & 0 deletions items/dataverify-guest.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// dataverify 项在客体(Android,linux/amd64)里跑的小程序:按写入计划把花样文件的一段段字节写进数据文件,
// 计划里标了直写的块用 O_DIRECT 写,其余用普通写。客体自带的 toybox dd 没有 oflag=direct,所以需要它。
// 不读不写别的东西:花样、计划都由宿主(items\dataverify-lib.ps1)算好推进来,写完的哈希由客体的 sha256sum 算。
// 宿主上的 items\dataverify.ps1 用 `go build`(GOOS=linux GOARCH=amd64 CGO_ENABLED=0)编它,不要在 Windows 宿主上运行。
//
// dvwrite <花样文件> <数据文件> <计划文件>
//
// 计划文件每行 `<偏移> <大小> <0|1>`(1 = 直写);文件偏移 o 处写的是花样[o mod 花样长度] 起的 <大小> 字节。
package main

import (
"bufio"
"fmt"
"os"
"syscall"
)

func fail(format string, a ...any) {
fmt.Printf("DV-WRITE-FAIL "+format+"\n", a...)
os.Exit(1)
}

func main() {
if len(os.Args) != 4 {
fmt.Fprintln(os.Stderr, "usage: dvwrite <pattern> <file> <plan>")
os.Exit(2)
}
pattern, err := os.ReadFile(os.Args[1])
if err != nil || len(pattern) == 0 {
fail("read pattern: %v", err)
}
bf, err := os.OpenFile(os.Args[2], os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
if err != nil {
fail("open buffered: %v", err)
}
df, err := os.OpenFile(os.Args[2], os.O_WRONLY|syscall.O_DIRECT, 0)
if err != nil {
fail("open O_DIRECT: %v", err)
}
// O_DIRECT 要内存对齐:匿名映射是页对齐的
buf, err := syscall.Mmap(-1, 0, 1<<20, syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_ANON|syscall.MAP_PRIVATE)
if err != nil {
fail("mmap: %v", err)
}
pf, err := os.Open(os.Args[3])
if err != nil {
fail("open plan: %v", err)
}
sc := bufio.NewScanner(pf)
chunks, direct := 0, 0
for sc.Scan() {
var off int64
var size, d int
if _, err := fmt.Sscanf(sc.Text(), "%d %d %d", &off, &size, &d); err != nil {
fail("plan line %q: %v", sc.Text(), err)
}
start := int(off % int64(len(pattern)))
if size <= 0 || size > len(buf) || start+size > len(pattern) {
fail("chunk %d@%d out of range", size, off)
}
src := pattern[start : start+size]
f := bf
if d == 1 {
copy(buf, src)
src, f = buf[:size], df
direct++
}
if n, err := f.WriteAt(src, off); err != nil || n != size {
fail("write %d@%d direct=%d: n=%d err=%v", size, off, d, n, err)
}
chunks++
}
for _, f := range []*os.File{bf, df} {
if err := f.Sync(); err != nil {
fail("fsync: %v", err)
}
f.Close()
}
fmt.Printf("DV-WRITE-OK chunks=%d direct=%d\n", chunks, direct)
}
Loading