Repository navigation
Fingerprint translation-stack crashes in game_exited - #2374
utkarshdalal wants to merge 2 commits into
Conversation
|
Caution Review failedThe pull request is closed. Walkthrough
Merge Risk:
|
There was a problem hiding this comment.
7 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="app/src/main/java/app/gamenative/utils/CrashFingerprint.kt">
<violation number="1" location="app/src/main/java/app/gamenative/utils/CrashFingerprint.kt:161">
P2: Matched patterns produce a fingerprint even on a clean exit, because the exit reason is only checked when nothing matches. Consider gating the matched-hit path on an abnormal exit, or confirm that benign `err:` lines on normal exits are intended to be reported.</violation>
<violation number="2" location="app/src/main/java/app/gamenative/utils/CrashFingerprint.kt:204">
P3: The size cap is not guaranteed. After the sample and versions fallbacks, `fp` can still exceed MAX_SERIALIZED_CHARS when escaped frames or module/code fields are large. Enforce the cap on the final JSON, for example by truncating frames or failing closed.</violation>
<violation number="3" location="app/src/main/java/app/gamenative/utils/CrashFingerprint.kt:220">
P2: This condition classifies every matching generic `Unhandled instruction` line as FEX, so it overrides the Wine exception detector and misattributes crashes. Require an explicit FEX marker before returning this hit.</violation>
<violation number="4" location="app/src/main/java/app/gamenative/utils/CrashFingerprint.kt:248">
P2: The WINE_STUB branch hard-codes the component to "wine", so stubs from d3d11/dxgi, vkd3d, or FEX DLLs are misattributed. Use `componentForModule(dll)`, which already maps these modules, so the clustering matches the rest of the extractor.</violation>
<violation number="5" location="app/src/main/java/app/gamenative/utils/CrashFingerprint.kt:275">
P2: This branch discards the vkd3d error message and uses only the emitter name as `code`, collapsing distinct vkd3d failures into one fingerprint. Use group 2, with group 1 only as an empty-message fallback.</violation>
</file>
<file name="app/src/main/java/app/gamenative/utils/SessionReport.kt">
<violation number="1" location="app/src/main/java/app/gamenative/utils/SessionReport.kt:170">
P3: This new function has no test. Add a case that covers the version-map assembly, including the `dxwrapper_version` selection, and the empty-tail `emptyMap()` path.</violation>
<violation number="2" location="app/src/main/java/app/gamenative/utils/SessionReport.kt:180">
P2: `dxwrapper_version` always reads the DXVK `version` key. For `vkd3d` containers the vkd3d-proton version (`vkd3dVersion`) is never reported, and for `wined3d` a DXVK version is reported even though DXVK is not in use. Select the key by wrapper, and drop the version when the wrapper is neither `dxvk` nor `vkd3d`.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| } | ||
|
|
||
| VKD3D_ERR.find(line)?.let { m -> | ||
| return Hit(40, i, "vkd3d", "other", m.groupValues[1].lowercase()) |
There was a problem hiding this comment.
P2: This branch discards the vkd3d error message and uses only the emitter name as code, collapsing distinct vkd3d failures into one fingerprint. Use group 2, with group 1 only as an empty-message fallback.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/CrashFingerprint.kt, line 275:
<comment>This branch discards the vkd3d error message and uses only the emitter name as `code`, collapsing distinct vkd3d failures into one fingerprint. Use group 2, with group 1 only as an empty-message fallback.</comment>
<file context>
@@ -0,0 +1,401 @@
+ }
+
+ VKD3D_ERR.find(line)?.let { m ->
+ return Hit(40, i, "vkd3d", "other", m.groupValues[1].lowercase())
+ }
+ DXVK_ERROR.find(line)?.let { m -> return Hit(40, i, "dxvk", "other", normalizeCode(m.groupValues[1].ifEmpty { "DxvkError" })) }
</file context>
| } | ||
| if (!lower.contains("box64")) { | ||
| FEX_INSTRUCTION.find(line)?.let { m -> | ||
| if (lower.contains("fex") || lower.contains("instruction")) { |
There was a problem hiding this comment.
P2: This condition classifies every matching generic Unhandled instruction line as FEX, so it overrides the Wine exception detector and misattributes crashes. Require an explicit FEX marker before returning this hit.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/CrashFingerprint.kt, line 220:
<comment>This condition classifies every matching generic `Unhandled instruction` line as FEX, so it overrides the Wine exception detector and misattributes crashes. Require an explicit FEX marker before returning this hit.</comment>
<file context>
@@ -0,0 +1,401 @@
+ }
+ if (!lower.contains("box64")) {
+ FEX_INSTRUCTION.find(line)?.let { m ->
+ if (lower.contains("fex") || lower.contains("instruction")) {
+ val rest = m.groupValues[1]
+ val code = INSTRUCTION_BYTES.find(rest)?.let { bytes(it.value) }?.ifEmpty { null } ?: normalizeCode(rest)
</file context>
| "graphics_driver_version" to KeyValueSet(container.getGraphicsDriverConfig()).get("version") | ||
| .ifEmpty { container.getGraphicsDriverVersion().orEmpty() }, | ||
| "dxwrapper" to container.getDXWrapper().orEmpty(), | ||
| "dxwrapper_version" to DXVKHelper.parseConfig(container.getDXWrapperConfig()).get("version"), |
There was a problem hiding this comment.
P2: dxwrapper_version always reads the DXVK version key. For vkd3d containers the vkd3d-proton version (vkd3dVersion) is never reported, and for wined3d a DXVK version is reported even though DXVK is not in use. Select the key by wrapper, and drop the version when the wrapper is neither dxvk nor vkd3d.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/SessionReport.kt, line 180:
<comment>`dxwrapper_version` always reads the DXVK `version` key. For `vkd3d` containers the vkd3d-proton version (`vkd3dVersion`) is never reported, and for `wined3d` a DXVK version is reported even though DXVK is not in use. Select the key by wrapper, and drop the version when the wrapper is neither `dxvk` nor `vkd3d`.</comment>
<file context>
@@ -165,6 +167,26 @@ object SessionReport {
+ "graphics_driver_version" to KeyValueSet(container.getGraphicsDriverConfig()).get("version")
+ .ifEmpty { container.getGraphicsDriverVersion().orEmpty() },
+ "dxwrapper" to container.getDXWrapper().orEmpty(),
+ "dxwrapper_version" to DXVKHelper.parseConfig(container.getDXWrapperConfig()).get("version"),
+ ).filterValues { it.isNotEmpty() }
+ CrashFingerprint.extract(GuestOutputTail.snapshot(), reason, versions)
</file context>
| for (i in lines.indices) matchLine(lines, i)?.let(hits::add) | ||
|
|
||
| val best = hits.maxWithOrNull(compareBy<Hit>({ it.priority }, { it.index })) | ||
| if (best == null) { |
There was a problem hiding this comment.
P2: Matched patterns produce a fingerprint even on a clean exit, because the exit reason is only checked when nothing matches. Consider gating the matched-hit path on an abnormal exit, or confirm that benign err: lines on normal exits are intended to be reported.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/CrashFingerprint.kt, line 161:
<comment>Matched patterns produce a fingerprint even on a clean exit, because the exit reason is only checked when nothing matches. Consider gating the matched-hit path on an abnormal exit, or confirm that benign `err:` lines on normal exits are intended to be reported.</comment>
<file context>
@@ -0,0 +1,401 @@
+ for (i in lines.indices) matchLine(lines, i)?.let(hits::add)
+
+ val best = hits.maxWithOrNull(compareBy<Hit>({ it.priority }, { it.index }))
+ if (best == null) {
+ if (exitReason == null || exitReason !in ABNORMAL_EXITS) return null
+ return build("none", "other", exitReason, "", emptyList(), emptyList(), componentVersions)
</file context>
|
|
||
| WINE_STUB.find(line)?.let { m -> | ||
| val (dll, fn) = splitDllFunction(m.groupValues[1]) | ||
| return Hit(75, i, "wine", "unimplemented", "$dll.$fn", dll) |
There was a problem hiding this comment.
P2: The WINE_STUB branch hard-codes the component to "wine", so stubs from d3d11/dxgi, vkd3d, or FEX DLLs are misattributed. Use componentForModule(dll), which already maps these modules, so the clustering matches the rest of the extractor.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/CrashFingerprint.kt, line 248:
<comment>The WINE_STUB branch hard-codes the component to "wine", so stubs from d3d11/dxgi, vkd3d, or FEX DLLs are misattributed. Use `componentForModule(dll)`, which already maps these modules, so the clustering matches the rest of the extractor.</comment>
<file context>
@@ -0,0 +1,401 @@
+
+ WINE_STUB.find(line)?.let { m ->
+ val (dll, fn) = splitDllFunction(m.groupValues[1])
+ return Hit(75, i, "wine", "unimplemented", "$dll.$fn", dll)
+ }
+
</file context>
| return Hit(75, i, "wine", "unimplemented", "$dll.$fn", dll) | |
| return Hit(75, i, componentForModule(dll), "unimplemented", "$dll.$fn", dll) |
|
|
||
| private const val MAX_RUN_FILES = 30 | ||
|
|
||
| fun crashFingerprintProperties(container: Container, reason: String): Map<String, Any> = try { |
There was a problem hiding this comment.
P3: This new function has no test. Add a case that covers the version-map assembly, including the dxwrapper_version selection, and the empty-tail emptyMap() path.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/SessionReport.kt, line 170:
<comment>This new function has no test. Add a case that covers the version-map assembly, including the `dxwrapper_version` selection, and the empty-tail `emptyMap()` path.</comment>
<file context>
@@ -165,6 +167,26 @@ object SessionReport {
private const val MAX_RUN_FILES = 30
+ fun crashFingerprintProperties(container: Container, reason: String): Map<String, Any> = try {
+ val versions = linkedMapOf(
+ "wine" to container.getWineVersion().orEmpty(),
</file context>
| while (fp.toJson().length > MAX_SERIALIZED_CHARS && fp.sample.isNotEmpty()) { | ||
| fp = fp.copy(sample = fp.sample.dropLast(1)) | ||
| } | ||
| if (fp.toJson().length > MAX_SERIALIZED_CHARS) { |
There was a problem hiding this comment.
P3: The size cap is not guaranteed. After the sample and versions fallbacks, fp can still exceed MAX_SERIALIZED_CHARS when escaped frames or module/code fields are large. Enforce the cap on the final JSON, for example by truncating frames or failing closed.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/CrashFingerprint.kt, line 204:
<comment>The size cap is not guaranteed. After the sample and versions fallbacks, `fp` can still exceed MAX_SERIALIZED_CHARS when escaped frames or module/code fields are large. Enforce the cap on the final JSON, for example by truncating frames or failing closed.</comment>
<file context>
@@ -0,0 +1,401 @@
+ while (fp.toJson().length > MAX_SERIALIZED_CHARS && fp.sample.isNotEmpty()) {
+ fp = fp.copy(sample = fp.sample.dropLast(1))
+ }
+ if (fp.toJson().length > MAX_SERIALIZED_CHARS) {
+ fp = fp.copy(versions = fp.versions.mapValues { it.value.take(48) }.entries.take(8).associate { it.key.take(32) to it.value })
+ }
</file context>
There was a problem hiding this comment.
3 existing issues remain and 3 new issues found across 4 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt">
<violation number="1" location="app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt:4456">
P2: Resetting one global tail does not isolate sessions because previous ProcessHelper readers can still call the newly registered callback after reset. Associate the tail with a session and ignore late callbacks from prior launches.</violation>
<violation number="2" location="app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt:5487">
P2: The fingerprint is extracted before ProcessHelper’s output-reader threads are guaranteed to drain, so a crash’s final diagnostic lines can be missed and `game_exited` can omit or misclassify the fingerprint. Add a termination/output-drain barrier before snapshotting the tail.</violation>
<violation number="3" location="app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt:5488">
P2: `runBlocking` makes the IO timeout a synchronous wait on the exit caller, blocking the UI thread for up to 200 ms on quick-menu, force-close, or activity teardown. Make exit’s fingerprint/telemetry path suspend or dispatch the exit work without blocking the main thread.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 3 unresolved issues already reported by Cubic.
View guided diff | Turn on auto-fix | Re-trigger cubic
| if (debugRun) DebugReportUtils.startLogcatCapture(context, appId) | ||
| } | ||
|
|
||
| GuestOutputTail.reset() |
There was a problem hiding this comment.
P2: Resetting one global tail does not isolate sessions because previous ProcessHelper readers can still call the newly registered callback after reset. Associate the tail with a session and ignore late callbacks from prior launches.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt, line 4456:
<comment>Resetting one global tail does not isolate sessions because previous ProcessHelper readers can still call the newly registered callback after reset. Associate the tail with a session and ignore late callbacks from prior launches.</comment>
<file context>
@@ -4451,7 +4453,9 @@ private fun setupXEnvironment(
if (debugRun) DebugReportUtils.startLogcatCapture(context, appId)
}
+ GuestOutputTail.reset()
ProcessHelper.addDebugCallback { line ->
+ GuestOutputTail.append(line)
</file context>
| }.getOrElse { emptyMap() } | ||
| }.getOrElse { emptyMap() } + runCatching { | ||
| val fingerprint = CoroutineScope(Dispatchers.IO).async { SessionReport.crashFingerprintProperties(container, reason) } | ||
| runBlocking { withTimeoutOrNull(200) { fingerprint.await() } } |
There was a problem hiding this comment.
P2: runBlocking makes the IO timeout a synchronous wait on the exit caller, blocking the UI thread for up to 200 ms on quick-menu, force-close, or activity teardown. Make exit’s fingerprint/telemetry path suspend or dispatch the exit work without blocking the main thread.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt, line 5488:
<comment>`runBlocking` makes the IO timeout a synchronous wait on the exit caller, blocking the UI thread for up to 200 ms on quick-menu, force-close, or activity teardown. Make exit’s fingerprint/telemetry path suspend or dispatch the exit work without blocking the main thread.</comment>
<file context>
@@ -5479,7 +5483,10 @@ private fun exit(
- }.getOrElse { emptyMap() }
+ }.getOrElse { emptyMap() } + runCatching {
+ val fingerprint = CoroutineScope(Dispatchers.IO).async { SessionReport.crashFingerprintProperties(container, reason) }
+ runBlocking { withTimeoutOrNull(200) { fingerprint.await() } }
+ }.getOrNull().orEmpty()
PostHog.capture(
</file context>
| GameCompatibilityService.badgeProperties(ContainerUtils.resolveGameName(appId)) | ||
| }.getOrElse { emptyMap() } | ||
| }.getOrElse { emptyMap() } + runCatching { | ||
| val fingerprint = CoroutineScope(Dispatchers.IO).async { SessionReport.crashFingerprintProperties(container, reason) } |
There was a problem hiding this comment.
P2: The fingerprint is extracted before ProcessHelper’s output-reader threads are guaranteed to drain, so a crash’s final diagnostic lines can be missed and game_exited can omit or misclassify the fingerprint. Add a termination/output-drain barrier before snapshotting the tail.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt, line 5487:
<comment>The fingerprint is extracted before ProcessHelper’s output-reader threads are guaranteed to drain, so a crash’s final diagnostic lines can be missed and `game_exited` can omit or misclassify the fingerprint. Add a termination/output-drain barrier before snapshotting the tail.</comment>
<file context>
@@ -5479,7 +5483,10 @@ private fun exit(
GameCompatibilityService.badgeProperties(ContainerUtils.resolveGameName(appId))
- }.getOrElse { emptyMap() }
+ }.getOrElse { emptyMap() } + runCatching {
+ val fingerprint = CoroutineScope(Dispatchers.IO).async { SessionReport.crashFingerprintProperties(container, reason) }
+ runBlocking { withTimeoutOrNull(200) { fingerprint.await() } }
+ }.getOrNull().orEmpty()
</file context>
|
Folded into #2363 |
Description
Adds a compact crash fingerprint to
game_exitedso the server can cluster crashes per game and component version.Capture. In a normal launch nothing from the guest is kept:
WINEDEBUG=-all,DXVK_LOG_LEVEL=none,MESA_DEBUG=silent, and the always-registered debug callback insetupXEnvironmentreads guest stdout/stderr line by line but only writes them to a file on debug runs or with Wine/Box64 logging enabled. That callback now also appends each line toGuestOutputTail, an in-memory ring capped at 256 KB (lines capped at 4 KB). There is no new thread, file or log during play, just one bounded append in a callback that already runs for every line.Fingerprint.
CrashFingerprint.extract(tail, exitReason, componentVersions)is pure. It matches:wine: Unhandled ..., winedbgUnhandled exception: ... in NN-bit code,Exception code, the top 3Backtraceframes asmodule!symbol+0xofformodule+0xoff,Call from ... to unimplemented function,err:module:import_dll/ othererr:modulefailures, and wineserver lossUnimplemented Opcode,Unsupported Syscall, dynarec errorsAssertion ... failed,VK_ERROR_DEVICE_LOST/ GPU hang,MESA: errorerr:andDxvkErrorerr:d3d12_*/err:vkd3d*The highest-priority match wins. Instruction faults beat asserts, which beat device loss, then exceptions, unimplemented functions, load failures, syscalls and plain
err:lines. A crash in a module that is not a known wine, DXVK, vkd3d, FEX or Mesa module is attributed togame.Shape:
{component, kind, code, module, frames[≤3], sample[≤5 lines ≤200 chars], hash, versions}. Hex runs of 6 or more digits become0x…, and wine/box64 thread and pid prefixes are stripped.hashis the first 16 hex of sha256(component|kind|code|module|frames). The serialized size is capped at 2 KB. It returns null when nothing matched and the exit was notguest_terminated. Aguest_terminatedexit with no match givescomponent=none, kind=other.Wiring.
game_exitedandrecordRunare sent from the same place and thread as on master. Inexit(), the props getcrash_fingerprint(map) andcrash_fingerprint_hash, extracted from the in-memory tail. Extraction runs onDispatchers.IOandexit()waits for it for at most 200 ms; on timeout or any failure it adds nothing (catch-all, logged with Timber). For aguest_terminatedexit the tail is complete by then; a user-initiated exit has nothing to find.versionscarries wine, emulator, fexcore, box64, graphics driver + version, and dxwrapper + version from the container. The fingerprint is also stored in.gamenative/runs/*.json, so the next launch can read it. It is pipeline diagnostics and follows the same gating asgame_exited, nothing more.Not built locally; CI compiles and runs
CrashFingerprintTest.Recording
Not applicable (telemetry only).
Type of Change
Checklist
#code-changes, I have discussed this change there and it has been green-lighted. If I do not have access, I have still provided clear context in this PR. If I skip both, I accept that this change may face delays in review, may not be reviewed at all, or may be closed.CONTRIBUTING.md.Summary by cubic
Adds a compact crash fingerprint to
game_exitedso crashes can be clustered per game and component version.exit()with a 200 ms timeout, sent withgame_exitedfrom the same place as before, and stored in.gamenative/runs/*.jsonfor the next launch.guest_terminatedexit without a match yieldscomponent=none, kind=other.Written for commit 599dd6f. Summary will update on new commits.
Summary by CodeRabbit