Skip to content

Download hosted FEX code caches before launch - #2361

Merged
utkarshdalal merged 8 commits into
fex-code-map-uploadfrom
fex-code-cache-download
Nov 7, 2026
Merged

utkarshdalal merged 8 commits into
fex-code-map-uploadfrom
fex-code-cache-download

Conversation

@utkarshdalal

@utkarshdalal utkarshdalal commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Description

Second app half of the FEX code cache pipeline, stacked on #2358. A launch dependency, FexCodeCacheDependency, runs for containers on a FEX series build with the cache fixes (2611-gn-6 or newer): it asks GET /api/code-caches for the caches compiled for this game, FEX build and SoC, and streams any that are missing or changed into the prefix's AppData/Local/fex-emu/cache/, verified by size and sha256 through a temp file. FEX then loads translated code at launch instead of compiling it; it only picks a cache whose name carries the loaded image's file id, so files left from an older game build are ignored. The list is re-checked at most every six hours per FEX version and any failure logs and lets the launch continue.

The server side is utkarshdalal/gamenative-cloudflare-workers#91. Caches are served per SoC until FEX stamps them with a CPU-feature fingerprint.

Recording

Loading screen shows "Checking FEX code cache" and "Downloading FEX code cache (n/m)"; logcat tag FexCodeCache.

Type of Change

  • Bug fix
  • Performance / stability improvement
  • Compatibility improvements
  • Other (requires prior approval)

Checklist

  • If I have access to #code-changes, I have discussed this change there and it has been green-lighted. If I do not have access, I have still provided clear context in this PR. If I skip both, I accept that this change may face delays in review, may not be reviewed at all, or may be closed.
  • This change aligns with the current project scope (core functionality, stability, or performance). If not, it has been explicitly approved beforehand.
  • I have attached a recording of the change.
  • I have read and agree to the contribution guidelines in CONTRIBUTING.md.

Summary by cubic

Adds a launch dependency that downloads hosted FEX code caches before launch so FEX loads precompiled translation instead of compiling at launch. It runs only when the FEX code cache toggle is on, the container is on a FEX series build with the cache fixes (2611-gn-6+), and the pack includes the offline compiler; otherwise it skips silently.

The dependency asks GET /api/code-caches for caches matching the game, FEX version and SoC, and streams missing or changed files into the prefix's AppData/Local/fex-emu/cache/. When the device already holds the previous cache version, only a zstd patch is fetched and applied with the old file as dictionary; otherwise the full gzip is streamed. Either way the result is inflated through a temp file and renamed into place only when size and sha256 match. A 200 reply without a cache list counts as a failure rather than an empty list, since an unrouted path on the API host is answered by another worker. The list is re-checked at most every six hours per FEX version, and a failure logs and lets the launch continue.

Written for commit ae7e36e. Summary will update on new commits.

View guided diff Turn on auto-fix

A launch dependency asks /api/code-caches for the caches compiled for this game, the
container's FEX build and this SoC, and puts any that are missing or changed into the
prefix's fex-emu/cache/ (verified by size and sha256, written through a temp file). FEX
then loads translated code at launch instead of compiling it. The list is re-checked at
most every six hours per FEX version, and a failure never blocks the launch.
@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3fd4125e-d63f-47e9-9558-5de5c9d0d750

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="app/src/main/java/app/gamenative/utils/LaunchDependencies.kt">

<violation number="1" location="app/src/main/java/app/gamenative/utils/LaunchDependencies.kt:33">
P2: `FexCodeCacheDependency` is the first optional, best-effort dependency in this list, but the loop here has no isolation. An uncaught exception from its `install` (for example, an `IOException` from the SHA-256 read in `matches()` or from writing the stamp file) propagates to `PluviaMain` and shows the launch-failure dialog. Consider wrapping the `install` call for this dependency in `runCatching` or a try/catch that logs and continues, so the cache step cannot block launch.</violation>
</file>

<file name="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt">

<violation number="1" location="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt:68">
P2: An unreadable existing cache file makes `matches` throw out of `install`, which blocks the launch with an error dialog. Wrap the per-file check so a failed check counts as not matching.</violation>

<violation number="2" location="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt:73">
P1: Validate API-provided cache names before joining them to `dir`; a name containing `../` can make this download write outside the FEX cache directory.</violation>

<violation number="3" location="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt:77">
P2: Download failures are logged, but the stamp is still written, so missing caches are not retried for six hours. Write the stamp only when every missing cache downloads and verifies, so the next launch retries the failed ones.</violation>

<violation number="4" location="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt:79">
P2: Handle stamp-write failures locally; `writeText` can throw and abort launch even though cache failures are meant to be non-blocking.</violation>
</file>

<file name="app/src/main/java/app/gamenative/api/CodeCacheApi.kt">

<violation number="1" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:31">
P2: `download` cannot be cancelled. Once started, it reads the entire body even if the launch coroutine is cancelled. Make it `suspend` and check `coroutineContext.isActive` (or `ensureActive()`) in the read loop, as `downloadSigned` does.</violation>

<violation number="2" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:35">
P1: `download` uses `GameNativeApi.httpClient`, whose 5-minute `callTimeout` aborts any transfer that runs longer. Large FEX caches on slow networks will time out, the `.part` file is deleted, and the cache is never installed. Build a dedicated client with `callTimeout(0)` and a read timeout, as `SupportApi.transferClient` does.</violation>

<violation number="3" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:46">
P2: Reject chunks that would exceed `cache.size` before writing them; the current loop can fill device storage before the final size check detects an oversized response.</violation>

<violation number="4" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:54">
P2: A failed rename returns `false`, but the caller treats that as a successful download and stamps the check for six hours. Throw when `renameTo` fails so the caller logs the failure accurately.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

val tmp = File(target.parentFile, "${target.name}.part")
try {
val request = Request.Builder().url(cache.url).get().build()
GameNativeApi.httpClient.newCall(request).execute().use { response ->

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: download uses GameNativeApi.httpClient, whose 5-minute callTimeout aborts any transfer that runs longer. Large FEX caches on slow networks will time out, the .part file is deleted, and the cache is never installed. Build a dedicated client with callTimeout(0) and a read timeout, as SupportApi.transferClient does.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 35:

<comment>`download` uses `GameNativeApi.httpClient`, whose 5-minute `callTimeout` aborts any transfer that runs longer. Large FEX caches on slow networks will time out, the `.part` file is deleted, and the cache is never installed. Build a dedicated client with `callTimeout(0)` and a read timeout, as `SupportApi.transferClient` does.</comment>

<file context>
@@ -0,0 +1,60 @@
+        val tmp = File(target.parentFile, "${target.name}.part")
+        try {
+            val request = Request.Builder().url(cache.url).get().build()
+            GameNativeApi.httpClient.newCall(request).execute().use { response ->
+                if (!response.isSuccessful) throw IOException("HTTP ${response.code}")
+                val digest = MessageDigest.getInstance("SHA-256")
</file context>

missing.forEachIndexed { index, cache ->
callbacks.setLoadingMessage("Downloading FEX code cache (${index + 1}/${missing.size})")
callbacks.setLoadingProgress(if (missing.size > 1) index.toFloat() / missing.size else -1f)
runCatching { CodeCacheApi.download(cache, File(dir, cache.name)) }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Validate API-provided cache names before joining them to dir; a name containing ../ can make this download write outside the FEX cache directory.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt, line 73:

<comment>Validate API-provided cache names before joining them to `dir`; a name containing `../` can make this download write outside the FEX cache directory.</comment>

<file context>
@@ -0,0 +1,98 @@
+        missing.forEachIndexed { index, cache ->
+            callbacks.setLoadingMessage("Downloading FEX code cache (${index + 1}/${missing.size})")
+            callbacks.setLoadingProgress(if (missing.size > 1) index.toFloat() / missing.size else -1f)
+            runCatching { CodeCacheApi.download(cache, File(dir, cache.name)) }
+                .onSuccess { Timber.tag(TAG).i("Downloaded ${cache.name} (${cache.size} B)") }
+                .onFailure { Timber.tag(TAG).w(it, "Download of ${cache.name} failed") }
</file context>

EpicSdkUpdateDependency,
BionicSteamAssetsDependency,
RockstarHelperDependency,
FexCodeCacheDependency,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: FexCodeCacheDependency is the first optional, best-effort dependency in this list, but the loop here has no isolation. An uncaught exception from its install (for example, an IOException from the SHA-256 read in matches() or from writing the stamp file) propagates to PluviaMain and shows the launch-failure dialog. Consider wrapping the install call for this dependency in runCatching or a try/catch that logs and continues, so the cache step cannot block launch.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/LaunchDependencies.kt, line 33:

<comment>`FexCodeCacheDependency` is the first optional, best-effort dependency in this list, but the loop here has no isolation. An uncaught exception from its `install` (for example, an `IOException` from the SHA-256 read in `matches()` or from writing the stamp file) propagates to `PluviaMain` and shows the launch-failure dialog. Consider wrapping the `install` call for this dependency in `runCatching` or a try/catch that logs and continues, so the cache step cannot block launch.</comment>

<file context>
@@ -29,6 +30,7 @@ class LaunchDependencies {
             EpicSdkUpdateDependency,
             BionicSteamAssetsDependency,
             RockstarHelperDependency,
+            FexCodeCacheDependency,
         )
 
</file context>

Timber.tag(TAG).w("Cannot create ${dir.path}")
return@withContext
}
val missing = listed.filter { cache -> !matches(File(dir, cache.name), cache) }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: An unreadable existing cache file makes matches throw out of install, which blocks the launch with an error dialog. Wrap the per-file check so a failed check counts as not matching.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt, line 68:

<comment>An unreadable existing cache file makes `matches` throw out of `install`, which blocks the launch with an error dialog. Wrap the per-file check so a failed check counts as not matching.</comment>

<file context>
@@ -0,0 +1,98 @@
+            Timber.tag(TAG).w("Cannot create ${dir.path}")
+            return@withContext
+        }
+        val missing = listed.filter { cache -> !matches(File(dir, cache.name), cache) }
+        Timber.tag(TAG).i("$fexVersion on $soc: ${listed.size} hosted caches, ${missing.size} to download")
+        missing.forEachIndexed { index, cache ->
</file context>
Suggested change
val missing = listed.filter { cache -> !matches(File(dir, cache.name), cache) }
val missing = listed.filter { cache -> !runCatching { matches(File(dir, cache.name), cache) }.getOrDefault(false) }

.onSuccess { Timber.tag(TAG).i("Downloaded ${cache.name} (${cache.size} B)") }
.onFailure { Timber.tag(TAG).w(it, "Download of ${cache.name} failed") }
}
stampFile(container).apply {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Download failures are logged, but the stamp is still written, so missing caches are not retried for six hours. Write the stamp only when every missing cache downloads and verifies, so the next launch retries the failed ones.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt, line 77:

<comment>Download failures are logged, but the stamp is still written, so missing caches are not retried for six hours. Write the stamp only when every missing cache downloads and verifies, so the next launch retries the failed ones.</comment>

<file context>
@@ -0,0 +1,98 @@
+                .onSuccess { Timber.tag(TAG).i("Downloaded ${cache.name} (${cache.size} B)") }
+                .onFailure { Timber.tag(TAG).w(it, "Download of ${cache.name} failed") }
+        }
+        stampFile(container).apply {
+            parentFile?.mkdirs()
+            writeText(fexVersion)
</file context>

}

/** Streams one cache into [target] through a temp file; throws (and leaves nothing) on a size or hash mismatch. */
fun download(cache: HostedCache, target: File): Boolean {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: download cannot be cancelled. Once started, it reads the entire body even if the launch coroutine is cancelled. Make it suspend and check coroutineContext.isActive (or ensureActive()) in the read loop, as downloadSigned does.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 31:

<comment>`download` cannot be cancelled. Once started, it reads the entire body even if the launch coroutine is cancelled. Make it `suspend` and check `coroutineContext.isActive` (or `ensureActive()`) in the read loop, as `downloadSigned` does.</comment>

<file context>
@@ -0,0 +1,60 @@
+    }
+
+    /** Streams one cache into [target] through a temp file; throws (and leaves nothing) on a size or hash mismatch. */
+    fun download(cache: HostedCache, target: File): Boolean {
+        val tmp = File(target.parentFile, "${target.name}.part")
+        try {
</file context>

val n = input.read(buffer)
if (n < 0) break
digest.update(buffer, 0, n)
output.write(buffer, 0, n)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Reject chunks that would exceed cache.size before writing them; the current loop can fill device storage before the final size check detects an oversized response.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 46:

<comment>Reject chunks that would exceed `cache.size` before writing them; the current loop can fill device storage before the final size check detects an oversized response.</comment>

<file context>
@@ -0,0 +1,60 @@
+                            val n = input.read(buffer)
+                            if (n < 0) break
+                            digest.update(buffer, 0, n)
+                            output.write(buffer, 0, n)
+                            total += n
+                        }
</file context>
Suggested change
output.write(buffer, 0, n)
if (total + n > cache.size) throw IOException("cache ${cache.name} exceeds declared size")
output.write(buffer, 0, n)

val sha256 = digest.digest().joinToString("") { "%02x".format(it) }
if (total != cache.size || sha256 != cache.sha256) throw IOException("cache ${cache.name} did not verify")
}
return tmp.renameTo(target)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A failed rename returns false, but the caller treats that as a successful download and stamps the check for six hours. Throw when renameTo fails so the caller logs the failure accurately.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 54:

<comment>A failed rename returns `false`, but the caller treats that as a successful download and stamps the check for six hours. Throw when `renameTo` fails so the caller logs the failure accurately.</comment>

<file context>
@@ -0,0 +1,60 @@
+                val sha256 = digest.digest().joinToString("") { "%02x".format(it) }
+                if (total != cache.size || sha256 != cache.sha256) throw IOException("cache ${cache.name} did not verify")
+            }
+            return tmp.renameTo(target)
+        } catch (e: Exception) {
+            tmp.delete()
</file context>
Suggested change
return tmp.renameTo(target)
if (!tmp.renameTo(target)) throw IOException("could not move cache ${cache.name}")
return true

}
stampFile(container).apply {
parentFile?.mkdirs()
writeText(fexVersion)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Handle stamp-write failures locally; writeText can throw and abort launch even though cache failures are meant to be non-blocking.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt, line 79:

<comment>Handle stamp-write failures locally; `writeText` can throw and abort launch even though cache failures are meant to be non-blocking.</comment>

<file context>
@@ -0,0 +1,98 @@
+        }
+        stampFile(container).apply {
+            parentFile?.mkdirs()
+            writeText(fexVersion)
+        }
+    }
</file context>
Suggested change
writeText(fexVersion)
runCatching { writeText(fexVersion) }
.onFailure { Timber.tag(TAG).w(it, "Could not write cache check stamp") }

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 existing issue remains and 1 new issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="app/src/main/java/app/gamenative/api/CodeCacheApi.kt">

<violation number="1" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:43">
P2: `download` always wraps the body in `GZIPInputStream`, but OkHttp already inflates responses sent with `Content-Encoding: gzip`. In that case the constructor throws `Not in GZIP format` and every cache download fails. Sniff the gzip magic bytes first, as `PicsAppInfo.decompressIfGzip` does, and import `java.io.BufferedInputStream`.</violation>
</file>

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

View guided diff | Turn on auto-fix | Re-trigger cubic

if (!response.isSuccessful) throw IOException("HTTP ${response.code}")
val digest = MessageDigest.getInstance("SHA-256")
var total = 0L
GZIPInputStream(response.body!!.byteStream()).use { input ->

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: download always wraps the body in GZIPInputStream, but OkHttp already inflates responses sent with Content-Encoding: gzip. In that case the constructor throws Not in GZIP format and every cache download fails. Sniff the gzip magic bytes first, as PicsAppInfo.decompressIfGzip does, and import java.io.BufferedInputStream.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 43:

<comment>`download` always wraps the body in `GZIPInputStream`, but OkHttp already inflates responses sent with `Content-Encoding: gzip`. In that case the constructor throws `Not in GZIP format` and every cache download fails. Sniff the gzip magic bytes first, as `PicsAppInfo.decompressIfGzip` does, and import `java.io.BufferedInputStream`.</comment>

<file context>
@@ -36,7 +40,7 @@ object CodeCacheApi {
                 val digest = MessageDigest.getInstance("SHA-256")
                 var total = 0L
-                response.body!!.byteStream().use { input ->
+                GZIPInputStream(response.body!!.byteStream()).use { input ->
                     FileOutputStream(tmp).use { output ->
                         val buffer = ByteArray(256 * 1024)
</file context>

Utkarsh Dalal added 2 commits November 7, 2026 11:09
…sion

A cache for the same FEX build grows as coverage grows. When the hosted entry names the
sha256 the device already has as its patch base, only the zstd patch is fetched and applied
with the old file as dictionary; a different or missing local version still gets the full
gzip. The result is always inflated through a temp file and verified before it replaces the
old cache.
@utkarshdalal

Copy link
Copy Markdown
Owner Author

Downloads are now incremental too. The hosted list carries patchFrom, the sha256 of the previous version a zstd patch applies to; a device whose local file has that sha256 asks the file route with ?have= and receives the patch, applied through ZstdInputStream with the old file as dictionary (window log 31, so it works for the 236 MB Brotato cache). Any other local state gets the full gzip as before, and the result is always inflated through a temp file and verified before it replaces the old cache. Server side is the latest workers PR 91. Compiles against the bundled zstd-jni 1.5.7; not yet exercised on a device.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 existing issue remains and 4 new issues found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt">

<violation number="1" location="app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt:77">
P3: The "patched" log flag is wrong on first downloads. When there is no local file and `cache.patchFrom` is null, `have == cache.patchFrom` is `null == null`, so a plain full download logs as patched. Add a null check: `have != null && have == cache.patchFrom`.</violation>
</file>

<file name="app/src/main/java/app/gamenative/api/CodeCacheApi.kt">

<violation number="1" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:57">
P3: The new patch branch, the gzip fallback, and the sha256/size rejection have no unit tests. Add MockWebServer tests for a patch response, a full gzip response, and a bad sha256 so the branch selection and verification stay covered.</violation>

<violation number="2" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:60">
P2: If the patch attempt fails, `download` throws and never tries the full gzip, so the device keeps the old cache. The caller still writes the 6-hour stamp, so the same patch fails on every recheck and the cache never updates. Catch the failure from the patch path and retry with `cache.url` without `?have=`.</violation>

<violation number="3" location="app/src/main/java/app/gamenative/api/CodeCacheApi.kt:71">
P1: `target.readBytes()` loads the entire old cache into the managed heap before patching; the documented cache size reaches 236 MB. This can exhaust the app heap during launch, so use a file-backed/native dictionary path or skip patching and fetch the full gzip for oversized bases.</violation>
</file>

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

View guided diff | Turn on auto-fix | Re-trigger cubic

// zstd --patch-from: the previous file is the dictionary, window up to 2 GB.
ZstdInputStream(body).apply {
setLongMax(31)
setDict(target.readBytes())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: target.readBytes() loads the entire old cache into the managed heap before patching; the documented cache size reaches 236 MB. This can exhaust the app heap during launch, so use a file-backed/native dictionary path or skip patching and fetch the full gzip for oversized bases.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 71:

<comment>`target.readBytes()` loads the entire old cache into the managed heap before patching; the documented cache size reaches 236 MB. This can exhaust the app heap during launch, so use a file-backed/native dictionary path or skip patching and fetch the full gzip for oversized bases.</comment>

<file context>
@@ -22,29 +32,54 @@ object CodeCacheApi {
+                    // zstd --patch-from: the previous file is the dictionary, window up to 2 GB.
+                    ZstdInputStream(body).apply {
+                        setLongMax(31)
+                        setDict(target.readBytes())
+                    }
+                } else {
</file context>

fun download(cache: HostedCache, target: File, haveSha256: String? = null): Boolean {
val tmp = File(target.parentFile, "${target.name}.part")
try {
val usePatch = haveSha256 != null && haveSha256 == cache.patchFrom && target.isFile

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: If the patch attempt fails, download throws and never tries the full gzip, so the device keeps the old cache. The caller still writes the 6-hour stamp, so the same patch fails on every recheck and the cache never updates. Catch the failure from the patch path and retry with cache.url without ?have=.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 60:

<comment>If the patch attempt fails, `download` throws and never tries the full gzip, so the device keeps the old cache. The caller still writes the 6-hour stamp, so the same patch fails on every recheck and the cache never updates. Catch the failure from the patch path and retry with `cache.url` without `?have=`.</comment>

<file context>
@@ -22,29 +32,54 @@ object CodeCacheApi {
         val tmp = File(target.parentFile, "${target.name}.part")
         try {
-            val request = Request.Builder().url(cache.url).get().build()
+            val usePatch = haveSha256 != null && haveSha256 == cache.patchFrom && target.isFile
+            val url = if (usePatch) "${cache.url}?have=$haveSha256" else cache.url
+            val request = Request.Builder().url(url).get().build()
</file context>

callbacks.setLoadingProgress(if (missing.size > 1) index.toFloat() / missing.size else -1f)
val have = local[cache]
runCatching { CodeCacheApi.download(cache, File(dir, cache.name), have) }
.onSuccess { Timber.tag(TAG).i("Updated ${cache.name} (${cache.size} B${if (have == cache.patchFrom) ", patched" else ""})") }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The "patched" log flag is wrong on first downloads. When there is no local file and cache.patchFrom is null, have == cache.patchFrom is null == null, so a plain full download logs as patched. Add a null check: have != null && have == cache.patchFrom.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/utils/launchdependencies/FexCodeCacheDependency.kt, line 77:

<comment>The "patched" log flag is wrong on first downloads. When there is no local file and `cache.patchFrom` is null, `have == cache.patchFrom` is `null == null`, so a plain full download logs as patched. Add a null check: `have != null && have == cache.patchFrom`.</comment>

<file context>
@@ -65,13 +65,16 @@ object FexCodeCacheDependency : LaunchDependency {
-                .onSuccess { Timber.tag(TAG).i("Downloaded ${cache.name} (${cache.size} B)") }
+            val have = local[cache]
+            runCatching { CodeCacheApi.download(cache, File(dir, cache.name), have) }
+                .onSuccess { Timber.tag(TAG).i("Updated ${cache.name} (${cache.size} B${if (have == cache.patchFrom) ", patched" else ""})") }
                 .onFailure { Timber.tag(TAG).w(it, "Download of ${cache.name} failed") }
         }
</file context>
Suggested change
.onSuccess { Timber.tag(TAG).i("Updated ${cache.name} (${cache.size} B${if (have == cache.patchFrom) ", patched" else ""})") }
.onSuccess { Timber.tag(TAG).i("Updated ${cache.name} (${cache.size} B${if (have != null && have == cache.patchFrom) ", patched" else ""})") }

@@ -0,0 +1,99 @@
package app.gamenative.api

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The new patch branch, the gzip fallback, and the sha256/size rejection have no unit tests. Add MockWebServer tests for a patch response, a full gzip response, and a bad sha256 so the branch selection and verification stay covered.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At app/src/main/java/app/gamenative/api/CodeCacheApi.kt, line 57:

<comment>The new patch branch, the gzip fallback, and the sha256/size rejection have no unit tests. Add MockWebServer tests for a patch response, a full gzip response, and a bad sha256 so the branch selection and verification stay covered.</comment>

<file context>
@@ -22,29 +32,54 @@ object CodeCacheApi {
+     * result is inflated through a temp file, hashed, and renamed only when size and sha256 match.
      */
-    fun download(cache: HostedCache, target: File): Boolean {
+    fun download(cache: HostedCache, target: File, haveSha256: String? = null): Boolean {
         val tmp = File(target.parentFile, "${target.name}.part")
         try {
</file context>

@utkarshdalal

Copy link
Copy Markdown
Owner Author

Serve side is live now (workers PR 91 merged, migration applied, deployed) with 17 caches hosted for the Retroid Pocket 6's SoC on 2611-gn-6: Brotato (4 files, 30.5 MB gzipped, 236 MB on disk after inflating) and Borderlands 2 (13 files, 16.3 MB gzipped, 105 MB on disk). The dependency now honours the "FEX code caches" toggle from #2358 and skips packs without the offline compiler, and it fails rather than reports "no caches" when the API host answers with something other than a cache list (an unrouted path there is answered 200 by the old update-check worker). The disk cost is the raw cache in the prefix, which is what the toggle is for; a device run of the download is next.

@utkarshdalal

Copy link
Copy Markdown
Owner Author

Device run on the Retroid Pocket 6 with this build (ae7e36e): two of Brotato's four hosted caches were removed from the prefix, and on the next launch the dependency listed "4 hosted caches, 2 to download" and restored both before the game started, byte-identical to the hosted files (sha256 matches). The game ran a wave at 60 fps and quit cleanly; after exit the uploader found nothing new to send ("249 entries, none new"), which is the ledger from #2358 doing its job. FEX stays silent by default so there is no "Loaded cache" line, but the session map shrinking from about 17k blocks to 249 shows the caches were in use. Downloaded files land mode 0600 while the compiler-made ones were 0666; same uid, no effect.

@utkarshdalal
utkarshdalal merged commit ae7e36e into fex-code-map-upload Nov 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant