diff --git a/CLAUDE.md b/CLAUDE.md index b0ffafe..efab20d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,10 +36,12 @@ each with a package doc comment stating its contract: - `model` — domain types and the identity rules. Carries `id` (the store product id) and deliberately not `productId`, which is a different value no endpoint accepts. -- `config` — user settings by precedence: defaults → `config.toml` → env → flags. There is - no browser session default, because a browser session cannot work here. -- `session` — builds the Cookie header from a pasted curl file or a `cookies.txt`, and - asserts the `LS` cookie is present before any request. +- `config` — user settings by precedence: defaults → `config.toml` → env → flags. +- `session` — builds the Cookie header from a Firefox-family session store, a pasted curl + file, or a `cookies.txt`, and asserts the `LS` cookie is present before any request. The + source is identified by reading it, not by configuration. `mozlz4.go` decodes Gecko's + compressed session store; the jar it holds spans every host the browsing session touched, + so it is filtered to `unity.com` before anything leaves the package. - `retry` — backoff policy. `retry.Permanent` lets a caller stop on a body-based verdict that the status code alone would have retried. - `unitypackage` — reads the store descriptor from a package's gzip FEXTRA field. @@ -57,7 +59,11 @@ each with a package doc comment stating its contract: - **`LS` is the credential.** Not the NextAuth session token, which neither endpoint consults. Its absence is reported before any request, because the store answers a - missing `LS` with an opaque 500. + missing `LS` with an opaque 500. It is absent from `cookies.sqlite` but present in a + Gecko session store, which is what makes the browser source possible. +- **No cookie value is ever logged**, and a session store is filtered to the `unity.com` + family inside `internal/session`. That file carries credentials for every host the + browsing session touched. - **No store client follows a redirect.** An unauthenticated download 302s to Unity's OAuth page. `selfupdate` is the deliberate exception: it talks to GitHub, whose asset API 302s to a signed CDN URL by design. diff --git a/README.md b/README.md index 6f199f7..b975942 100644 --- a/README.md +++ b/README.md @@ -58,12 +58,33 @@ default; override with `library_path`, `UNITY_SYNC_LIBRARY`, or `--library`. ## Session The store gates everything behind your signed-in session, and the cookie it actually -checks — `LS` — is a session cookie that lives only in your browser's memory. No browser -cookie database has it, so unity-sync cannot read your session automatically the way a -tool for some other store might. You paste one instead: +checks is `LS`. It is a session cookie, so `cookies.sqlite` never holds it — but a +Firefox-family session store does, because Gecko records the cookies of every host the +browsing session touched. -In DevTools → Network, right-click any `assetstore.unity.com` request → Copy → Copy as -cURL, and save it: +**If you use Firefox, Zen, LibreWolf, Waterfox or Floorp**, sign in to the Asset Store once +in that browser and point unity-sync at it: + +```toml +# ~/.config/unity-sync/config.toml +session_source = "browser" +``` + +```bash +unity-sync status # no paste, no expiry to babysit +``` + +It reads only the `unity.com` cookies out of the session store and discards the rest of the +file. You do not need to keep an Asset Store tab open; the cookie lasts as long as the +browsing session does. `--session browser` does the same thing for one run, and the run +prints which profile it read. + +Chromium-family browsers keep session cookies somewhere else entirely, encrypted, so they +are not supported. + +**Otherwise, paste a session.** In DevTools → Network, right-click any +`assetstore.unity.com` request → Copy → Copy as cURL, and save the whole thing verbatim — +no extracting values, no escaping: ```bash $EDITOR ~/.config/unity-sync/session.curl # paste, save @@ -75,10 +96,15 @@ A Netscape `cookies.txt` export works too, as long as your exporter keeps HttpOn `session_source` in `config.toml`, or just save it as `session.curl` or `cookies.txt` in the config dir, where unity-sync looks by default. -If the file is missing the `LS` cookie, unity-sync says so before making any request, -because the store's own answer in that case is an HTTP 500 that reads like a server fault. +`--session` also takes a browser profile directory or a `recovery.jsonlz4` straight, which +covers a Gecko browser this does not know where to look for. Whatever you point it at, +unity-sync works out what the file is by reading it. + +If the session has no `LS` cookie, unity-sync says so before making any request, because +the store's own answer in that case is an HTTP 500 that reads like a server fault. -A pasted session expires. When it does, re-copy it. +A pasted session expires. When it does, re-copy it, or switch to `session_source = +"browser"` and stop re-copying. ## Commands diff --git a/config.example.toml b/config.example.toml index 7395996..61e31e0 100644 --- a/config.example.toml +++ b/config.example.toml @@ -14,6 +14,9 @@ # # There is no browser option. The cookie the store checks is a session cookie that never # reaches a browser's cookie database, so reading one could not authenticate. +# session_source = "browser" reads a signed-in Firefox-family session store, which is the +# only way to avoid re-pasting when the session expires. It also accepts a path to a pasted +# curl file, a cookies.txt, a browser profile directory, or a recovery.jsonlz4. # session_source = "~/.config/unity-sync/session.curl" # Where packages are cached. Default: $XDG_DATA_HOME/unity-sync diff --git a/docs/design.md b/docs/design.md index e345c37..1788d2c 100644 --- a/docs/design.md +++ b/docs/design.md @@ -38,9 +38,27 @@ The GraphQL body is a batch: a JSON array of operations, answered by a positiona full owned list; a junk or absent `__Secure-next-auth.session-token` changes nothing; removing `LS` turns any user-scoped query into an HTTP 500 with an empty `GraphqlError`. -`LS` is a session cookie, so it never reaches a browser's cookie database. That is why the -only supported session sources are a pasted curl command and a `cookies.txt` export, and -why there is no browser-reading mode: it could not work. +`LS` is a session cookie, so it never reaches `cookies.sqlite`. It does reach a +Firefox-family **session store**, which is a different file for a different purpose: +Gecko's `sessionstore-backups/recovery.jsonlz4` records the cookies of every host the +browsing session touched so the session can be restored. Measured on a real profile: `LS` +and `_csrf` are both in there, and the two of them alone return HTTP 200 with the full +owned list. + +Two properties decide what that can promise. The jar is **not tab-scoped** — on the profile +measured, 156 of 169 cookie hosts had no tab open anywhere in the session, and +`assetstore.unity.com` was one of them — so the credential survives closing the tab and +lasts as long as the browsing session. And the file is rewritten **periodically**, not on +every cookie change, so it lags a sign-in by seconds. + +The supported sources are therefore a session store, a pasted curl command, and a +`cookies.txt` export. Which one a path is gets decided by reading it: a session store is +identified by its `mozLz40\0` magic, a curl paste by its structure. Chromium keeps session +cookies in an encrypted SQLite database instead, so it is out of scope. + +The session store is read narrowly on purpose. It holds credentials for every host the +session touched, so `internal/session` filters to the `unity.com` family before anything +leaves the package, and no cookie value is ever logged. The `_csrf` cookie is a double-submit token required by the GraphQL endpoint only. Not every storefront route issues it — `/` and `/publishers/{id}` answer 200 and set nothing, diff --git a/internal/config/config.go b/internal/config/config.go index 92aeff2..a7a2fd6 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -15,7 +15,8 @@ import ( // Config is the resolved user-scoped configuration. type Config struct { - // SessionSource is a path to a pasted-curl file or a cookies.txt. + // SessionSource is "browser" to read a signed-in Firefox-family session, or a path to + // a pasted-curl file, a cookies.txt, a browser profile, or a recovery.jsonlz4. SessionSource string // LibraryPath is where packages are mirrored. A user may point this at Unity's own diff --git a/internal/session/browser.go b/internal/session/browser.go new file mode 100644 index 0000000..001d4bd --- /dev/null +++ b/internal/session/browser.go @@ -0,0 +1,263 @@ +package session + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "runtime" + "strings" +) + +// sessionStoreName is the file Gecko rewrites periodically with the live session, +// including the cookies for every host visited during it. +const sessionStoreName = "recovery.jsonlz4" + +// BrowserKeyword asks Resolve to find a signed-in Gecko profile instead of reading a file. +const BrowserKeyword = "browser" + +// geckoRoots lists where Gecko browsers keep their profile directories, relative to the +// home directory. Only Zen's layout was verified against a real install; the others are +// the standard Gecko layout, which every fork inherits along with profiles.ini. +// +// The list is a convenience, not the mechanism: Resolve also accepts a path to a profile +// directory or straight to a recovery.jsonlz4, so a browser missing from here still works +// by pointing at it. +func geckoRoots() []string { + home, err := os.UserHomeDir() + if err != nil { + return nil + } + var rel []string + switch runtime.GOOS { + case "darwin": + rel = []string{ + "Library/Application Support/zen", + "Library/Application Support/Firefox", + "Library/Application Support/LibreWolf", + "Library/Application Support/Waterfox", + "Library/Application Support/Floorp", + } + case "windows": + rel = []string{"AppData/Roaming/zen", "AppData/Roaming/Mozilla/Firefox"} + default: + rel = []string{ + ".config/zen", ".zen", + ".mozilla/firefox", + ".librewolf", + ".waterfox", + ".floorp", ".config/floorp", + } + } + roots := make([]string, 0, len(rel)) + for _, r := range rel { + roots = append(roots, filepath.Join(home, filepath.FromSlash(r))) + } + return roots +} + +// profileDirs returns every profile under a Gecko root, the one the browser is actually +// running first. +// +// The ordering is the point. profiles.ini can mark one profile `Default=1` while the +// install is running a different one, and the install's choice is the profile with a live +// session: on the machine this was built against, the `Default=1` profile has no +// sessionstore-backups directory at all. Reading only the flag finds an empty profile and +// reports no session on a machine where there plainly is one. +func profileDirs(root string) []string { + preferred := installDefaults(root) + var rest []string + for _, p := range iniPaths(filepath.Join(root, "profiles.ini")) { + full := filepath.Join(root, filepath.FromSlash(p)) + if !contains(preferred, full) { + rest = append(rest, full) + } + } + return append(preferred, rest...) +} + +// installDefaults reads installs.ini, which records the profile each installation of the +// browser last used. +func installDefaults(root string) []string { + var out []string + for _, p := range iniValues(filepath.Join(root, "installs.ini"), "Default") { + out = append(out, filepath.Join(root, filepath.FromSlash(p))) + } + return out +} + +// iniPaths pulls the Path= entries out of a profiles.ini. +func iniPaths(path string) []string { return iniValues(path, "Path") } + +// iniValues reads one key from every section of a Mozilla ini. The format is plain enough +// that a full ini parser would be more code than it saves, and both files this reads are +// written by the browser rather than by a user. +func iniValues(path, key string) []string { + raw, err := os.ReadFile(path) + if err != nil { + return nil + } + var out []string + for _, line := range strings.Split(string(raw), "\n") { + line = strings.TrimSpace(line) + k, v, ok := strings.Cut(line, "=") + if !ok || !strings.EqualFold(strings.TrimSpace(k), key) { + continue + } + if v = strings.TrimSpace(v); v != "" { + out = append(out, v) + } + } + return out +} + +func contains(haystack []string, needle string) bool { + for _, h := range haystack { + if h == needle { + return true + } + } + return false +} + +// storeCookie is the shape Gecko records a cookie in. The value is read and used, never +// logged: this file holds the credentials for every host the browsing session touched. +type storeCookie struct { + Host string `json:"host"` + Name string `json:"name"` + Value string `json:"value"` +} + +// sessionStore is the slice of recovery.jsonlz4 this needs. Cookies sit under each window; +// the top-level array is accepted too, since it costs one field to not depend on which of +// the two a given version writes. +type sessionStore struct { + Windows []struct { + Cookies []storeCookie `json:"cookies"` + } `json:"windows"` + Cookies []storeCookie `json:"cookies"` +} + +// fromSessionStore reads a Gecko session store and keeps only the store's own cookies. +// +// Everything else in the file is discarded before it can reach a caller. The jar spans +// every host visited in the browsing session, not just the ones with a tab still open, so +// narrowing to the unity.com family here — rather than anywhere later — keeps unrelated +// credentials out of the rest of the program. +func fromSessionStore(raw []byte) (map[string]string, error) { + decoded, err := decodeMozLZ4(raw) + if err != nil { + return nil, err + } + var store sessionStore + if err := json.Unmarshal(decoded, &store); err != nil { + return nil, fmt.Errorf("session store is not the JSON this expects: %w", err) + } + + pairs := map[string]string{} + take := func(jar []storeCookie) { + for _, c := range jar { + if hostMatches(c.Host) { + pairs[c.Name] = c.Value + } + } + } + for _, w := range store.Windows { + take(w.Cookies) + } + take(store.Cookies) + + if len(pairs) == 0 { + return nil, fmt.Errorf("no %s cookies in the session store", cookieDomain) + } + return pairs, nil +} + +// isMozLZ4 reports whether a file starts with Mozilla's compressed-blob magic, which is +// how a session store is told apart from a pasted curl or a cookies.txt without asking +// the user to declare which one they saved. +func isMozLZ4(raw []byte) bool { + return len(raw) >= len(mozlz4Magic) && string(raw[:len(mozlz4Magic)]) == mozlz4Magic +} + +// storeCandidates lists the session-store files worth trying for a source, in the order +// they should be tried. +// +// A source is one of: the browser keyword, which sweeps every known Gecko root; a browser +// root holding profiles.ini; a single profile directory; or a path straight to a session +// store. Anything the caller names is tried first and alone, so pointing at a specific +// profile never silently falls through to a different browser. +func storeCandidates(source string) []string { + if source == BrowserKeyword { + var out []string + for _, root := range geckoRoots() { + out = append(out, storesUnder(root)...) + } + return out + } + if fi, err := os.Stat(source); err == nil && !fi.IsDir() { + return []string{source} + } + return storesUnder(source) +} + +// storesUnder finds the session stores beneath a browser root or a single profile. +func storesUnder(dir string) []string { + var out []string + direct := filepath.Join(dir, "sessionstore-backups", sessionStoreName) + if _, err := os.Stat(direct); err == nil { + out = append(out, direct) + } + for _, p := range profileDirs(dir) { + candidate := filepath.Join(p, "sessionstore-backups", sessionStoreName) + if _, err := os.Stat(candidate); err == nil && !contains(out, candidate) { + out = append(out, candidate) + } + } + return out +} + +// resolveBrowser returns the Cookie header from the first session store that carries the +// credential, along with the file it came from. +// +// A profile that parses but holds no LS is skipped rather than fatal: a second browser, +// or a second profile in the same browser, is where the signed-in tab usually is. +func resolveBrowser(source string) (header, from string, err error) { + candidates := storeCandidates(source) + if len(candidates) == 0 { + return "", "", fmt.Errorf("no Firefox-family session store found for %q", source) + } + var skipped []string + for _, path := range candidates { + raw, readErr := os.ReadFile(path) + if readErr != nil { + skipped = append(skipped, fmt.Sprintf("%s (%v)", path, readErr)) + continue + } + pairs, parseErr := fromSessionStore(raw) + if parseErr != nil { + skipped = append(skipped, fmt.Sprintf("%s (%v)", path, parseErr)) + continue + } + if _, ok := pairs[CredentialCookie]; !ok { + skipped = append(skipped, fmt.Sprintf("%s (no %s cookie)", path, CredentialCookie)) + continue + } + return join(pairs), path, nil + } + return "", "", &ErrNoBrowserCredential{Source: source, Skipped: skipped} +} + +// ErrNoBrowserCredential means session stores were found and read but none held the +// credential. It lists what was tried, because the usual cause is that the browser has +// never signed in during this browsing session rather than anything being misconfigured. +type ErrNoBrowserCredential struct { + Source string + Skipped []string +} + +func (e *ErrNoBrowserCredential) Error() string { + return fmt.Sprintf("no %s cookie in any Firefox-family session store for %q: the browser keeps it "+ + "for the life of a browsing session, so sign in to the Asset Store in that browser and "+ + "try again (looked at: %s)", CredentialCookie, e.Source, strings.Join(e.Skipped, "; ")) +} diff --git a/internal/session/browser_test.go b/internal/session/browser_test.go new file mode 100644 index 0000000..4efb2bb --- /dev/null +++ b/internal/session/browser_test.go @@ -0,0 +1,324 @@ +package session + +import ( + "encoding/binary" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" +) + +// mozlz4Stored wraps payload in the container Gecko writes, as the single all-literals +// sequence LZ4 emits for incompressible input. It has to be one sequence: only the last +// one in a block may omit its match, so a chain of literal-only sequences is not a legal +// block and the decoder is right to reject it. +func mozlz4Stored(t *testing.T, payload []byte) []byte { + t.Helper() + out := []byte(mozlz4Magic) + out = binary.LittleEndian.AppendUint32(out, uint32(len(payload))) + + if n := len(payload); n < 15 { + out = append(out, byte(n)<<4) + } else { + out = append(out, byte(15)<<4) + for remainder := n - 15; ; { + if remainder >= 255 { + out = append(out, 255) + remainder -= 255 + continue + } + out = append(out, byte(remainder)) + break + } + } + return append(out, payload...) +} + +// A match may point into bytes the same match is still writing, which is how the format +// encodes a repeating run. Copying in bulk would read the pre-overlap bytes instead of +// the ones just produced, so this is hand-built rather than generated: the encoder above +// emits literals only and would never exercise it. +func TestMozLZ4ExpandsAnOverlappingMatch(t *testing.T) { + // literals "abc", then a match of length 9 at offset 3 — each copied byte is one the + // match itself just wrote. + block := []byte{0x35, 'a', 'b', 'c', 0x03, 0x00} + raw := append([]byte(mozlz4Magic), 0, 0, 0, 0) + binary.LittleEndian.PutUint32(raw[len(mozlz4Magic):], 12) + raw = append(raw, block...) + + got, err := decodeMozLZ4(raw) + if err != nil { + t.Fatalf("decode: %v", err) + } + if string(got) != "abcabcabcabc" { + t.Errorf("decoded %q, want %q", got, "abcabcabcabc") + } +} + +// A match pointing before the start of the output is the shape a corrupt or hostile file +// takes; it must be refused rather than read out of bounds. +func TestMozLZ4RefusesAMatchBeforeTheStart(t *testing.T) { + block := []byte{0x35, 'a', 'b', 'c', 0xFF, 0x00} // offset 255, only 3 bytes decoded + raw := append([]byte(mozlz4Magic), 0, 0, 0, 0) + binary.LittleEndian.PutUint32(raw[len(mozlz4Magic):], 12) + raw = append(raw, block...) + + if _, err := decodeMozLZ4(raw); err == nil { + t.Error("decode followed a match offset past the start of the output") + } +} + +func storeJSON(t *testing.T, cookies []storeCookie) []byte { + t.Helper() + doc := map[string]any{ + "windows": []any{map[string]any{"cookies": cookies, "tabs": []any{}}}, + } + raw, err := json.Marshal(doc) + if err != nil { + t.Fatal(err) + } + return raw +} + +// writeProfile lays out a Gecko root the way a real install does. +func writeProfile(t *testing.T, root, profile string, cookies []storeCookie) string { + t.Helper() + dir := filepath.Join(root, profile, "sessionstore-backups") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, sessionStoreName) + if err := os.WriteFile(path, mozlz4Stored(t, storeJSON(t, cookies)), 0o600); err != nil { + t.Fatal(err) + } + return path +} + +func TestMozLZ4RoundTripsAndRejectsRubbish(t *testing.T) { + payload := []byte(strings.Repeat(`{"windows":[{"cookies":[]}]}`, 200)) + got, err := decodeMozLZ4(mozlz4Stored(t, payload)) + if err != nil { + t.Fatalf("decode: %v", err) + } + if string(got) != string(payload) { + t.Error("round trip changed the payload") + } + + if _, err := decodeMozLZ4([]byte("not compressed at all")); err != errNotMozLZ4 { + t.Errorf("decode of a plain file = %v, want errNotMozLZ4", err) + } + + // A header claiming more than the block delivers must be an error, not a short read + // that later parses as truncated JSON. + lying := mozlz4Stored(t, payload) + binary.LittleEndian.PutUint32(lying[len(mozlz4Magic):], uint32(len(payload)+500)) + if _, err := decodeMozLZ4(lying); err == nil { + t.Error("decode accepted a header that overstates the payload") + } + + // The size comes off disk, so an absurd claim must be refused rather than allocated. + huge := mozlz4Stored(t, payload) + binary.LittleEndian.PutUint32(huge[len(mozlz4Magic):], 1<<30) + if _, err := decodeMozLZ4(huge); err == nil || !strings.Contains(err.Error(), "ceiling") { + t.Errorf("decode of an absurd size = %v, want the ceiling to refuse it", err) + } +} + +// A truncated or scrambled block must fail rather than return whatever it managed, since +// the caller would otherwise treat partial JSON as a missing credential. +func TestMozLZ4RefusesACorruptBlock(t *testing.T) { + full := mozlz4Stored(t, []byte(strings.Repeat("payload ", 64))) + for _, cut := range []int{len(full) / 2, len(full) - 1} { + if _, err := decodeMozLZ4(full[:cut]); err == nil { + t.Errorf("decode of a block truncated to %d bytes returned no error", cut) + } + } +} + +// The jar holds every host the browsing session touched. Only the store's own cookies may +// leave this package. +func TestOnlyUnityCookiesLeaveTheSessionStore(t *testing.T) { + raw := mozlz4Stored(t, storeJSON(t, []storeCookie{ + {Host: "assetstore.unity.com", Name: "LS", Value: "credential"}, + {Host: "assetstore.unity.com", Name: "_csrf", Value: "token"}, + {Host: ".unity.com", Name: "PIM-SESSION-ID", Value: "pim"}, + {Host: "bank.example.com", Name: "session", Value: "SHOULD-NOT-LEAK"}, + {Host: "notunity.com", Name: "session", Value: "SHOULD-NOT-LEAK"}, + {Host: "evil-unity.com.attacker.test", Name: "session", Value: "SHOULD-NOT-LEAK"}, + })) + pairs, err := fromSessionStore(raw) + if err != nil { + t.Fatal(err) + } + if len(pairs) != 3 { + t.Errorf("kept %d cookies, want the 3 unity.com ones: %v", len(pairs), keys(pairs)) + } + for name, value := range pairs { + if strings.Contains(value, "SHOULD-NOT-LEAK") { + t.Errorf("cookie %q came from another site", name) + } + } + header := join(pairs) + if strings.Contains(header, "SHOULD-NOT-LEAK") { + t.Error("an unrelated site's cookie reached the Cookie header") + } + if !strings.Contains(header, "LS=credential") { + t.Errorf("header %q is missing the credential", header) + } +} + +// profiles.ini can mark one profile Default=1 while the browser runs another, and only +// the running one has a live session. Preferring the flag finds an empty profile and +// reports no session on a machine where there plainly is one. +func TestTheRunningProfileWinsOverTheDefaultFlag(t *testing.T) { + root := t.TempDir() + writeProfile(t, root, "aaaa.Default Profile", []storeCookie{ + {Host: "assetstore.unity.com", Name: "LS", Value: "stale-profile"}, + }) + writeProfile(t, root, "bbbb.Default (release)", []storeCookie{ + {Host: "assetstore.unity.com", Name: "LS", Value: "running-profile"}, + }) + os.WriteFile(filepath.Join(root, "profiles.ini"), []byte( + "[Profile1]\nName=Default Profile\nIsRelative=1\nPath=aaaa.Default Profile\nDefault=1\n\n"+ + "[Profile0]\nName=Default (release)\nIsRelative=1\nPath=bbbb.Default (release)\n"), 0o644) + os.WriteFile(filepath.Join(root, "installs.ini"), []byte( + "[15B76BAA26BA15E7]\nDefault=bbbb.Default (release)\nLocked=1\n"), 0o644) + + header, from, err := ResolveFrom(root) + if err != nil { + t.Fatalf("ResolveFrom: %v", err) + } + if !strings.Contains(header, "running-profile") { + t.Errorf("header came from the Default=1 profile, not the one installs.ini names: %q", from) + } +} + +// A profile whose session has no Asset Store credential is skipped, not fatal: the +// signed-in one is usually another profile or another browser. +func TestAProfileWithoutTheCredentialIsSkipped(t *testing.T) { + root := t.TempDir() + writeProfile(t, root, "aaaa.empty", []storeCookie{ + {Host: "assetstore.unity.com", Name: "_csrf", Value: "token-but-no-credential"}, + }) + writeProfile(t, root, "bbbb.signed-in", []storeCookie{ + {Host: "assetstore.unity.com", Name: "LS", Value: "credential"}, + }) + os.WriteFile(filepath.Join(root, "profiles.ini"), []byte( + "[Profile0]\nIsRelative=1\nPath=aaaa.empty\nDefault=1\n\n"+ + "[Profile1]\nIsRelative=1\nPath=bbbb.signed-in\n"), 0o644) + + header, _, err := ResolveFrom(root) + if err != nil { + t.Fatalf("ResolveFrom: %v", err) + } + if !strings.Contains(header, "LS=credential") { + t.Errorf("header = %q, want the signed-in profile's credential", header) + } +} + +// When nothing holds the credential the error has to say what was tried, because the +// cause is usually "that browser never signed in" rather than a broken setup. +func TestNoCredentialAnywhereNamesWhatWasTried(t *testing.T) { + root := t.TempDir() + writeProfile(t, root, "aaaa.empty", []storeCookie{ + {Host: "assetstore.unity.com", Name: "_csrf", Value: "token"}, + }) + os.WriteFile(filepath.Join(root, "profiles.ini"), + []byte("[Profile0]\nIsRelative=1\nPath=aaaa.empty\n"), 0o644) + + _, _, err := ResolveFrom(root) + if err == nil { + t.Fatal("ResolveFrom succeeded with no credential anywhere") + } + var missing *ErrNoBrowserCredential + if !asErr(err, &missing) { + t.Fatalf("error is %T, want *ErrNoBrowserCredential", err) + } + if !strings.Contains(err.Error(), "aaaa.empty") { + t.Errorf("error does not name the profile it read: %v", err) + } + if !strings.Contains(err.Error(), "sign in") { + t.Errorf("error does not say what to do about it: %v", err) + } +} + +// A session store is identified by its contents, so the same flag takes a paste, a +// cookies.txt or a recovery file without the user declaring which. +func TestResolveTellsASessionStoreFromAPasteByContent(t *testing.T) { + dir := t.TempDir() + + store := filepath.Join(dir, "recovery.jsonlz4") + os.WriteFile(store, mozlz4Stored(t, storeJSON(t, []storeCookie{ + {Host: "assetstore.unity.com", Name: "LS", Value: "from-store"}, + })), 0o600) + + paste := filepath.Join(dir, "session.curl") + os.WriteFile(paste, []byte(`curl 'https://assetstore.unity.com/api/graphql/batch' -H 'Cookie: LS=from-paste; _csrf=t'`), 0o600) + + for path, want := range map[string]string{store: "from-store", paste: "from-paste"} { + header, _, err := ResolveFrom(path) + if err != nil { + t.Fatalf("%s: %v", filepath.Base(path), err) + } + if !strings.Contains(header, "LS="+want) { + t.Errorf("%s produced %q, want LS=%s", filepath.Base(path), header, want) + } + } +} + +func asErr(err error, target any) bool { return errors.As(err, target) } + +func keys(m map[string]string) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + return out +} + +// A match longer than 18 bytes encodes its length the same way a long literal run does, +// through the 15-plus-extension-bytes escape. Nothing above reaches that path. +func TestMozLZ4ExpandsAnExtendedMatchLength(t *testing.T) { + // literals "ab", then a 30-byte match at offset 2: low nibble 15 plus one extension + // byte of 11, since the encoder stores length minus the four-byte minimum. + block := []byte{0x2F, 'a', 'b', 0x02, 0x00, 11} + raw := append([]byte(mozlz4Magic), 0, 0, 0, 0) + binary.LittleEndian.PutUint32(raw[len(mozlz4Magic):], 32) + raw = append(raw, block...) + + got, err := decodeMozLZ4(raw) + if err != nil { + t.Fatalf("decode: %v", err) + } + if want := strings.Repeat("ab", 16); string(got) != want { + t.Errorf("decoded %q, want %q", got, want) + } +} + +// Synthetic blocks only prove the decoder against blocks this file wrote. Point this at a +// real profile to check it against bytes Gecko produced: +// +// UNITY_SYNC_REAL_SESSIONSTORE=~/.config/zen//sessionstore-backups/recovery.jsonlz4 go test ./internal/session/ +// +// Skipped by default: the file is a live credential store and belongs to no CI run. +func TestDecodesARealSessionStore(t *testing.T) { + path := os.Getenv("UNITY_SYNC_REAL_SESSIONSTORE") + if path == "" { + t.Skip("set UNITY_SYNC_REAL_SESSIONSTORE to a recovery.jsonlz4") + } + raw, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + decoded, err := decodeMozLZ4(raw) + if err != nil { + t.Fatalf("decode: %v", err) + } + var doc any + if err := json.Unmarshal(decoded, &doc); err != nil { + t.Fatalf("decoded %d bytes that are not JSON: %v", len(decoded), err) + } + t.Logf("decoded %d compressed bytes into %d bytes of valid JSON", len(raw), len(decoded)) +} diff --git a/internal/session/mozlz4.go b/internal/session/mozlz4.go new file mode 100644 index 0000000..b76da32 --- /dev/null +++ b/internal/session/mozlz4.go @@ -0,0 +1,123 @@ +package session + +import ( + "encoding/binary" + "errors" + "fmt" +) + +// mozlz4Magic prefixes every file Mozilla writes through its lz4 helper. +const mozlz4Magic = "mozLz40\x00" + +// errNotMozLZ4 means the file is not one of Mozilla's compressed blobs. It is separate +// from a corruption error because a caller scanning several profiles wants to skip a +// stray file rather than fail the whole scan. +var errNotMozLZ4 = errors.New("not a mozlz4 file") + +// decodeMozLZ4 unpacks the "mozLz40\0" container: the magic, a little-endian uint32 +// holding the decompressed size, then a single raw LZ4 block. +// +// The block format is decoded here rather than pulled from a dependency. This is the +// tool's only compressed input and the format is a few dozen lines, against a module that +// would have to be vendored into a build that otherwise has one dependency. +func decodeMozLZ4(raw []byte) ([]byte, error) { + if len(raw) < len(mozlz4Magic)+4 || string(raw[:len(mozlz4Magic)]) != mozlz4Magic { + return nil, errNotMozLZ4 + } + size := binary.LittleEndian.Uint32(raw[len(mozlz4Magic):]) + // The header is the only thing declaring the output length, and it comes off disk, so + // it is a claim rather than a fact. Cap it: a session store runs to a few megabytes, + // and a corrupt or hostile header should not turn into a gigabyte allocation. + const maxDecompressed = 256 << 20 + if size > maxDecompressed { + return nil, fmt.Errorf("mozlz4 header claims %d bytes, past the %d-byte ceiling", size, maxDecompressed) + } + return lz4Decompress(raw[len(mozlz4Magic)+4:], int(size)) +} + +// lz4Decompress expands one raw LZ4 block into exactly want bytes. +// +// A block is a sequence of sequences. Each starts with a token byte: the high nibble is +// the literal run length, the low nibble is the match length minus four. A nibble of 15 +// means "add the following bytes until one is not 255". After the literals comes a +// little-endian uint16 back-reference offset into the output produced so far. The final +// sequence carries literals and stops, with no offset. +// +// Every index is bounds-checked against the input, and every match against the output +// already written, because this parses a file that another program wrote. +func lz4Decompress(src []byte, want int) ([]byte, error) { + dst := make([]byte, 0, want) + var i int + + readLength := func(n int) (int, error) { + if n != 15 { + return n, nil + } + for { + if i >= len(src) { + return 0, errors.New("lz4: length runs past the end of the block") + } + b := int(src[i]) + i++ + n += b + if b != 255 { + return n, nil + } + if n > want { + return 0, errors.New("lz4: length exceeds the declared output size") + } + } + } + + for i < len(src) { + token := src[i] + i++ + + literals, err := readLength(int(token >> 4)) + if err != nil { + return nil, err + } + if literals > 0 { + if i+literals > len(src) { + return nil, errors.New("lz4: literal run runs past the end of the block") + } + dst = append(dst, src[i:i+literals]...) + i += literals + } + + // The last sequence is literals only: no offset follows it. + if i == len(src) { + break + } + if i+2 > len(src) { + return nil, errors.New("lz4: truncated match offset") + } + offset := int(binary.LittleEndian.Uint16(src[i:])) + i += 2 + if offset == 0 || offset > len(dst) { + return nil, fmt.Errorf("lz4: match offset %d is outside the %d bytes decoded so far", offset, len(dst)) + } + + length, err := readLength(int(token & 0x0F)) + if err != nil { + return nil, err + } + length += 4 // the minimum match, which the encoder subtracts + + // Copied one byte at a time on purpose: a match may overlap the region it is + // copying from, which is how the format encodes a repeating run, and a bulk copy + // would read the pre-overlap bytes instead of the ones just written. + start := len(dst) - offset + for n := 0; n < length; n++ { + dst = append(dst, dst[start+n]) + } + if len(dst) > want { + return nil, fmt.Errorf("lz4: output grew past the declared %d bytes", want) + } + } + + if len(dst) != want { + return nil, fmt.Errorf("lz4: decoded %d bytes, header declared %d", len(dst), want) + } + return dst, nil +} diff --git a/internal/session/session.go b/internal/session/session.go index 2cfc052..63edf26 100644 --- a/internal/session/session.go +++ b/internal/session/session.go @@ -1,9 +1,12 @@ // Package session turns a saved browser session into the Cookie header the Asset Store // requires. The credential is the LS cookie: measured against the live store, _csrf plus // LS alone returns a full owned-asset list, and the NextAuth session token is not -// consulted by either endpoint this tool uses. LS is a session cookie, so no browser -// cookie database ever holds it — which is why the only supported sources are a pasted -// curl command or a cookies.txt export. +// consulted by either endpoint this tool uses. +// +// LS is a session cookie, so cookies.sqlite never holds it. A Firefox-family session +// store does, because Gecko records the cookies of the hosts its open tabs are using, and +// that file is a supported source alongside a pasted curl command and a cookies.txt +// export. Which one a path is gets decided by reading it, not by configuration. package session import ( @@ -35,27 +38,56 @@ const httpOnlyPrefix = "#HttpOnly_" type ErrNoCredential struct{ Source string } func (e *ErrNoCredential) Error() string { - return fmt.Sprintf("session %s has no %s cookie: Unity keeps it in memory only, so a browser "+ - "cookie database never has it — re-copy the session from DevTools (Network > any "+ - "assetstore.unity.com request > Copy as cURL) while signed in", e.Source, CredentialCookie) + return fmt.Sprintf("session %s has no %s cookie: Unity keeps it in memory only, so cookies.sqlite "+ + "never has it — re-copy the session from DevTools (Network > any assetstore.unity.com "+ + "request > Copy as cURL) while signed in, or set session_source = %q to read it from a "+ + "signed-in Firefox-family tab", e.Source, CredentialCookie, BrowserKeyword) } -// Resolve reads a session file and returns the Cookie header for the store. It also -// asserts the credential is present, whatever the source, so the diagnostic names the -// real problem instead of leaving it to a 500. -func Resolve(path string) (string, error) { - raw, err := os.ReadFile(path) +// Resolve turns a session source into the Cookie header for the store. It also asserts +// the credential is present, whatever the source, so the diagnostic names the real problem +// instead of leaving it to a 500. +// +// A source is the browser keyword, a directory (a Gecko profile or the root holding +// several), or a file. A file is identified by its contents: a compressed session store, a +// pasted curl command, or a cookies.txt. +func Resolve(source string) (string, error) { + header, _, err := ResolveFrom(source) + return header, err +} + +// ResolveFrom is Resolve, also reporting which file the credential came from. The browser +// keyword can search several profiles, and a run that picked one of them should be able to +// say so rather than leaving the user to guess which tab it read. +func ResolveFrom(source string) (header, from string, err error) { + if source == BrowserKeyword { + return resolveBrowser(source) + } + fi, statErr := os.Stat(source) + if statErr != nil { + return "", "", statErr + } + if fi.IsDir() { + return resolveBrowser(source) + } + + raw, err := os.ReadFile(source) if err != nil { - return "", err + return "", "", err + } + var pairs map[string]string + if isMozLZ4(raw) { + pairs, err = fromSessionStore(raw) + } else { + pairs, err = parse(string(raw)) } - pairs, err := parse(string(raw)) if err != nil { - return "", fmt.Errorf("%s: %w", path, err) + return "", "", fmt.Errorf("%s: %w", source, err) } if _, ok := pairs[CredentialCookie]; !ok { - return "", &ErrNoCredential{Source: path} + return "", "", &ErrNoCredential{Source: source} } - return join(pairs), nil + return join(pairs), source, nil } // Discover looks for a session file in the user config dir, so a first run needs no diff --git a/main.go b/main.go index 201135d..854734d 100644 --- a/main.go +++ b/main.go @@ -179,11 +179,20 @@ func resolveSession(cfg config.Config, configDir string) (string, error) { } } if src == "" { - return "", fmt.Errorf("no session configured: save a pasted-curl file as %s, "+ - "set session_source in config.toml, or pass --session", - filepath.Join(configDir, "session.curl")) + return "", fmt.Errorf("no session configured: set session_source = %q in config.toml to read "+ + "it from a signed-in Firefox-family browser, save a pasted-curl file as %s, or pass "+ + "--session", session.BrowserKeyword, filepath.Join(configDir, "session.curl")) } - return session.Resolve(src) + header, from, err := session.ResolveFrom(src) + if err != nil { + return "", err + } + // Which profile a browser search settled on is not obvious from `--session browser`, + // and a run against the wrong signed-in account is otherwise silent. + if src == session.BrowserKeyword { + fmt.Fprintln(os.Stderr, "session: read from", from) + } + return header, nil } // enumerator is the slice of the store client that `select` needs.