Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/stream-large-downloads.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"firebase-storage-kit": patch
---

Add `downloadStream()` for progress-aware, cancellable downloads without buffering the entire object in memory.
26 changes: 26 additions & 0 deletions apps/docs/content/docs/api/storage-manager.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,32 @@ getDownloadURL(path: string): Promise<string>

Returns a download URL for the object at `path`.

### downloadStream

```ts
downloadStream(
path: string,
options?: DownloadStreamOptions
): Promise<DownloadStreamResult>
```

Fetches the object and returns a `ReadableStream<Uint8Array>` without buffering the whole download in memory. `onProgress(loaded, total)` runs as the caller consumes chunks from the stream, and `signal` accepts an `AbortSignal` for cancellation.

```ts
const controller = new AbortController();
const { stream, totalBytes, contentType } = await manager.downloadStream(
"videos/launch-demo.mp4",
{
signal: controller.signal,
onProgress: (loaded, total) => {
console.log(`${Math.round((loaded / total) * 100)}%`);
},
}
);
```

The returned `totalBytes` comes from file's object metadata. Progress does not advance until `stream` is read, piped, or otherwise consumed.

### delete

```ts
Expand Down
27 changes: 27 additions & 0 deletions apps/docs/content/docs/api/types.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ Key types re-exported from `firebase-storage-kit`. Import them alongside classes

```ts
import type {
DownloadStreamOptions,
DownloadStreamResult,
UploadOptions,
UploadItem,
StorageState,
Expand All @@ -30,6 +32,31 @@ interface UploadOptions {

`onConflict` defaults to `"overwrite"`. See [Conflict handling](/docs/guides/conflict-handling) for behavior, examples, and cross-client limitations.

## DownloadStreamOptions

Options for `downloadStream`:

```ts
interface DownloadStreamOptions {
onProgress?: (loaded: number, total: number) => void;
signal?: AbortSignal;
}
```

`onProgress` runs when the returned stream is consumed. Abort `signal` to cancel both the request and further stream reads.

## DownloadStreamResult

```ts
interface DownloadStreamResult {
stream: ReadableStream<Uint8Array>;
totalBytes: number;
contentType?: string;
}
```

Pipe or read `stream` to receive the object without first collecting the entire download into a `Blob`.

## UploadValidationOptions

Pre-upload validation rules. See [Validation](/docs/guides/validation) for usage and error handling.
Expand Down
33 changes: 33 additions & 0 deletions apps/docs/content/docs/guides/querying-files.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,39 @@ const url = await manager.getDownloadURL("uploads/photo.jpg");

Returns a public or tokenized URL depending on your bucket rules and object ACLs.

## Stream a large download

Use `downloadStream` when the object should flow to another destination without first becoming one large in-memory `Blob`:

```ts
const fileHandle = await window.showSaveFilePicker({
suggestedName: "launch-demo.mp4",
});
const writable = await fileHandle.createWritable();

const { stream } = await manager.downloadStream("videos/launch-demo.mp4", {
onProgress: (loaded, total) => {
console.log(`${Math.round((loaded / total) * 100)}%`);
},
});

await stream.pipeTo(writable);
```

Progress starts when `pipeTo` consumes the stream. The File System Access API used above is primarily available in Chromium-based browsers; in other browsers, pipe the stream to a supported destination or use `getDownloadURL` for normal browser playback and downloads.

Cancel an active request with an `AbortController`:

```ts
const controller = new AbortController();
const { stream } = await manager.downloadStream("exports/account-data.zip", {
signal: controller.signal,
});

cancelButton.addEventListener("click", () => controller.abort());
await stream.pipeTo(writable);
```

## Delete a file

```ts
Expand Down
14 changes: 14 additions & 0 deletions packages/firebase-storage-kit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,20 @@ handle.on("error", (upload) => {
});
```

### Stream large downloads

```ts
const { stream } = await manager.downloadStream("videos/launch-demo.mp4", {
onProgress: (loaded, total) => {
console.log(`${Math.round((loaded / total) * 100)}%`);
},
});

await stream.pipeTo(writable);
```

The object is delivered as a `ReadableStream`, so it can be piped without buffering the full file in memory.

### React

```tsx
Expand Down
48 changes: 48 additions & 0 deletions packages/firebase-storage-kit/src/core/storage-manager.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
import type { StorageProvider } from "../providers/provider";
import type {
DownloadStreamOptions,
DownloadStreamResult,
} from "../types/download";
import type { ListOptions, StorageListResult } from "../types/list";
import type { FileMetadata } from "../types/metadata";
import type { ProviderUploadTask, UploadOptions } from "../types/provider";
Expand Down Expand Up @@ -69,6 +73,50 @@ export class StorageManager {
return await this.provider.getDownloadURL(path);
}

/**
* Streams the object at `path` without buffering it in memory.
*
* Progress is reported as the returned stream is consumed.
*/
async downloadStream(
path: string,
options: DownloadStreamOptions = {}
): Promise<DownloadStreamResult> {
const [downloadURL, metadata] = await Promise.all([
this.provider.getDownloadURL(path),
this.provider.getMetadata(path),
]);
const response = await fetch(downloadURL, { signal: options.signal });

if (!response.ok) {
throw new Error(
`Download failed with ${response.status} ${response.statusText}`.trim()
);
}
if (!response.body) {
throw new Error("Download failed because the response body is empty");
}

let loaded = 0;
const stream = response.body.pipeThrough(
new TransformStream<Uint8Array, Uint8Array>({
transform: (chunk, controller) => {
loaded += chunk.byteLength;
options.onProgress?.(loaded, metadata.size);
controller.enqueue(chunk);
},
})
);
const contentType =
response.headers.get("content-type") ?? metadata.contentType;

return {
stream,
totalBytes: metadata.size,
...(contentType === undefined ? {} : { contentType }),
};
}

/** Deletes the object at `path`. */
async delete(path: string): Promise<void> {
await this.provider.delete(path);
Expand Down
1 change: 1 addition & 0 deletions packages/firebase-storage-kit/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export type { ValidationErrorCode } from "./core/validation";
export * from "./core/upload-handle";
export { StorageManager } from "./firebase-storage-manager";

export type * from "./types/download";
export type * from "./types/list";
export type * from "./types/metadata";
export type * from "./types/provider";
Expand Down
15 changes: 15 additions & 0 deletions packages/firebase-storage-kit/src/types/download.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
export interface DownloadStreamOptions {
/** Called as chunks are consumed from the returned stream. */
onProgress?: (loaded: number, total: number) => void;
/** Cancels the request and stream when aborted. */
signal?: AbortSignal;
}

export interface DownloadStreamResult {
/** The object's MIME type when available. */
contentType?: string;
/** The streamed object bytes. */
stream: ReadableStream<Uint8Array>;
/** Total object size in bytes. */
totalBytes: number;
}
115 changes: 114 additions & 1 deletion packages/firebase-storage-kit/tests/storage-manager.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { describe, expect, it, mock } from "bun:test";
import { describe, expect, it, mock, spyOn } from "bun:test";

import { StorageManager } from "../src/core/storage-manager";
import {
Expand Down Expand Up @@ -134,6 +134,119 @@ describe("StorageManager", () => {
});
});

describe("downloadStream", () => {
it("streams bytes and reports cumulative progress as they are consumed", async () => {
const { provider, spies } = createMockProvider({
getDownloadURL: async () => {
await Promise.resolve();
return "https://cdn.example/large-video.mp4";
},
getMetadata: async (path) => {
await Promise.resolve();
return {
contentType: "video/mp4",
createdAt: new Date("2024-01-01T00:00:00Z"),
path,
size: 11,
updatedAt: new Date("2024-01-02T00:00:00Z"),
};
},
});
const encoder = new TextEncoder();
const responseBody = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(encoder.encode("hello "));
controller.enqueue(encoder.encode("world"));
controller.close();
},
});
const fetchMock = spyOn(globalThis, "fetch").mockResolvedValue(
new Response(responseBody, {
headers: { "content-type": "video/mp4" },
})
);
const manager = new StorageManager(provider);
const progress: [number, number][] = [];
const abortController = new AbortController();

try {
const result = await manager.downloadStream("videos/large-video.mp4", {
onProgress: (loaded, total) => {
progress.push([loaded, total]);
},
signal: abortController.signal,
});

expect(progress).toEqual([]);
expect(result.totalBytes).toBe(11);
expect(result.contentType).toBe("video/mp4");
expect(
new TextDecoder().decode(
await new Response(result.stream).arrayBuffer()
)
).toBe("hello world");
expect(progress).toEqual([
[6, 11],
[11, 11],
]);
expect(spies.getDownloadURL).toHaveBeenCalledWith(
"videos/large-video.mp4"
);
expect(spies.getMetadata).toHaveBeenCalledWith(
"videos/large-video.mp4"
);
expect(fetchMock).toHaveBeenCalledWith(
"https://cdn.example/large-video.mp4",
{ signal: abortController.signal }
);
} finally {
fetchMock.mockRestore();
}
});

it("rejects unsuccessful download responses", async () => {
const { provider } = createMockProvider();
const fetchMock = spyOn(globalThis, "fetch").mockResolvedValue(
new Response("Forbidden", { status: 403, statusText: "Forbidden" })
);
const manager = new StorageManager(provider);

try {
let caught: unknown;
try {
await manager.downloadStream("private/report.pdf");
} catch (error) {
caught = error;
}
expect(caught).toEqual(new Error("Download failed with 403 Forbidden"));
} finally {
fetchMock.mockRestore();
}
});

it("rejects successful responses without a body", async () => {
const { provider } = createMockProvider();
const fetchMock = spyOn(globalThis, "fetch").mockResolvedValue(
new Response(null)
);
const manager = new StorageManager(provider);

try {
let caught: unknown;
try {
await manager.downloadStream("empty/object");
} catch (error) {
caught = error;
}
expect(caught).toEqual(
new Error("Download failed because the response body is empty")
);
} finally {
fetchMock.mockRestore();
}
});
});

describe("query delegation", () => {
it("delegates exists to the provider", async () => {
const { provider, spies } = createMockProvider({
Expand Down