Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 39 additions & 21 deletions docs/worker-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,22 +153,21 @@ anti-goal above: events kept in TS memory stay visible to the collector under `g
counts under `rc` (`Shared.share`), and move under `own`, with no new runtime code. A C++ queue
(mutex and condition variable in `thread.cpp`) would be simpler to make correct, but every pointer
it holds would need extra handling per memory model (an uncollectable allocation under `gc`, an
explicit retain/release under `rc`). If the TS queue can't be made to hold objects under `-mm=own`,
the builder should fall back to the C++ queue for the locking and waiting only, still keeping the
events themselves in a TS array.
explicit retain/release under `rc`). Step 1 built it this way and it holds objects under every
model, `own` included (see "Memory models"), so the C++ fallback is not needed.

The queue is its own class (`MessageQueue<T>` or similar, not exported in the README at first) so
`MessageChannel` / `MessagePort` can be built on it later.

### Messages: handed over, not copied

`postMessage(m)` puts `m` itself in the queue. Under `-mm=own` that is expected to be a move, so the
sender can't use `m` afterwards. Not yet checked: whether a method parameter stored into a field
(the queue's array) moves or is only lent (the README says `ThreadHandle.join()` *lends* its result);
this is on the `own` find-out list below. Under `gc`, `rc` and `none` the sender *can* still reach
`postMessage(m)` puts `m` itself in the queue. Under `-mm=own` that is a move, so the sender can't
use `m` afterwards: `MessageQueue.push` keeps its argument, and using it after the push is a compile
error ("used here after its value was moved"; checked in step 1). Under `gc`, `rc` and `none` the sender *can* still reach
`m`, and the README says not to change a message after posting it (the same as passing an object
in a browser's transfer list). Under `-mm=rc` the message is `Shared.share`d before it is queued, so
its count changes atomically.
its count changes atomically. `Shared.share` marks every counted block the value reaches (the
compiler's `mlirGenSharedShare`), so a message's fields are covered too, not just the object itself.

Numbers, booleans and strings behave the same as in a browser, since they are values or immutable.
For a real copy the program can post `JSON.stringify(x)` and `JSON.parse` it on the other side; the
Expand All @@ -195,19 +194,36 @@ running.
|---|---|---|---|---|
| Thread registration | as `Thread` (registered with the collector) | as `Thread` | as `Thread` | as `Thread` |
| Messages | shared object, collector sees it in the TS queue | `Shared.share`d before queuing | shared object, never freed | moved into the queue, moved out to the event |
| Handlers capturing creator-side objects | allowed | allowed (shared) | allowed | captured values move into the worker; sharing goes through a `Shared<T>`, as with `Thread` |
| Handlers capturing creator-side objects | allowed | allowed (shared) | allowed | captured values move with the handler; a local used again after the handler is set is a compile error, so shared state goes through a `Shared<T>` |

What the `own` column can actually do is the main thing to find out first: an object as `In`/`Out`
(the README says `own` array elements are "values, strings, `any` or `Shared<T>` handles", so an
object may have to go in as a `Shared<T>`), whether `postMessage` moves its argument into the queue
or only lends it, and a handler stored in a field and called later; anything that does not work goes in the README
as a limit, as the `Thread` notes do today.
What step 1 (`MessageQueue<T>`, `tests/message_queue.ts`) settled, in release builds, compiled and
under the JIT:

- **Objects as messages work under `own`**, without `Shared<T>`: class instances go through a plain
`T[]` inside the queue and come out whole, both ways between two threads.
- **A push moves** under `own` (above).
- **Both threads reach the queue through a `const` `Shared<MessageQueue<T>>`** each function
captures, under every model, as threads sharing an `Int32Array` do.

Still open for step 2: **a handler that captures a local of the creating thread.** Under `-mm=own` a
closure stored in an object takes what it captures with it, so this is a compile error:

```ts
let total = 0;
worker.onmessage = (e) => { total += e.data; }; // the closure escapes into the Worker
print(total); // error: 'total' is captured by a closure that escapes, and is still used here
```

The creator's handler has to keep its state somewhere both can reach: a `Shared<T>` it captures, or
a field of an object the handler alone owns, or it can use `receive()` instead of `onmessage`. The
builder should pick the shape the `own` tests and README example use; anything that does not work
goes in the README as a limit, as the `Thread` notes do today.

## Order of work

1. The queue, with a test that two threads pass 10 000 numbers, strings and class instances
through it in order, under all four models. The class instances are there to settle the `own`
questions above before anything is built on the queue.
1. **Done:** `MessageQueue<T>` (`push`, `take(timeout?)`, `close`, `clear`, `closed`, `length`) in
`lib.generics.ts`, and `tests/message_queue.ts`, where two threads pass 10 000 numbers, strings
and class instances through it in order, under all four models.
2. `Worker` / `WorkerScope` with `postMessage`, `onmessage`, `receive`, `join`, `close`.
3. Errors: `onerror`, throws from `body` and from handlers.
4. `terminate()` and `dispatch(timeout)`.
Expand Down Expand Up @@ -235,11 +251,13 @@ see the comment on `Atomics.isLockFree`).

## Left to the builder

- Exact names of the queue class and its internals; whether the worker's entry reuses
`ThreadHandle` or calls `tslang_thread_spawn` directly.
- Whether the worker's entry reuses `ThreadHandle` or calls `tslang_thread_spawn` directly.
- Whether `onmessage` / `onerror` are optional fields or fields with no-op defaults (whichever the
compiler handles in a generic class; note the known crash with `cond ? <T>x : undefined` in a
generic class, use `if`/`else`).
- How `receive` returns "nothing" if `MessageEvent<Out> | undefined` doesn't compile in a generic
class under some model (a `tryReceive` returning a boolean plus an out field is fine).
- `MessageQueue.take` takes items out with `shift()`, which is O(n) in what is queued. Fine at
10 000 items; a ring buffer is the change if a worker ever falls far behind.

(`receive` can return `MessageEvent<Out> | undefined` as written: `MessageQueue.take` returns
`T | undefined` and compiles under every model.)
- The `{ name }` option can be dropped if it costs anything.
102 changes: 102 additions & 0 deletions src/generics/lib.generics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -914,6 +914,108 @@ namespace Thread {
}
}

// A one-way queue between threads: one side pushes, the other takes, in the order pushed. Both
// threads reach it through a Shared each function captures (see Thread.spawn). What is pushed is
// handed over, not copied: under -mm=own it moves in and out; elsewhere the pusher should leave it
// alone. The items stay in a TypeScript array, so the collector sees them under -mm=gc and they keep
// their counts under -mm=rc. Worker is built on it (docs/worker-design.md).
class MessageQueue<T> {
private items: T[] = [];
// [0] the lock (1 taken); [1] bumped on each push and on close, which a taker waits on;
// [2] closed (1)
private sync = new Int32Array(3);

// queues the item and wakes a taker; false, and the item is dropped, once the queue is closed
push(item: T): boolean {
// -mm=rc: the taker drops what the pusher made, so its count changes atomically from here
// (erased under the other models)
Shared.share(item);

this.lock();
if (this.sync[2] != 0) {
this.unlock();
return false;
}

this.items.push(item);
Atomics.add(this.sync, 1, 1);
this.unlock();
Atomics.notify(this.sync, 1);
return true;
}

// The next item. Waits for one up to `timeout` milliseconds (none: as long as it takes); undefined
// when the time is up, or once the queue is closed and nothing is left in it.
take(timeout?: number): T | undefined {
const forever = timeout == undefined;
const deadline = forever ? 0 : Date.now() + timeout;
while (true) {
this.lock();
if (this.items.length > 0) {
const item = this.items.shift();
this.unlock();
return item;
}

const closed = this.sync[2] != 0;
const seen = Atomics.load(this.sync, 1);
this.unlock();
if (closed) {
return undefined;
}

if (forever) {
Atomics.wait(this.sync, 1, seen);
} else {
const left = deadline - Date.now();
if (left <= 0) {
return undefined;
}

Atomics.wait(this.sync, 1, seen, left);
}
}
}

// no more pushes; a taker gets what is left, then undefined
close() {
this.lock();
this.sync[2] = 1;
Atomics.add(this.sync, 1, 1);
this.unlock();
Atomics.notify(this.sync, 1);
}

// drops every item still queued
clear() {
this.lock();
this.items.length = 0;
this.unlock();
}

get closed() {
return Atomics.load(this.sync, 2) != 0;
}

get length() {
this.lock();
const length = this.items.length;
this.unlock();
return length;
}

private lock() {
while (Atomics.compareExchange(this.sync, 0, 0, 1) != 0) {
Atomics.wait(this.sync, 0, 1);
}
}

private unlock() {
Atomics.store(this.sync, 0, 0);
Atomics.notify(this.sync, 0, 1);
}
}

// Bytes to and from the standard streams, as they are: on Windows each call puts its stream in
// binary mode and back, so a "\n" stays one byte and a 0x1A does not end the input (console.log
// and console.readLine keep text mode). A Uint8Array is not a view over a buffer here, so `read`
Expand Down
114 changes: 114 additions & 0 deletions tests/message_queue.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
// MessageQueue<T>: a one-way queue between threads, what Worker will be built on. Each side holds the
// queue through a Shared both functions capture, as threads sharing an array do. Items arrive in
// the order they were pushed; take() waits for one, or gives undefined once the queue is closed and
// empty, or when its timeout passes.

class Parcel {
constructor(public id: int, public label: string) {}
}

function numbersInOrder() {
const queue = new Shared(new MessageQueue<int>());
const producer = Thread.spawn((): int => {
for (let i = 0; i < 10000; i++) queue.value.push(i);
queue.value.close();
return 0;
});

let expected = 0;
while (true) {
const item = queue.value.take();
if (item == undefined) break;
assert(item == expected, "numbers arrive in order");
expected++;
}

producer.join();
assert(expected == 10000, "every number arrived");
}

function stringsInOrder() {
const queue = new Shared(new MessageQueue<string>());
const producer = Thread.spawn((): int => {
for (let i = 0; i < 10000; i++) queue.value.push(`item ${i}`);
queue.value.close();
return 0;
});

let count = 0;
while (true) {
const item = queue.value.take();
if (item == undefined) break;
assert(item == `item ${count}`, "strings arrive in order");
count++;
}

producer.join();
assert(count == 10000, "every string arrived");
}

// objects both ways: the main thread sends parcels, the other thread relabels each and sends it back
function objectsBothWays() {
const requests = new Shared(new MessageQueue<Parcel>());
const replies = new Shared(new MessageQueue<Parcel>());
const worker = Thread.spawn((): int => {
let handled = 0;
while (true) {
const parcel = requests.value.take();
if (parcel == undefined) break;
replies.value.push(new Parcel(parcel.id * 2, `${parcel.label} done`));
handled++;
}

replies.value.close();
return handled;
});

for (let i = 0; i < 10000; i++) requests.value.push(new Parcel(i, `parcel ${i}`));
requests.value.close();

let count = 0;
while (true) {
const reply = replies.value.take();
if (reply == undefined) break;
assert(reply.id == count * 2, "replies arrive in order");
assert(reply.label == `parcel ${count} done`, "the reply carries its label");
count++;
}

assert(worker.join() == 10000, "the worker handled every parcel");
assert(count == 10000, "every reply arrived");
}

function timeoutAndClose() {
const queue = new MessageQueue<int>();

// nothing pushed: take gives up after its timeout
assert(queue.take(20) == undefined, "take times out on an empty queue");

queue.push(1);
assert(queue.length == 1, "one item queued");
queue.close();
assert(queue.closed, "closed");

// what was pushed before close still comes out; a push after close is dropped
assert(!queue.push(2), "push after close is refused");
assert(queue.take() == 1, "an item pushed before close");
assert(queue.take() == undefined, "closed and empty");

// clear drops what is queued
const other = new MessageQueue<string>();
other.push("a");
other.push("b");
other.clear();
assert(other.length == 0, "clear empties the queue");
assert(other.take(0) == undefined, "nothing left after clear");
}

function main() {
numbersInOrder();
stringsInOrder();
objectsBothWays();
timeoutAndClose();
print("ALL DONE");
}
Loading