Async generators: await and yield, orthogonal

A generator function returns an iterator; an async function returns a promise. What should an async generator return — an iterator of promises, or a promise for iterations? The answer decides what done and errors mean, and it's the shape JavaScript eventually shipped as Symbol.asyncIterator and for await.

An asynchronous generator function uses both await and yield. The await term allows the function to idle until some asynchronous work has settled, and the yield allows the function to produce a value. An asynchronous generator returns a promise iterator, the output side of a stream.

Two shapes for an asynchronous next()

The source holds three titles but only learns each one (and whether there is another) after 900ms. Call next() four times.

Iterator<Promise<T>>

next() returns Iteration<Promise<T>>

done is decided synchronously, before the source knows whether a 4th value exists. The iterator had to guess false; the 4th promise can only reject. An error here means “couldn't transport one value”, not “the sequence ended”.

PromiseIterator<T>

next() returns Promise<Iteration<T>>

Both value and done arrive together, when the source knows. Rejection models abnormal termination of the whole sequence. This is the shape gtor — and Symbol.asyncIterator — chose: the output side of a stream.

async function *shakespeare(titles)

Run it: three plays, three quotes each. getQuotes takes 700ms.

Producer and consumer steps appear here.

quotes consumed

0 / 9

total length (reduce)

producer paused at yield

—

elapsed

—

A native async generator never runs ahead: it only resumes when the consumer calls next(), so a slow consumer leaves it paused at every yield and the next getQuotes starts late.

from the essay (await yield = wait for the consumer's ack)

async function *shakespeare(titles) {
  for (let title of titles) {
    var quotes = await getQuotes(title);
    for (let quote of quotes) {
      await yield quote;
    }
  }
}

var reader = shakespeare(["Hamlet", "Macbeth", "Othello"]);
reader.reduce(function (length, quote) {
  return length + quote.length;
}, 0, null, 100)
.then(function (totalLength) {
  console.log(totalLength);
});

for … of: walk a synchronous iterator

for (let a of [1, 2, 3]) {
  console.log(a);
}

// is equivalent to:
var anIterator = anIterable[Symbol.iterator]();
while (true) {
  let anIteration = anIterator.next();
  if (anIteration.done) {
    break;
  } else {
    console.log(anIteration.value);
  }
}

for … on: the same loop, one await added

for (let a on anAsyncIterable) {   // today: for await (... of ...)
  console.log(a);
}

// is equivalent to:
var anAsyncIterator = anAsyncIterable[Symbol.asyncIterator]();
while (true) {
  let anIteration = await anAsyncIterator.next();
  if (anIteration.done) {
    break;
  } else {
    console.log(anIteration.value);
  }
}

copy / pipe in terms of forEach

Stream.prototype.copy = function (stream) {
  return this.forEach(stream.yield)
    .then(stream.return, stream.throw);
};

// iterator.copy(generator) — or stream.pipe(stream) —
// forwards every value; the promise yield returns
// pushes back on forEach, and so on the source.

Look for the single await in the desugared for … on: that is the entire difference between consuming a collection in space and one in time. Because await accepts plain values too, the loop works on synchronous iterables as well.

Jafar Husain's proposal had asyncIterator.observe(asyncGenerator); in this framework that is just copy, built from forEach, which is built from next — the same layering as for a synchronous iterator.

why it mattersPromise<Iteration<T>> lets the source say “done” when it actually knows, and lets a rejection mean the sequence ended abnormally rather than one value went missing.
why it mattersA native async generator is pull-only: with a slow consumer it sits paused at every yield. Switch to the buffer to see the producer fetch ahead while the consumer works.
why it mattersKeeping await and yield separate lets you choose: await the ack to respect pressure, or yield without waiting and let the buffer grow.