Async iterables aren't necessarily a great solution either because of the exact same promise and stack switching overhead - it can be huge compared to sync iterables.

If you're dealing with small objects at the production side, like individual tag names, attributes, bindings, etc. during SSR., the natural thing to do is to just write() each string. But then you see that performance is terrible compared to sync iterables, and you face a choice:

  1. Buffer to produce larger chunks and less stack switching. This is the exact same thing you need to do with Streams. or

  2. Use sync iterables and forgo being able to support async components.

The article proposes sync streams to get around this some, but the problem is that in any traversal of data where some of the data might trigger an async operation, you don't necessarily know ahead of time if you need a sync or async stream or not. It's when you hit an async component that you need it. What you really want is a way for only the data that needs it to be async.

We faced this problem in Lit-SSR and our solution was to move to sync iterables that can contain thunks. If the producer needs to do something async it sends a thunk, and if the consumer receives a thunk it must call and await the thunk before getting the next value. If the consumer doesn't even support async values (like in a sync renderToString() context) then it can throw if it receives one.

This produced a 12-18x speedup in SSR benchmarks over components extracted from a real-world website.

I don't think a Streams API could adopt such a fragile contract (ie, you call next() too soon it will break), but having some kind of way where a consumer can pull as many values as possible in one microtask and then await only if an async value is encountered would be really valuable, IMO. Something like `write()` and `writeAsync()`.

The sad thing here is that generators are really the right shape for a lot of these streaming APIs that work over tree-like data, but generators are far too slow.

▲ conartist6 6 hours ago | parent | next [-]

Yeah that problem you have is pretty much what I'm offering a solution to. It's the same thing you're already doing but more robust.

Also I'm curious why you say that generators are far too slow. Were you using async generators perhaps? Here's what I cooked up using sync generators: https://github.com/bablr-lang/stream-iterator/blob/trunk/lib...

This is the magic bit:

  return step.value.then((value) => {
    return this.next(value);
  });

	▲	conartist6 5 hours ago \| parent [-]
		You know now that I look at it I do think I need to change this code to defend better against multiple eager calls to `next()` when one of them returns a promise. With async generators there's a queue built in but since I'm using sync generators I need to build that defense myself before this solution is sound in the face of next();next(). That shouldn't be too hard though.

▲ jauntywundrkind 6 hours ago | parent | prev [-]

I liked conartist6's proposal,

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

Where T=Uint8Array. Sync where possible, async where not.

Engineers had a collective freak out panic back in 2013 over Do not unleash Zalgo, a worry about using callbacks with different activation patterns. Theres wisdom there, for callbacks especially; it's confusing if sometime the callback fires right away, sometimes is in fact async. https://blog.izs.me/2013/08/designing-apis-for-asynchrony/

And this sort of narrow specific control has been with us since. It's generally not cool to use MaybeAsync<T> = T | Promise<T>, for similar "it's better to be uniform" reasons. We've been so afraid of Zalgo for so long now.

That fear just seems so overblown and it feels like it hurts us so much that we can't do nice fast things. And go async when we need to.

Regarding the pulling multiple, it really depends doesn't it? It wouldn't be hard to make a utility function that lets you pull as many as you want queueing deferrables, allowing one at a time to flow. But I suspect at least some stream sources would be just fine yielding multiple results without waiting. They can internally wait for the previous promise, use that as a cursor.

I wasn't aware that generators were far too slow. It feels like we are using the main bit of the generator interface here, which is good enough.

▲

conartist6 5 hours ago | parent [-]

Yeah I think people took away "It's better to be uniform" since they were trying to block out the memory of much-feared Zalgo, but if you read the article carefully it says in big letters "Avoid Synthetic Deferrals" then goes on to advocate for patterns exactly like MaybeAsync to be used "if the result is usually available right now, and performance matters a lot".

I was so sick of being slapped around by LJHarb who claimed to me again and again that TC39 was honoring the Zalgo post (by slapping synthetic deferrals on everything) that I actually got Isaacs to join the forum and set him straight: https://es.discourse.group/t/for-await-of/2452/5

	▲	spankalee 4 hours ago \| parent [-]
		That's an amazing thread, thanks for posting it! I've wanted `for await?()` for exactly these situations. I feel like my deep dives into iterator performance are somewhat wasted because I might have made my project faster, but it's borderline dark magic and doesn't scale to the rest of the ecosystem because the language is broken.