Callback based, simplified async/await

Hi, I hope I'm in the right place.

I was looking at ways to structure asynchronous code. I like what async/await does but it still requires knowing a lot about promises to wield properly. I dug into async/await implementations and found that the promises seemed unnecessary and kinda superfluous. It's fairly easy to create something simpler with plain old callbacks: https://github.com/bessiambre/casync (there is also the somewhat similar https://github.com/tj/co, but this one seems to have evolved into something quite complex).

Now without language level syntax sugar, it doesn't look as simple as it could and it makes it difficult to write generator functions in the same style since using yield to 'await' makes it confusing to use it to yield a value also.

I would find a simplified, callback based async/await (sync/wait?) very useful when working with callback based code.

As a library it looks like:

const {casync} = require('casync');
 
let addTitleToReadme=casync(function*(prepend,done,next){
    let data = yield fs.readFile('README.md',next);
    return prepend+"\n"+data;
});

Then from another casync function:

let readmeWithTitle=yield addTitleToReadme("#This is a Good Title",next);
console.log(readmeWithTitle);

Or from outside a casync function:

addTitleToReadme("#This is a Good Title",(err,readmeWithTitle)=>{
    console.log(err || readmeWithTitle);
});

You could syntax sugar away the next and maybe the done callbacks to get something like:

sync function addTitleToReadme(prepend){
    let data = wait fs.readFile('README.md');
    return prepend+"\n"+data;
});

This would provide straightforward asynchronous code without extra state machines or caching layers (without promises), with better encapsulation (no promises to be passed around) and with better performance. It would allow you to program in normal direct style all the time just like with synchronous code.

With the generator based implementation, it 's fairly easy to do proper error handling as described in the github readme.

Maybe this has been considered before?

Your benchmark introduces an extra tick for the promise test case. With the code below I observe that the promises test case runs about 7x faster than the nextTick test case. This is because nextTick thrashes between C++ and JS boundaries in Node.js, not because callbacks are slow (they're just function calls, after all). In reality, promises and callbacks are used when handling I/O, and the speed between scheduling a callback to be called and resolving a promise (which is scheduling a callback to be called) is more or less identical.

Performance aside, promises have a lot of benefits compared to raw callbacks. They ensure that the handlers cannot be called sooner than the next tick, preventing timing bugs. They can be composed into logical chains without halting surrounding execution (chaining .then). They make error handling less complex, again due to chaining.

const { performance } = require('perf_hooks');

const CYCLES = 100000;

async function testPromise() {
  for (let i = 0; i < CYCLES; i += 1) {
    await Promise.resolve();
  }
}

function* testCallback(next) {
  for (let i = 0; i < CYCLES; i += 1) {
    yield process.nextTick(next);
    // if we change this to
    // yield %EnqueueMicrotask(next);
    // we can get within 5% perf of promises but it is still slower
    // because %EnqueueMicrotask can not be optimized the same way Promise.resolve() can.
  }
}

(async () => {
  {
    const start = performance.now();
    await testPromise();
    console.log(performance.now() - start);
  }

  {
    const start = performance.now();
    await new Promise((resolve) => {
      const it = testCallback((err, v) => {
        if (err) {
          it.throw(err);
        } else {
          next(v);
        }
      });

      const next = (v) => {
        const { done } = it.next(v);
        if (done) {
          resolve();
        }
      };

      next();
    });
    console.log(performance.now() - start);
  }
})();

Thanks for your tweaked benchmarks. I see that's it's not always faster. However I still like the simplicity.

This can be handled similarly without promises and in fact is handled by casync right now.

What would be wrong with doing the following:?

try{
  wait a();
  wait b();
  wait c();
}catch(err){

}

This also works in casync right now.

I was thinking more like:

await Promise.all([
  x()
    .then(foo)
    .then(bar),
  y()
    .then(baz)
    .then(qux),
]);

This would look like:

wait async.parallel([
    sync function() {
        wait x();
        wait foo();
        wait bar();
    },
    sync function() {
        wait y();
        wait baz();
        wait qux();
    }
]);

(Using the async library)

You don't have to understand distinctions between passing onFulfilled , onRejected handlers, passing promise returning functions vs ones returning values or other complexities of then chaining. You're always just waiting on functions that implicitly take callbacks to be called when they're done.

Currently without the syntax sugar using casync it would be:

yield async.parallel([
        casync(function*(done,next) {
            yield x(next);
            yield foo(next);
            yield bar(next);
        }),
        casync(function*(done,next) {
            yield y(next);
            yield baz(next);
            yield qux(next);
        })
    ],next);

What do you guys think? Wouldn't this simplify asynchronous programing? Is there something I can do to help move this forward?

No, I don't think this would simplify anything. There is no real advantage over using promises and async/await. Understanding the difference between the two syntax styles that ostensibly would do do the same rather make everything more complicated and inconvenient.
You seem to suggest that removing the state machine and value encapsulation would be beneficial - I see only disadvantages in dropping these features and introducing an incompatible alternative. If you think there's a performance limitation, I'd rather allow better compiler optimisations for async/await code.

The extra performance is just a cherry on top. This is not be the main benefit. The main benefit would be staying closer to direct style programing with the simple addition of the possibility of waiting on asyncronous operations. No creating stateful objects. No having to understand then chaining or "dynamic replacement" of promises. No having to understand 'Pending', 'Fulfilled', 'Rejected', 'Settled' states. No deciding between passing rejection handlers or a callback to a catch function. Less talk about whether you should return a promise or call resolve or reject or throw or return a value.

The dual syntax issue would likely only be a temporary transitional thing. In most cases, there would no longer be a reason for using promises/async/await anymore.

There are also other concepts that you get to sidestep (many examples described in this chapter on promises): “thenable”, “future value”, “uninversion of control”, “revealing constructor” as well as other complexities (it's a pretty long chapter). This stuff is difficult for people learning javascript.

I don't see any benefit in staying closer to callback programming. You always have to pass around functions accepting other functions. Your proposed wait syntax only works on call expressions, into which it injects an invisible callback.

Promise-based async/await syntax already is as simple as it gets. You don't have to understand promises or their states (although it definitely helps), you don't need to know how to use .then() and/or .catch(), you don't need to know how to construct a promise yourself if you are always working with promise-returning APIs.

While I agree that async/await improves the situation significantly compared to pure promises, it does so mainly by constraining the use of promises into having the continuation immediately provided at the location where the asynchronous code is called. It basically coerces promises into something closer to the pure callback case so you have less worries about the convoluted stuff happening underneath and are prevented from attaching the continuation somewhere else later and writing byzantine execution flows.

But then why keep the promise at all? Why not provide a clean solution without all that baggage? I don't buy the idea of saying that people don't need to understand, that they should just use promise returning APIs. I think often times, people do need to understand, they do need to be able to write the APIs, that it's useful for languages to be easy to understand and for the explanations of how they work to be straightforward.

only works on call expressions

I think this is a benefit. It would likely make the implementation simpler, cleaner and more functional. No need to support all variations of statements, the keyword would only apply to function calls.

Because promises are so immensely useful. They might be not as simple as plain callbacks, but they're much cleaner, and harder to get wrong.

  • They guarantee there is only a single completion value, either a result or an error. Not both, not multiple times.
  • They guarantee that the completion is actually asynchronous. Not calling you back before you're ready.
  • They are tangible values that can be passed around, stored in variables, or returned from functions.

Especially the last point is not just baggage, but an important advantage. It is what allows you to write combinators like Promise.all that are super simple to use, e.g. in

await Promise.all([getJson("one"), getJson("two")]);

It is not possible to write this as concisely with callbacks - you need

wait async.parallel([cb => getJson("one", cb), cb => getJson("two", cb)])

or

wait async.parallel([sync () => wait getJson("one"), sync () => wait getJson("two")])

for that. It allows standard refactorings, where const value = await getJson("one") is equivalent to const promise = getJson("one"); const value = await promise. (And also it allows for multiple subscribers and more elaborate execution flows, but that's not the main point).

I'm not saying that people shouldn't understand how promises work - I'm all for people learning about promises. I'm saying that they can easily use async/await without needing to understand what promises are, just like they can easily use your proposed sync/wait syntax without needing to understand what callbacks are. There's not much difference in their ease of use. People write promise-based APIs all the time without caring for the promise constructor, resolver functions, or the parameters of the .then() method.

The devil is in the details. If I was using the proposed sync functions, I would worry what happens if the API I am using is calling the callback multiple times. I would wonder what happens if I call a sync function but don't pass a callback. I would wonder what happens if I call a callback synchronously.
Promises do answer all these questions, and make coding easier.

I think the concerns about all/parallel syntax are fairly superficial but for completeness, using the async library you could also do:

wait async.parallel([async.apply(getJson,"one"), async.apply(getJson,"two")]);

The other guarantees, you could all have with sync/wait.

The casync module already throws if you don't provide a callback at the right position when you call a casync'ed function. It throws if you call done multiple times (though this would not be an issue with a syntax sugared version where you wouldn't have access to done). It also automatically adds a nextTick if you call next before an asynchronous operation.

The one check I didn't think of adding when I wrote the first version a few weeks ago, is the check for multiple calls to the next callback passed to other asynchronous functions. This can't happen for calls to other casync'ed functions because of the check for repeated calls to done but it might be a rare issue if you call other buggy asynchronous code. However, this too would be an easy fix and I did consider implementing it in casync. I haven't yet because I couldn't think of a way without breaking backwards compatibility. I think the syntax would have to change to yield fn(next()) instead of yield fn(next) (next() would just return a wrapped version of next that you can only call once). This wouldn't be visible in a syntax sugared version. The other reason I didn't implement it yet is that I wasn't sure whether repeated calls to the callback should be ignored or if an error should be thrown. My intuition is that an error would be more useful however, with promises, I believe repeated calls to resolve are silently ignored. There might be a good reason for this behavior that I'm not aware of.

They are tangible values that can be passed around, stored in variables, or returned from functions.

To me this is a downside that adds too much state, overhead, corner cases and promotes convoluted flows of execution. This pattern should only be used in exceptional circumstances. At least the await keyword prevents it.

just like they can easily use your proposed sync / wait syntax without needing to understand what callbacks are

Callbacks are a very basic fundamental construct used everywhere in javascript. You couldn't use map or sort or setTimeout without them. I don't think you can avoid learning about them. I think it would be easy to avoid having to learn about the complexities of promises.

I just want to mention that I've been using casync for a few months now and, except for the lack of language level tooling, it's quite nice to work with (the debugger going into casync code is a slight downside).

Whenever I encounter JS beginners that attempted to run asynchronous operations one after the other for the first time and invariably ran all of them concurrently in a loop, I usually mention async/await, the discussion veers towards promises and then I have to mention the state machine and the caching layer for errors and return values, pending, fulfilled, rejected, settled states, reject/resolve callbacks and how it's all tied with 'then' chaining, 'dynamic replacement' etc. by which point they usually seem to want to go to back to the language they're coming from.

I really yearn for a world where I could just say: Here, with this keyword, the function will pause until the callback is called. Just put your function in a normal loop and 'wait' it.

It would be so clean, stateless, and functional.