Problems with structuring async code – callback hell

In the previous section, we introduced the callback pattern as a way to deal with asynchronous calls. The pattern offers a structured way of dealing with such calls, in that we can always know what to expect in its method signature; the error is the first parameter, the second parameter is the response, and so on. But the pattern does have its shortcomings. The shortcomings might not be obvious at first, because you might only call the code like this:

openFile('filename', (err, content) => {
  console.log( content );
  statement4;
  statement5;
})

statement2;
statement3

What we see here is how we invoke the method openFile(). Once that runs to completion, the callback is called and, inside of the callback, we continue invoking statement4 and statement5.

This looks okay in the sense that it is still readable. The problem arises when you need to do several async calls one after another and those calls are dependent upon each other. It might be that you first need to log in to a system and then fetch the other data, or it might mean that you need to make a call as to whose data needs to be used as input for the next call, like in this example:

getData('url', (err, data) => {
  getMoreData('newurl/'+ data.id, (moreData) => {
    getEvenMoreData('moreurl/'+ moreData.id, () => {
      console.log('done here');
    })
  })
})

The anti-pattern we see emerging here is that of tabulation and lost readability. For each call we make, we see the code is tabbed one step in; it's nested. When we have three calls like this, we can see that the code doesn't look very nice; it is readable, but not very pleasing to the eye. Another drawback is it's also technically hard to get right, in that we might struggle to place the parentheses and curly brackets in the correct place. Throw a few if...else clauses in there and you will have a hard time matching all the symbols.

There are several ways you can address this problem:

• Keep the code shallow and use named functions over anonymous ones
• Reduce the cognitive load and move functions into their own modules
• Use more advanced constructs, such as promises, generators, and async functions from ES7 and other async libraries

Keeping the code shallow is about giving our anonymous functions a dedicated name and breaking them out into their own functions; this will make our code look like this:

function getEvenMoreDataCallback(err, evenMoreData) {
  console.log('done here');
}

function getMoreDataCallback(err, moreData){
  getEvenMoreData('moreurl/'+ moreData.id, getEvenMoreDataCallback);
}

function getDataCallback(err, data){
  getMoreData('newurl/'+ data.id, getMoreDataCallback);
}

getData('url', getDataCallback)

This clearly flattens out the code and makes it more easier to read. It also removes the need to match curly brackets correctly as the functions are only one level deep.

This gets the code part out of the way, but there is still a cognitive load as we have to process three function definitions and one function call. We can move them out to their own dedicated modules, like this:

let getDataCallback  = require('./datacallback');
getData('url', getDataCallback);

And for the other method, it would look like this:

function getEvenMoreDataCallback(err, evenMoreData) {
  console.log('done here');
}

And this:

var getEvenMoreDataCallback = require('./evenmorecallback');

function getMoreDataCallback(err, moreData){
  getEvenMoreData('moreurl/'+ moreData.id, getEvenMoreDataCallback);
}

Now we have removed quite a lot of the cognitive code. It may not have paid for itself in this case, as the methods were not that long, but imagine the methods spanned 30 or 40 lines in size; putting them in a separate module would have made a lot more sense. 

The third option is to deal with this kind of code using more advanced constructs. We will address these in the upcoming sections.