The async library
(This is intended to follow Server-side AJAX with Node.js+Express.)
In using Node.js, one inevitably runs into difficult callback
patterns. A number of simple libraries have been built to
simplify the patterns; the most popular is probably async.
By the way, additional references on async include
the following.
Official async documentation
A different tutorial
The waterfall function
One of the most common patterns involves a query that
requires several verification steps before finally modifying
a database. For example, if a student requests to enroll in a
course, we might first want to confirm that the student's password
is correct, then that the course actually exists in the
database, then that the student isn't already enrolled in the
course, before finally saving that the student is now
enrolled. The overall structure of this would be the following
(with A_sql being the password-verification query,
B_sql being the course-validation query,
C_sql being the already-subscribed query,
and D_sql being the enrollment-addition query).
A; // sequence of arbitrary code
db.all(A_sql, function (err, rows) {
if (err !== null) {
H; // error handling code
} else {
B; // more arbitrary code, working with result from A_sql
db.all(B_sql, function (err, rows) {
if (err !== null) {
H; // error handling code
} else {
C; // more arbitrary code, working with result from B_sql
db.all(C_sql, function (err, rows) {
if (err !== null) {
H; // error handling code
} else {
D; // more arbitrary code, working with result from C_sql
db.run(D_sql, function (err) {
if (err !== null) {
H; // error handling code
} else {
E; // more arbitrary code
}
});
}
});
}
});
}
});
This deep nesting is burdensome. The waterfall
function provided by async provides a way around this:
async.waterfall(
[
function (next) {
A; // sequence of arbitrary code
db.all(A_sql, next);
},
function (rows, next) {
B; // more arbitrary code, working with result from A_sql
db.all(B_sql, next);
},
function (rows, next) {
C; // more arbitrary code, working with result from B_sql
db.all(C_sql, next);
},
function (rows, next) {
D; // more arbitrary code, working with result from C_sql
db.run(D_sql, next);
},
function (next) {
E;
}
],
function (err) {
if (err !== null) {
H; // error handling code
}
}
});
The waterfall takes a list of functions as its first
argument. It calls the first function in the array, passing as
a parameter to that function next, which is essentially
a function that invokes the second function in the array.
In calling the second function, any results passed into next
are passed as arguments to the second function, as well as
another next, which
when invoked essentially invokes the third function in the array.
A complication to the waterfall is its error-handling.
The waterfall method assumes that the first argument
to any callback is an “error” result. When next
is finally called, the next function first checks
whether the first argument is null or not. If it is
non-null, then there is an error, and so waterfall
stops going through the list, and instead it goes directly to
the final function (the second argument to waterfall,
after the list of functions that forms the first argument)
passing that non-null error as the argument.
However, if the error argument to next is null,
only then does waterfall go on to calling the next
function in the list.
Admittedly, this is all rather confusing when you hear it described; but in practice, it is a lot simpler than nesting callback within callback within callback.
(By the way, async is useful beyond just database
queries: Throughout Node.js, it is entirely typical for any
blocking call to follow the same format of taking a callback
function where an error argument is passed first to the
callback. Those can easily be used by waterfall.)
Functions handling arrays
Another very useful function in async
is eachSeries, which is for doing the same action to each
element in a list, where each action is blocking.
To modify our example above, suppose that we want to allow a
student to enroll in several courses simultaneously.
It is quite a challenge to write SQL to perform an arbitrary number of
INSERT queries. But eachSeries is well-suited to
this.
async.eachSeries(courses,
function (course, next) {
db.run(A_sql, next);
},
function (err) {
if (err !== null) {
H; // error handling code
}
});
The eachSeries function takes an array as the first
argument and functions as the second and third arguments.
It calls the second-argument function once for each element of
the array, passing the array element as the first argument and
a next function as the second argument.
This function should process the array value as appropriate and
then call next once it is done — probably within
the callback to some other blocking function.
If an error is passed into next, then no more array
elements are processed, and the third-argument function is
called. This third-argument function is also called
when all array elements have been
processed successfully; in that case, null is passed as
the argument to this third-argument function.
The waterfall and eachSeries are just two of
the most useful functions in the async module.
There are many others, particularly several that work on
different types of operations with arrays:
map to create a new array of computed results based on
each element of an existing array,
some to find whether a condition applies to any element
of an array, and so on.