Node.js Design Patterns: Master a series of patterns and techniques to create modular, scalable, and efficient applications

by Mario Casciaro

People realize that using JavaScript on the server is not as bad as it is in the browser, and they will soon start to love it for its pragmatism and for its hybrid nature, half way between object-oriented and functional programming.

The second revolutionizing factor is its single-threaded, asynchronous architecture.

In Node.js, one of the most evangelized principles is to design small modules, not only in terms of code size, but most importantly in terms of scope.

I/O is definitely the slowest among the fundamental operations of a computer. Accessing the RAM is in the order of nanoseconds (10e-9 seconds), while accessing data on the disk or the network is in the order of milliseconds (10e-3 seconds). For the bandwidth, it is the same story; RAM has a transfer rate consistently in the order of GB/s, while disk and network varies from MB/s to, optimistically, GB/s. I/O is usually not expensive in terms of CPU, but it adds a delay between the moment the request is sent and the moment the operation completes.

We can now introduce the reactor pattern, which is a specialization of the algorithm presented in the previous section. The main idea behind it is to have a handler (which in Node.js is represented by a callback function) associated with each I/O operation, which will be invoked as soon as an event is produced and processed by the event loop.

All these inconsistencies across and within the different operating systems required a higher-level abstraction to be built for the Event Demultiplexer. This is exactly why the Node.js core team created a C library called libuv, with the objective to make Node.js compatible with all the major platforms and normalize the non-blocking behavior of the different types of resource; libuv today represents the low-level I/O engine of Node.js. Besides abstracting the underlying system calls, libuv also implements the reactor pattern, thus providing an API for creating event loops, managing the event queue, running asynchronous I/O operations, and queuing other types of tasks.

Pattern: prefer the direct style for purely synchronous functions.

As you can see from the signature of the preceding function, the callback is always put in last position, even in the presence of optional arguments. The motivation for this convention is that the function call is more readable in case the callback is defined in place.

Pattern (substack): expose the main functionality of a module by exporting only one function. Use the exported function as namespace to expose any auxiliary functionality.

but since this pattern can be useful and safe under some circumstances (for example, for testing) and is sometimes used in the wild, it is worth to know and understand it. So, we said a module can modify other modules or objects in the global scope. Well, this is called monkey patching, which generally refers to the practice of modifying the existing objects at runtime to change or extend their behavior or to apply temporary fixes.

Pattern (observer): defines an object (called subject), which can notify a set of observers (or listeners), when a change in its state happens. The main difference from the callback pattern is that the subject can actually notify multiple observers, while a traditional continuation-passing style callback will usually propagate its result to only one listener, the callback.

This is a pretty common pattern in the Node.js ecosystem, for example, the Server object of the core http module defines methods such as listen(), close(), setTimeout(), and internally it also inherits from the EventEmitter function, thus allowing it to produce events, such as request, when a new request is received, or connection, when a new connection is established, or closed, when the server is closed. Other notable examples of objects extending the EventEmitter are Node.js streams.

callbacks should be used when a result must be returned in an asynchronous way; events should instead be used when there is a need to communicate that something has just happened.

As we can see, the practice of exposing a simple, clean, and minimal entry point while still providing more advanced or less important features with secondary means is quite common in Node.js, and combining EventEmitter with traditional callbacks is one of the ways to achieve that.

Pattern: create a function that accepts a callback and returns an EventEmitter, thus providing a simple and clear entry point for the main functionality, while emitting more fine-grained events using the EventEmitter.

Pattern (unlimited parallel execution): run a set of asynchronous tasks in parallel by spawning them all at once, and then wait for all of them to complete by counting the number of times their callbacks are invoked.

In very simple terms, promises are an abstraction that allow an asynchronous function to return an object called a promise, which represents the eventual result of the operation. In the promises jargon, we say that a promise is pending when the asynchronous operation is not yet complete, it's fulfilled when the operation successfully completes, and rejected when the operation terminates with an error. Once a promise is either fulfilled or rejected, it's considered settled.

A thenable is a promise-like object with a then() method. This term is used to indicate a promise that is foreign to the particular promise implementation in use. This feature allows us to build chains of promises, allowing easy aggregation and arrangement of asynchronous operations in several configurations. Also, if we don't specify an onFulfilled() or onRejected() handler, the fulfillment value or rejection reasons are automatically forwarded to the next promise in the chain.

Pattern (sequential iteration with promises): dynamically builds a chain of promises using a loop.

Clearly, the main advantage of this approach is reusability, but as we can see from the code we presented so far, streams also enable cleaner and more modular code. For these reasons, streams are often used not just to deal with pure I/O, but also as a means to simplify and modularize the code.

A Duplex stream is a stream that is both Readable and Writable. It is useful when we want to describe an entity that is both a data source and a data destination, as for example, network sockets. Duplex streams inherit the methods of both stream.Readable and stream.Writable, so this is nothing new to us. This means that we can read() or write() data, or listen for both the readable and drain events.

The Transform streams are a special kind of Duplex stream that are specifically designed to handle data transformations.

By default, streams will handle data in a sequence, for example, a _transform() function of a Transform stream will never be invoked again with the next chunk of data, until the previous invocation completes by executing callback(). This is an important property of streams, crucial for processing each chunk in the right order, but it can also be exploited to turn streams into an elegant alternative to the traditional control flow patterns.

Pattern: Use a stream, or combination of streams, to easily iterate over a set of asynchronous tasks in sequence.

Applying this set of object-oriented design patterns in JavaScript is not as linear and formal as it would happen in a class-based object-oriented language. As we know, JavaScript is multi-paradigm, object-oriented, and prototype-based, and has dynamic typing; it treats functions as first class citizens, and allows functional programming styles. These characteristics make JavaScript a very versatile language, which gives tremendous power to the developer, but at the same time, it causes a fragmentation of programming styles, conventions, techniques, and ultimately the patterns of its ecosystem. There are so many ways to achieve the same result using JavaScript that everybody has their own opinion on what the best way is to approach a problem.

Our factory also allows us to not expose the constructors of the objects it creates, and prevents them from being extended or modified (remember the principle of small surface area?). In Node.js, this can be achieved by exporting only the factory, while keeping each constructor private.

Encapsulation refers to the technique of controlling the access to some internal details of an object by preventing the external code from manipulating them directly. The interaction with the object happens only through its public interface, isolating the external code from the changes in the implementation details of the object. This practice is also referred to as information hiding. Encapsulation is also a fundamental principle of object-oriented design, together with inheritance, polymorphism, and abstraction.

A proxy is useful in several circumstances, for example, consider the following ones: Data validation: The proxy validates the input before forwarding it to the subject Security: The proxy verifies that the client is authorized to perform the operation and it passes the request to the subject only if the outcome of the check is positive Caching: The proxy keeps an internal cache so that the operations are executed on the subject only if the data is not yet present in the cache Lazy initialization: If the creation of the subject is expensive, the proxy can delay it to when it's really necessary Logging: The proxy intercepts the method invocations and the relative parameters, recoding them as they happen Remote objects: A proxy can take an object that is located remotely, and make it appear local

Decorator is a structural pattern that consists of dynamically augmenting the behavior of an existing object. It's different from classical inheritance, because the behavior is not added to all the objects of the same class but only to the instances that are explicitly decorated. Implementation-wise, it is very similar to the Proxy pattern, but instead of enhancing or modifying the behavior of the existing interface of an object, it augments it with new functionalities, as described in the following figure:

State is a variation of the Strategy pattern where the strategy changes depending on the state of the context. We have seen in the previous section how a strategy can be selected based on different variables such as user preferences, a configuration parameter, the input provided and once this selection is done, the strategy stays unchanged for the rest of the lifespan of the context. In the State pattern instead, the strategy (also called State in this circumstance) is dynamic and can change during the lifetime of the context, thus allowing its behavior to adapt depending on its internal state,

pattern can help increase the modularity and readability

The next pattern that we are going to analyze is called Template and it also has a lot in common with the Strategy pattern. Template consists of defining an abstract pseudo class that represents the skeleton of an algorithm where some of its steps are left undefined. Subclasses can then fill the gaps in the algorithm by implementing the missing steps, called template methods. The intent of this pattern is making it possible to define a family of classes that are all variations of a similar algorithm.

Today, in Node.js, the word middleware is used well beyond the boundaries of the express framework, and indicates a particular pattern whereby a set of processing units, filters, and handlers, under the form of functions are connected to form an asynchronous sequence in order to perform preprocessing and postprocessing of any kind of data. The main advantage of this pattern is flexibility; in fact, this pattern allows us to obtain a plugin infrastructure with incredibly little effort, providing an unobtrusive way for extending a system with new filters and handlers.

Another design pattern with huge importance in Node.js is Command. In its most generic definition, we can consider a command as any object that encapsulates all the information necessary to perform an action at a later time. So, instead of invoking a method or a function directly, we create an object representing the intention to perform such an invocation; it will then be the responsibility of another component to materialize the intent, transforming it into an actual action.

The typical organization of the Command pattern can be described as follows: Command: This is the object encapsulating the information necessary to invoke a method or function. Client: This creates the Command and provides it to the Invoker. Invoker: This is responsible for executing the Command on the Target. Target (or Receiver): This is the subject of the invocation. It can be a lone function or the method of an object.

We emphasized how simple patterns such as Factory can greatly improve the flexibility of our code and how with Proxy, Decorator, and Adapter we can manipulate, extend, and adapt the interface of existing objects. Strategy, State, and Template, instead, have shown us how to split a bigger algorithm into its static and variable parts, allowing us to improve the code reuse and extensibility of our components. By learning the Middleware pattern, we are now able to process our data using a simple, extensible, and elegant paradigm. Finally, the Command pattern provided us with a simple abstraction to make any operation more flexible and powerful.

The desirable scenario is to have a high cohesion and a loose coupling, which usually results in more understandable, reusable, and extensible modules.

//'db.js' module module.exports = new Database('my-app-db'); By simply exporting a new instance of our database, we can already assume that within the current package (which can easily be the entire code of our application), we are going to have only one instance of the db module. This is possible because, as we know, Node.js will cache the module after the first invocation of require(), making sure to not execute it again at any subsequent invocation, returning instead the cached instance.

But there is a caveat; the module is cached using its full path as lookup key, therefore it is guaranteed to be a Singleton only within the current package.

We start our analysis by looking at the most conventional relationship between two modules, which is the hardcoded dependency. In Node.js, this is obtained when a client module explicitly loads another module using require().

The main idea behind the dependency injection pattern is the dependencies of a component being provided as input by an external entity. Such an entity can be a client component or a global container, which centralizes the wiring of all the modules of the system. The main advantage of this approach is an improved decoupling, especially for modules depending on stateful instances. Using DI, each dependency, instead of being hardcoded into the module, is received from the outside. This means that the module can be configured to use any dependency and therefore can be reused in different contexts.

In the authentication server example, using dependency injection we were able to decouple our modules from a particular dependency instance. The result is that we can now reuse each module with minimal effort and without any change in their code. Testing a module that uses the dependency injection pattern is also greatly simplified; we can easily provide mocked dependencies and test our modules in isolation from the state of the rest of the system.

another possible solution to simplify the wiring of modules in complex architectures is to use a dependency injection container, a component exclusively responsible for instantiating and wiring all the dependencies of an application.

Another pattern with a very similar intent is Service Locator. Its core principle is to have a central registry in order to manage the components of the system and to act as a mediator whenever a module needs to load a dependency. The idea is to ask the service locator for the dependency, instead of hardcoding

Also, like dependency injection, using a service locator makes it harder to identify the relationship between the components, as they are resolved at runtime. But in addition it also makes it more difficult to know exactly what dependency a particular component is going to require. With dependency injection this is expressed in a much clearer way, by declaring the dependencies in the factory or constructor arguments. With a service locator, this is much less clear and would require a code inspection or an explicit statement in the documentation explaining what dependencies a particular component will try to load.

However, the benefits of using packages are not only limited to external plugins. In fact, one popular pattern is to build entire applications by wrapping their components into packages, as if they were internal plugins. So, instead of organizing the modules in the main package of the application, we can create a separate package for each big chunk of functionality and install it into the node_modules directory.

Pattern: use packages as a means to organize your application, not just for distributing code in combination with npm.

Inversion of Control is a very broad principle that can be applied not only to the problem of application extensibility. In fact, in more general terms it can be said that by implementing some form of IoC, instead of the custom code controlling the infrastructure, the infrastructure controls the custom code. With IoC, the various components of an application trade off their power of controlling the flow in exchange for an improved level of decoupling. This is also known as the Hollywood principle or "don't call us, we'll call you".

The idea of caching should not be new to an experienced developer, but what makes this pattern different in asynchronous programming is that it should be combined with the request batching, to be optimal. The reason is because multiple requests might run concurrently while the cache is not set, and when those requests complete, the cache will be set multiple times.