Requests and response—Chain of Responsibility — global error handling and logging — you don’t know what you don’t know

Express is a popular backend framework for Node applications. It’s often one of the earliest frameworks new JavaScript developers will learn to create RESTful APIs. And why not? It’s a powerful tool that seriously cuts down on the time needed to create a concise, scalable API for your site. It’s worth using Express just for its documentation, an area the Node does not seem to prioritize.

However, although Express provides a lot of great features and makes API setup, testing, and debugging a breeze, it also takes pride in being un-opinionated — you can do whatever you want in Express. That also means you can fail spectacularly. All the great features that Express brings can be rendered useless with scrambled, improperly documented, and unreadable code. If you are newer to writing RESTful APIs with Express — you’ve come to the right place. This article will cover some of the native features of Express that will help you keep your codebase clean.

Requests and Responses

The fundamental building blocks of communication on the web are Requests and Responses. In Express, these are represented by the req and res objects that are declaratively passed into each of your controllers. Most beginners will be familiar with just a few of the properties of each of these objects — such as cookies, query, params, or body on the req object. However, there is a wealth of useful information on each object. Newcomers to Express need to understand what is available natively on req and res so they don’t end up over-engineering solutions to simple problems.

Get to know the request object

I’ve seen a lack of knowledge of the fundamentals of the native req object lead to confused engineers and baffling code. As new developers move from writing simple CRUD applications for a portfolio into more complex, production-level backends with multiple architectural pieces, there is a lot of opportunity to utilize more properties of the req object. However, it can be easy to stick to the basics without cultivating an inner curiosity.

For example, if you are setting up a logging system for your application, it might be worthwhile to look into what req tracks as it moves through the application. For example, you can access req.method which returns the request method (GET, POST, DELETE, and so on). You can access req.originalUrl which contains the full endpoint for the request. You can alternatively find thereq.baseUrl or req.path. If the originalUrl was /teachers/new?grade=3, the baseUrl would be /teachers and the path would be /new. You might already see how some properties could help you set up conditional error logging or even log response times by endpoint.

You can also access information about the client that initiated the request, provided you have set the value of trust proxy to a truthy expression.

app.enable('trust proxy')

// or - direct from the Express documentation

app.set('trust proxy', (ip) => {
  if (ip === '127.0.0.1' || ip === '123.123.123.123') return true // trusted IPs
  else return false
})

You can detect the client’s IP address via req.ip or req.ips if one or more proxies separate the server and the client. That information can be invaluable for setting up security features like rate-limiting.

Know your response object

Knowing the res object is equally as important. To start with the basics, you must utilize the locals property to maintain variables within a request-response cycle. Each res is prepopulated with an empty object at the locals key. In each middleware in your function, you can extract, add, and manipulate data on the locals object. The example below is pulled directly from the Express documentation.

app.use(function (req, res, next) {
  // Make `user` and `authenticated` available in templates
  res.locals.user = req.user
  res.locals.authenticated = !req.user.anonymous
  next()
})

In the example above, user and authenticated will now be accessible from the res.locals object in any following middleware. I have personally developed Express APIs without the knowledge res.locals. The unfortunate consequence is inevitably the “Big Ball of Mud” anti-pattern, especially as your application’s needs grow more complex. If you can’t chain middleware together, you lose much of the benefit of working with Express in the first place.

The res object also offers built-in cookie handling with the res.cookie and res.clearCookie methods. Unless you have extremely specific needs, there is no reason to import additional third-party libraries to handle cookies and increase the node_modules size unnecessarily. Don’t re-engineer or import what is already available to you.

Finally, get to know the different ways you can send a response with Express. You can res.redirect the client, res.sendFile to a client, res.send a response body. You can even res.render an HTML view for a client, use res.json or res.jsonp to convert the response body to JSON or JSONP, or simply res.end the response.

Don’t leave the client hanging

In Node and Express APIs, there is no default response. If you fail to handle response logic or catch errors accurately, you could leave the client waiting long enough that it stops listening for the response. This is such a widespread problem with Node servers that Heroku (which has a blanket timeout of 30 seconds) dedicated a section in its “preventing timeouts” article to Node.

It’s easy to cause a timeout in a Node/Express server. One primary way, highlighted by Heroku’s article, is forgetting to handle response logic (with end, send, json, etc.). The client connection will timeout, waiting for your response. You should always ensure each of your endpoints handles response logic.

You can also leave the client hanging by blocking the Node event loop. Node is single-threaded but offers a Worker Pool to handle I/O and other expensive tasks. The single-threaded nature of the Node event loop makes it scalable — it doesn’t have to worry about the overhead for additional threads. However, it also means that a Node developer needs to be aware of event loop-blocking tasks.

Because the event loop is single-threaded, an expensive operation can prevent current and new client connections from getting a turn to have their request processed. This can lead to longer response times and even the dreaded timeout error. Always be aware of CPU-intensive operations — examples include JSON.parse or JSON.stringify on large inputs, reading files instead of accessing memory, or even nesting O(n) native array functions to manipulate complicated data on large arrays.

Use the Chain of Responsibility Design Pattern

Unopinionated or not, if you’re using Express and aren’t following any design pattern, you’re likely following an anti-pattern. It’s essential to have a plan for architecting an API in a way that allows you to refactor, add new features, and reuse code easily.

What is the Chain of Responsibility

The Chain of Responsibility (CoR) is a design pattern where particular functionality is transformed into modular objects called handlers. These handlers can be used to process requests in a sequential chain. Each handler stores a reference to the next handler, and each handler has the opportunity to discontinue processing the request.

A perfect example is authentication. Let’s say that your site utilizes cookies, and only authenticated users with appropriate cookies can take certain actions in your API. An example could be a social networking site where you want anonymous users to be able to browse content but only authenticated users with accounts to be able to comment or publish their own content.

You could break this API into several CRUD endpoints and create a single controller that handles each method. A user could GET or POST to the /comment endpoint. Likewise, they could GET or POST to the /update endpoint. If you want to authenticate users for each POST request, you could theoretically create an endpoint for each type of POST and directly invoke an authentication helper function in each endpoint’s controller.

That’s a start — but what if the chain starts getting longer? Maybe you want a separate helper function verifying the content is appropriate. Maybe that function needs to call additional processing helper functions. Pretty soon, you’ll have calls to other functions that call other functions that call yet other functions. Each endpoint has some reusability, but the chain of logic becomes difficult to follow in the codebase as you split functionality into different helper function files, and all these files reference one another. This can easily lead to the “Big Ball of Mud” anti-pattern.

Chain of Responsibility in Express — Reuse your controllers

While there are many ways to write reusable, modular code, CoR in Express is extremely simple and produces reusable, maintainable code. You can utilize CoR in Express by breaking up your router into multiple files, each file handling similar endpoints. For example, you might declare the following router in your server:

app.use('/comment', commentRouter)

In your comment router file, you could have the following two endpoints:

import express from 'express'
import authController from 'controllers/authController'
import commentController from 'controllers/commentController'

const router = express.Router()

router.post('/', 
  authController.authenticate,
  commentController.postComment,
  (_req, res) => return res.status(200).json({
    // data to send back
  })
)

router.get('/', 
  commentController.getCommentsForUpdate,
  (_req, res) => return res.status(200).json({
    comments: res.locals.comments
  })
)

export default router

Here, the CoR is natively used in the Express router. Each controller may contain several handlers (e.g., getCommentForUpdate and postComment). Each controller is declaratively referenced in the router file. And each controller’s functionality can be used across a variety of routers. The controller decides how to handle the request independently of the other controllers, potentially relying on information stored in res.locals, and then passes on the request to the next controller in the chain by calling the next function.

You can chain as many controllers as you’d like, splitting off reusable logic into separate controllers so that they can be easily accessed on other routers. For example, you can reuse your authentication function in your /update router.

router.post('/', 
  authController.authenticate,
  updateRouter.postUpdate,
  (_req, res) => return res.status(200).send()
)

This is an extremely simple example to illustrate the native power of Express. It shows how easy it is to write endpoints in a CoR design pattern that passes a request down to multiple handlers, each handler acting independently and deciding whether to proceed or terminate processing a request.

When a controller decides to terminate processing a request

Let’s say one of your unauthenticated users circumvents your frontend logic and attempts to post a comment on another user’s update. Here, we again see the power of the CoR design pattern in Express. If you encapsulate the authentication functionality in a single handler, it can be used to handle these faulty requests gracefully. The beauty of using this design pattern is that a request with the same faulty input will fail the same way every time.

Here is a very simple example. If a user does not have cookies in your social networking application, they cannot be authenticated. You might handle this in the following way:

authenticate: (req: Request, res: Response, next: NextFunction): Promise<void> => {
  try {
    if (!req.headers.cookie) { 
      const error = new Error(`Not authorized`)
      error.statusCode = 403
      return next(error)
    }
    // check the user's cookies here
  } catch (e) {
    return next({
      log: `Error in authController.authenticate`, 
      status: 500,
      message: `An error occured.`
    })
  }
}, 

It’s important to note that when next is passed an error on invocation, Express immediately knows to halt the execution of logic in subsequent handlers and call your global error handling function. Here, you can see that the same error will be passed to the global error handling function whenever a user attempts an action and does not have cookies in their request.

Likewise, when next is called in the catch portion of a try-catch block, it will also halt the execution of subsequent handlers and invoke the global error handling function. Here, you can see the potential for capturing the location of programmatic or operational errors and logging them in one centralized global error-handling function. Each function can pass a unique name or id to the global error handler so that your logs are easy to read and bugs are easy to squash.

Set Up Global Error Handling and Logging

Once your codebase grows, even the most meticulous programmers will inevitably introduce bugs into the application. This is especially true as teams grow and split into segmented portions of a codebase — some engineers working solely in the frontend, others confined to the API or the database. It’s important to integrate error handling into every controller in your application and implement a logging system to capture errors, response times, and any other information important to your server’s integrity.

Use the Express global error handling function

If you are using Express and have not set up the global error-handling function, you are wasting a valuable native resource. The global error handler is identified by the presence of four arguments: err, req, res, and next. It might look something like this:

app.use((err: any, req: Request, res: Response, _next: NextFunction) => {
    const defaultErr = {
      log: err.log || "Express caught unknown error",
      status: err.status || 500,
      message: err.message || "Something went wrong.",
    };
    const errorObj = Object.assign({}, defaultErr, err);
    
    // perform any necessary operations on the error

    return res.status(errorObj.status).json(errorObj.message);
}

The global error handler is a one-stop-shop you can route all your errors through and a backup in case your application encounters unknown errors. It helps maintain the integrity of the CoR design pattern. It’s also the perfect place to perform logical operations on your error responses. This little function packs a big punch in a production-level application.

Protect your application from errors

No matter what your API accomplishes or how simple it is, even if you think there is no way it can break — the second your application becomes complex enough or gathers enough users, it will break in ways that seem spiteful and designed to give you a bad day.

There are two main types of errors: operational errors and programmatic errors. Programmatic errors in your application are the consequence of your own mistakes. These can be captured by encapsulating your code consistently in try/catch blocks and ensuring to call the next function concerning the unique location of the error in your codebase.

Operational errors are harder to tackle. These can stem from various issues — network outages, unexpected input, or even hardware failures. Operational errors will occur, no matter how robust your application is. Utilizing a global error-handling function and integrating a logging solution to capture the occurrence of these errors can help you identify weak points in your application.

Always, always, always protect the logic handling your client requests. Whether you use try/catch blocks and async/await or directly catch a rejected promise, you need to pass Express the value of your error so that it knows to invoke the global error handling function. Whether you provide next as the final catch handler for a function or invoke next inside your catch block, Express will be able to handle these errors.

“You Don’t Know What You Don’t Know”

My last piece of advice to newer engineers building RESTful APIs in Express (or building anything else ) is to develop your innate curiosity. The quote above, attributed to several different people, describes our lack of understanding of our own gaps in knowledge. If you don’t look at what’s out there, you will never know what you are missing. If you don’t read technical blogs, dive into the documentation, and consistently revisit and refactor your code, you may find yourself repeating the same mistakes over and over again.

Develop a passion for technical learning. Always assume that what you did today could have been done better and ruminate over the how — “How can I do this better?” Review your own code. Read about design patterns. Find a mentor. Ask them questions and challenge them, too.

Much of what this article covers is present in the Express documentation. One of the many reasons to use Express is its incredible, accessible documentation. You can find information on the global error handler, the req and res objects, and the next function. Whenever you use a library or a framework, always make a point of reading the documentation. You may be over-engineering solutions to problems already addressed in the technology.

Write Clean, Concise, and Debuggable Code in Express.js was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.