Build your own expressjs | Part 1

This is going to be a multipart series in which we are going to build a minimal, simple and yet powerful version of Express.js, called Minimal.js. We are going to talk about Node.js in-built modules, HTTP server, routing, middlewares, and much more.

We are learning on the go so if you find any mistake or any better way to do certain things or just want to share your feedback then I am all ears and open to collaboration. Let me know your opinions here.

Introduction

Express has become the de-facto standard framework for web server applications in Node.js. It is easy to use, has a low learning curve, exceptionally well plug & play middleware system and it's minimal by design. As its homepage says,

Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications.

In this series, we are going to build a similar (but quite simpler) web framework like Express. Our architectural decisions and API design will be as close to Express as possible. However, some implementations would be different so take it with a pinch of salt. :P

Prerequisites

1. Latest stable version of Node.js installed
2. A Basic understanding of JavaScript and Node.js runtime.

Part 1

This part would be a very simple introduction to Node.js modules, HTTP and we are going to create a basic server from scratch. If you already know all this then you can skip this one and move to part 2.

I would recommend that you should code along. So, go ahead, clone the repo and check out the start branch

git clone https://github.com/yomeshgupta/minimaljs.git
git checkout start

HTTP

This page is generated by a mix of HTML, CSS and JavaScript, sent to you by Devtools via the internet. Internet is full of pages like this and a lot of cat pictures. A LOT! 🐱 These pages are hosted on different servers all around the world. When we visit the internet and access any content, our browser must ask the servers for the content it wants and then display it to us. The content here is also known as a resource that can be of varied types such as HTML files, images, videos, scripts and many more. The protocol which governs all this communication is known as HTTP.

HTTP stands for Hypertext Transfer Protocol (HTTP). It is a protocol that is used to structure communication between client and server. The client requests the server and the server provides the apt response. It is a stateless protocol i.e. two requests to a server are mutually exclusive and the server does not keep any data between those requests.

The transfer of resources between server and client happens using TCP (Transmission Control Protocol). When you type an URL such as www.devtools.tech into your browser then you are asking it to open a TCP channel to the server that resolves to that URL. The server receives the request, processes it, sends back the response to the client (your browser) and closes the connection. When you again open the URL then the entire procedure is followed again.

HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. They are commonly referred to as HTTP verbs. I am listing some verbs below:

1. GET - Requests made to retrieve data.
2. POST - Requests made to submit data to server, resulting in the change of state or side effects on the server.
3. PUT - Requests made to replace all current representations of the target resource with the request payload.
4. DELETE - Requests made to delete the specified resource on the server.

Complete list can be found here.

Just like the requests method, we have response status codes which are important for interpreting the server's response on the client-side. Some of the status codes are

1. 200 - Successful
2. 404 - Not Found
3. 500 - Internal Server Error
4. 301 - Redirect

Complete list can be found here.

To read more about HTTP, check out this MDN resource page.

Let's build

Node.js provides a lot of powerful modules built-in; HTTP is one of those modules. As docs put it,

The HTTP interfaces in Node.js are designed to support many features of the protocol which have been traditionally difficult to use.

We are going to require http in our server.js

const http = require("http");

It provides us with a method createServer which takes a callback requestListener as an argument and returns a new instance of http.Server. Let's use this.

const http = require("http");

const server = http.createServer((req, res) => {
  // request handling
});

Now, we the http.Server instance in the server variable. Calling, the listen method on it will allow our server to receive requests as in it will bind the server to a port and listen for incoming connections.

...
const server = http.createServer((req, res) => {});

server.listen(8080, () => console.log("Server running on port 8080"));

By doing this much, our server is live! However, what to do when an actual request comes in?? How to handle that?

The requestListener we talked about earlier is the one that executes when a request comes in. It receives two parameters:

1. request object contains information about the current request such as URL, HTTP headers, and much more.
2. response object contains methods that are used to send data back to the client.

...
const server = http.createServer((req, res) => {
  res.writeHead(200, {"Content-Type": "text/html"});
  res.write("Hello world");
  res.end();
});
...

In the above code snippet,

1. We are calling response.writeHead() which sends an HTTP status code and a collection of response headers back to the client. Here, we are setting statusCode 200 and Content-Type: text/html.
2. We are calling response.write() which is used to send data to the client.
3. By calling response.end(), we are informing the server that response headers and body have been sent and the request has been fulfilled. The server closes the connection after this method call.

Let's refactor a bit and create a config.js file to store our app's configurations.

touch config.js

Add the following code to it and require it in our server.js

module.exports = {
  PORT: 8080, // or any other port you wish to run your server on
};

The road so far...

const http = require("http");
const { PORT } = require("./config");

const server = http.createServer((req, res) => {
  res.writeHead(200, { "Content-Type": "text/html" });
  res.write("Hello world");
  res.end();
});

server.listen(PORT, () => console.log(`Server running on ${PORT}`));

Our server works so far and we have implemented a catch-all route that serves the same Hello World content for any URL you visit on the server. Let's make it a little nice and show some actual good old HTML. 😄

Create a public folder in your root directory and inside that folder make an index.html

mkdir public
cd ./public
touch index.html

Add the following Html to index.html

<!DOCTYPE html>
<html>
  <head>
    <title>Minimal.js | Part 1</title>
    <style>
      * {
        margin: 0px;
        padding: 0px;
        font-family: "Roboto";
      }
      html,
      body {
        width: 100%;
        height: 100%;
      }
      body {
        background-color: #ececec;
        background-image: url("http://wallpaper.yomeshgupta.com/images/5.jpg");
        background-size: contain;
        background-position: center top;
      }
      h1 {
        max-width: 400px;
        margin: 0 auto;
        padding: 40px 0px;
        font-size: 18px;
        text-align: center;
      }
      a {
        color: #f67b45;
      }
      a:hover {
        color: #227093;
      }
    </style>
  </head>
  <body>
    <h1>
      Hello World. To see more wallpapers like this and make your new tab more
      delightful. Check out this
      <a
        href="https://chrome.google.com/webstore/detail/backdrops/beanogjmmfajlfkfmlchaoamcoelddjf"
        >Chrome Extension</a
      >.
    </h1>
  </body>
</html>

Now, let's require two Node.js in-built modules, fs and path

const fs = require("fs");
const path = require("path");

fs module is the File System module that provides an API for interacting with the file system. If you want to read any file, write to any file, make a directory, change permissions or anything else file system related; fs is THE CHOSEN ONE.

path module is a collection of utilities that helps while working with the file system. It provides capabilities like resolving a path, finding directory names, finding extensions of a given file/path and so much more!

Use these modules to read and serve our newly created index.html to incoming requests

...
const fs = require('fs');
const path = require('path');

const server = http.createServer((req, res) => {
  fs.readFile(path.resolve(__dirname, 'public', 'index.html'), (err, data) => {
    res.setHeader('Content-Type', 'text/html');
    if (err) {
      res.writeHead(500);
      return res.end('Some error occured');
    }
    res.writeHead(200);
    return res.end(data);
  });
});
...

Here,

1. We are using fs.readFile() method to read the contents of our index.html. It takes two arguments, file path and callback which will be executed once the file is read.
2. In our callback, if we encounter any error then we are sending an error response else we are serving index.html's content.
3. We are also using path.resolve to find the exact location of index.html on the disk.

You can read about these modules here.

Phew! Our first part is over. We, now, have our first without express HTTP server up and running! In part-2 we are going to take this up a notch and will start working on our framework. Stay tuned!

The complete code for this part can be found in this Github repo.

Wallpaper used in the example here comes bundled with a super amazing minimal chrome extension, Backdrops. Check it out here.