Understanding Webpack's Require

In this blog post, we are going to talk about webpack's require function, how it manages our modules and explore a use-case where such knowledge might help debug.

What is Webpack?

JavaScript, in the current modern era, is a different beast altogether than in previous years. Way too many libraries, frameworks, tooling and information are available. There are dozens of scripts, each might be created by different persons/teams. There are third party libraries, which are known as dependencies. These dependencies need to be loaded and the order of loading also matters. On top of that, lack of language features support makes things worse.

To solve these problems, we need to keep track of each script, transpile code, add them to the webpage, maintain the loading order and take care of asynchronous behaviour.

Webpack helps us to solve these problems in a much more hassle free-way. Sure, the learning curve can be daunting at times however latest release Webpack 4 comes with zero configuration out of the box.

Webpack is a module bundler. It runs at the time of development/deployment and the final output runs on your webpage. It takes a configuration file which provides instructions like what is the entry point of the codebase, where should it write the final bundle, how to handle different encountered resources and what all optimizations to perform. It starts traversing the codebase starting from the entry point and handles each imported resource as per the provided loader configuration. You need to describe to Webpack how to load and manipulate files such as *.js. After processing, it creates a final bundle and writes it to the output destination on the disk.

Webpack Runtime

Webpack Runtime often called "Webpack Bootstrap", is the boilerplate code added to the final bundle which is responsible for functionalities like managing all the imported modules, cross-browser support, defining the public path and much more. Following is the webpackBootstrap code.

/******/ (function(modules) { // webpackBootstrap
/******/  // The module cache
/******/  var installedModules = {};
/******/
/******/  // The require function
/******/  function __webpack_require__(moduleId) {
/******/
/******/    // Check if module is in cache
/******/    if(installedModules[moduleId]) {
/******/      return installedModules[moduleId].exports;
/******/    }
/******/    // Create a new module (and put it into the cache)
/******/    var module = installedModules[moduleId] = {
/******/      i: moduleId,
/******/      l: false,
/******/      exports: {}
/******/    };
/******/
/******/    // Execute the module function
/******/    modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
/******/
/******/    // Flag the module as loaded
/******/    module.l = true;
/******/
/******/    // Return the exports of the module
/******/    return module.exports;
/******/  }
/******/
/******/
/******/  // expose the modules object (__webpack_modules__)
/******/  __webpack_require__.m = modules;
/******/
/******/  // expose the module cache
/******/  __webpack_require__.c = installedModules;
/******/
/******/  // define getter function for harmony exports
/******/  __webpack_require__.d = function(exports, name, getter) {
/******/    if(!__webpack_require__.o(exports, name)) {
/******/      Object.defineProperty(exports, name, {
/******/        configurable: false,
/******/        enumerable: true,
/******/        get: getter
/******/      });
/******/    }
/******/  };
/******/
/******/  // define __esModule on exports
/******/  __webpack_require__.r = function(exports) {
/******/    Object.defineProperty(exports, '__esModule', { value: true });
/******/  };
/******/
/******/  // getDefaultExport function for compatibility with non-harmony modules
/******/  __webpack_require__.n = function(module) {
/******/    var getter = module && module.__esModule ?
/******/      function getDefault() { return module['default']; } :
/******/      function getModuleExports() { return module; };
/******/    __webpack_require__.d(getter, 'a', getter);
/******/    return getter;
/******/  };
/******/
/******/  // Object.prototype.hasOwnProperty.call
/******/  __webpack_require__.o = function(object, property) { return Object.prototype.hasOwnProperty.call(object, property); };
/******/
/******/  // __webpack_public_path__
/******/  __webpack_require__.p = "";
/******/
/******/
/******/  // Load entry module and return exports
/******/  return __webpack_require__(__webpack_require__.s = 0);
/******/ })
/************************************************************************/
/******/ ([
/* 0 */
/***/ (function(module, exports, __webpack_require__) {
....
/***/ ])

Yes, this seems like a lot of code and kind of overwhelming. Folks at Webpack are kind enough to leave comments for us, mere mortals, to understand this code.

Let's break it up

In this blog post, we are going to look at the __webpack_require__ functionality. We are going to take this code apart into tiny snippets and then talk about it.

Remember, first comes the code snippet than the explanation.

/******/ (function(modules) { // webpackBootstrap
/******/ ...
/******/ })
/************************************************************************/
/******/ ([
/* 0 */
/***/ (function(module, exports, __webpack_require__) {
....
/***/ ])

If we take a look at the larger picture then the above code is an IIFE which takes an array of IIFE(s) as the argument. Every array element represents a module in our code that is assigned a moduleId like first is 0 which is also the entry module.

/******/ // The module cache
/******/ var installedModules = {};

It is a cache that would store all the modules as the code executes.

/******/  // The require function
/******/  function __webpack_require__(moduleId) {
/******/
/******/    // Check if module is in cache
/******/    if(installedModules[moduleId]) {
/******/      return installedModules[moduleId].exports;
/******/    }
/******/  ...

Here, starts the function declaration of __webpack_require__. It takes moduleId as a parameter and whenever this function is invoked when the cache is being checked if the module is already loaded or not. If yes then the module is returned from the cache else we load the module.

.exports is not native module export. It is a custom property defined by webpack runtime.

/******/    // Create a new module (and put it into the cache)
/******/    var module = installedModules[moduleId] = {
/******/      i: moduleId,
/******/      l: false,
/******/      exports: {}
/******/    };
/******/    ...

If the module is not found in the cache then we load it. A variable called module is defined which is an object and has three properties on it.

1. i refers to the module.
2. l is a boolean flag that signifies if the module is already loaded or not.
3. exports is an object.

At the same time, we put this module in the cache via installedModules[moduleId].

/******/    // Execute the module function
/******/    modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
/******/    ...

Now, we call the module which we just placed in the cache.

1. We look it up via moduleId.
2. Every module is an IIFE. We invoke it using the .call method with the calling context module.exports.
3. We provide it with the necessary arguments required by the module.
4. We pass the existing __webpack_require__ function so that the current module can require other modules if needed.

/******/    // Flag the module as loaded
/******/    module.l = true;
/******/
/******/    // Return the exports of the module
/******/    return module.exports;
/******/    ...

We set the l flag to true which means the module is now loaded then return the module.exports.

This completes the __webpack_require__ function. The rest of the code takes care of exposing certain properties, support for non-harmony modules and others.

/******/ // Load entry module and return exports
/******/ return __webpack_require__((__webpack_require__.s = 0));

We end the IIFE by calling __webpack_require__ with the first module a.k.a the entry module as the parameter. It initiates the above cycle we discussed of finding it in the cache or loading it in the cache and returning the module.exports.

Problem Usecase

This entire blog post is more good to know. You might not need it ever in your day job. However, at times it helps. I will tell you one small use case where knowing this code helped me track and solve a bug.

Recently, I was assigned to optimize a JS bundle. Consider, a very large bundle that consists of many modules spanning different parts of a repo. Now, we realized that the bundle is being shipped with duplicate copies of jQuery which lead to an unnecessary increase in payload size.

I am going to dumb down the complete problem case as the issue was much more complex and I cannot share all the details. I will focus on just how webpack require knowledge helped us.

Now, this duplication shouldn't have happened because one module imported the jQuery and bound it to window scoped variable. All other modules used it from there rather than importing it. The task was to identify how the two copies are being imported and shipped.

While going through the bundle, I came across the following code snippets.

/* 0 */
/***/ (function(module, exports, __webpack_require__) {

var __WEBPACK_AMD_DEFINE_ARRAY__, __WEBPACK_AMD_DEFINE_RESULT__;/*!
 * jQuery JavaScript Library v1.11.2
  ...

/* 6 */
/***/ (function(module, exports, __webpack_require__) {

var __WEBPACK_AMD_DEFINE_ARRAY__, __WEBPACK_AMD_DEFINE_RESULT__;/*!
 * jQuery JavaScript Library v1.11.2
  ...

1. This meant the same library is being imported twice in different modules rather than using the window scoped jQuery.
2. moduleId(s) for different copies of jQuery are 0 and 6.
3. Knowing that these moduleIds are assigned by webpack, what they mean and how they are used by __webpack_require__ to load the modules helped in finding the part of the bundle where these modules are being required.

Instance One

var config = __webpack_require__(1),
    jQuery = __webpack_require__(0),
    consts = __webpack_require__(4),
    yetAnotherModule = __webpack_require__(5),

Instance Two

var jQuery = __webpack_require__(6),
    renderModule = __webpack_require__(7),
    logicModule = __webpack_require__(8),
    someOtherModule = __webpack_require__(9),

The above code snippet demonstrates that two modules required the jQuery whereas in the ideal case, the first module should require it and bind it. The second module should use the window scoped variable rather than importing it.

We were able to find the part of the codebase which provided us with context that second jQuery is being imported in the module which also requires renderModule, logicModule.

So now, the only thing left was to run a simple search in IDE and change the code accordingly in the second module to make it use the window scoped jQuery rather than importing it.

Boom, with that change 247 KB disappeared from the bundle! :P

I know, this might not seem like much or a magic wand but having such knowledge drastically reduced the debugging time and efforts.