Modern JavaScript

If you did not learn JavaScript from the beginning, understanding its modern form can be difficult. The ecosystem grows and changes quickly, making it hard to understand the problems its many tools were created to solve. I started programming in 1998, but only began to understand JavaScript in 2014. I remember looking through Browserify and reading its slogan:

Browserify lets you require modules in the browser by bundling all your dependencies

I did not understand a word of that sentence and began investigating how it could help me as a developer.

This article explains the context in which JavaScript tools evolved up to 2017. We will start at the beginning and build a website the old-fashioned way, using plain HTML and JavaScript without any tools. We will then introduce tools one at a time and examine the problems each solves. This historical context should help you understand and adapt to JavaScript's constantly changing landscape.

Using JavaScript the old-fashioned way

Let's build an old-fashioned website using only HTML and JavaScript. This means downloading and linking files manually. Here is a simple index.html that references a JavaScript file:

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>JavaScript Example</title>
  <script src="index.js"></script>
</head>
<body>
  <h1>Hello from HTML!</h1>
</body>
</html>

The line <script src="index.js"></script> references the JavaScript file index.js, located in the same directory:

// index.js
console.log("Hello from JavaScript!");

That is all we need to build a website! Suppose you want to add the third-party library moment.js to format dates for readability. For example, you could use the moment function in JavaScript:

moment().startOf('day').fromNow();        // 20 часов назад

But that only works once moment.js has been added to your website! The moment.js home page provides instructions:

There is a lot in the Installation section. For now, we can ignore it: simply download moment.min.js into the same directory and include it in index.html.

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Example</title>
  <link rel="stylesheet" href="index.css">
  <script src="moment.min.js"></script>
  <script src="index.js"></script>
</head>
<body>
  <h1>Hello from HTML!</h1>
</body>
</html>

Notice that moment.min.js loads before index.js, so the moment function is available inside index.js:

// index.js
console.log("Hello from JavaScript!");
console.log(moment().startOf('day').fromNow());

This is how we used to build websites with JavaScript libraries. The process was easy to understand, but finding and downloading new versions whenever libraries were updated was frustrating.

Using a JavaScript package manager (npm)

Around 2010, several competing package managers emerged to automate downloading and updating libraries from a central repository. Bower was the most popular in 2013, but by 2015 it had been overtaken by npm. Since late 2016, yarn has also been widely used as an alternative npm interface, although it still works with npm packages under the hood.

Originally, npm was created specifically as a package manager for node.js, a JavaScript runtime intended for servers rather than front-end development. Using it to manage libraries that run in browsers therefore seems a little unusual.

Note: package managers usually require the command line, which front-end development previously did not need. If you have never used it, start with this guide. In modern JavaScript, knowing your way around the command line matters, and it opens doors to other areas of development too.

Let's use npm to install moment.js automatically instead of downloading it manually. If Node.js is installed, npm is already available. Open the directory containing index.html in your terminal and enter:

$ npm init

You will be asked a few questions; you can simply press Enter to accept the defaults. A new package.json file will then be generated. This configuration file stores npm's information about the project. Its default contents look like this:

{
  "name": "your-project-name",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

To install the moment.js package, follow the npm website's instructions and enter:

$ npm install moment --save

This command does two things:

  1. Downloads all the code from the moment.js package into a directory named node_modules.
  2. Automatically updates package.json to track moment.js as a project dependency.
{
  "name": "modern-javascript-example",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "moment": "^2.19.1"
  }
}

This is useful when working with other developers. Instead of sharing the node_modules directory, which can be very large, you only need to share package.json. Other developers can then install the required packages automatically using npm install.

We no longer need to download moment.js manually: npm handles downloading and updating it. Inside node_modules/moment/min, we can find moment.min.js. This means index.html can reference the copy installed through npm:

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>JavaScript Example</title>
  <script src="node_modules/moment/min/moment.min.js"></script>
  <script src="index.js"></script>
</head>
<body>
  <h1>Hello from HTML!</h1>
</body>
</html>

Being able to download and update packages through npm and the command line is useful. But we still have to search node_modules for each package's location and add it to the HTML manually. Let's automate that too.

Using a JavaScript module bundler (webpack)

Most programming languages can import code from one file into another. JavaScript originally could not, because it was designed to run in browsers without access to a user's file system for security reasons. For a long time, organizing JavaScript across multiple files meant loading each file with globally accessible variables.

That is exactly what we did with moment.js: loading the entire moment.min.js file in HTML defines a global moment variable. Any file loaded afterward can access it, whether or not that file needs it.

The CommonJS project began in 2009 to define a JavaScript ecosystem outside the browser. Much of its work concerned modules, allowing JavaScript files to import and export code without using global variables, as many other languages do. Node.js became the best-known implementation of CommonJS modules.

node.js

As mentioned earlier, Node.js is a JavaScript runtime designed for servers. Using its modules originally looked like this: instead of loading all of moment.min.js through an HTML script tag, you could load it directly from JavaScript:

// index.js
var moment = require('moment');
console.log("Hello from JavaScript!");
console.log(moment().startOf('day').fromNow());

Module loading works well because Node.js runs on a server and has file-system access. It also knows where npm modules are installed, so you can write require('moment') instead of require('./node_modules/moment/min/moment.min.js').

However, trying this code in a browser produces an error saying that require is not defined. Browsers do not have file-system access, so implementing module loading is tricky: files must be loaded dynamically, either synchronously, which slows execution, or asynchronously, which can cause synchronization problems.

This is where a bundler comes in. It has file-system access and combines modules into bundles that browsers can use without accessing the file system themselves. Here, the bundler finds every require expression and replaces it with the contents of the required file. The result is one JavaScript file without any require expressions!

The first widely popular bundler was Browserify, released in 2011. It pioneered using Node.js-style require expressions in front-end code, helping npm become the leading package manager. By 2015, webpack had taken the lead, helped by the popularity of the front-end framework React, which made extensive use of its capabilities.

Let's use webpack to make our require('moment') example work in a browser. First, install the bundler in the project. Webpack itself is an npm package, so enter:

$ npm install webpack --save-dev

Notice the --save-dev argument: it records webpack as a development dependency, which is not needed on the production server. This is reflected automatically in package.json:

{
  "name": "modern-javascript-example",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "moment": "^2.19.1"
  },
  "devDependencies": {
    "webpack": "^3.7.1"
  }
}

Webpack is now installed among the packages in node_modules. You can run it from the command line:

$ ./node_modules/.bin/webpack index.js bundle.js

This command runs webpack on index.js, finds every require expression and substitutes the relevant code to produce one output file, bundle.js. We no longer need index.js in the browser, since it contains unsupported require expressions. Instead, we use bundle.js and update index.html accordingly:

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>JavaScript Example</title>
  <script src="bundle.js"></script>
</head>
<body>
  <h1>Hello from HTML!</h1>
</body>
</html>

Refresh the browser and everything should work as before.

We have to run webpack every time index.js changes. This becomes tedious, especially as we add more advanced features such as source-map generation to debug original source code after transpilation. Webpack can read its options from a webpack.config.js file in the project root:

// webpack.config.js
module.exports = {
  entry: './index.js',
  output: {
    filename: 'bundle.js'
  }
};

Now, whenever index.js changes, we can run webpack with this command:

$ ./node_modules/.bin/webpack

There is no need to specify index.js and bundle.js, because webpack reads them from webpack.config.js. This is better, but typing the command after every code change is still tedious. Let's simplify it further.

This approach transformed the workflow. We no longer load external scripts through global variables. New JavaScript libraries are included through require expressions rather than HTML <script> tags. A single JavaScript bundle often improves performance. Adding a build step also enables several powerful development features.

Transpiling code to use new language features (Babel)

Transpiling means converting code into another, similar language. It is an important part of front-end development. Because browsers adopt new features slowly, languages with experimental features have been created and transpiled into browser-compatible code.

For CSS, examples include Sass, Less and Stylus. For JavaScript, a popular transpiler for a while was CoffeeScript (released around 2010). Today, many developers use babel or TypeScript. CoffeeScript improves JavaScript by substantially changing its syntax, with optional parentheses, significant whitespace and more. Babel is not a new language: it transpiles next-generation JavaScript, whose features are not yet available in every browser (ES2015 and later), into older, more compatible JavaScript (ES5). TypeScript is essentially next-generation JavaScript with optional static typing. Many developers prefer Babel because it stays closer to plain JavaScript.

Let's add Babel to the webpack build. First, install the transpiler's npm packages in the project:

$ npm install babel-core babel-preset-env babel-loader --save-dev

We installed three separate packages as development dependencies:

  • babel-core — Babel's core functionality;
  • babel-preset-env — the preset specifying which new JavaScript features to transpile;
  • babel-loader — the package that connects Babel to webpack.

Configure webpack to use babel-loader by editing webpack.config.js:

// webpack.config.js
module.exports = {
  entry: './index.js',
  output: {
    filename: 'bundle.js'
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader',
          options: {
            presets: ['env']
          }
        }
      }
    ]
  }
};

The syntax may be confusing, but fortunately this configuration rarely needs changing. We ask webpack to find all .js files outside node_modules and transpile them through babel-loader with the babel-preset-env preset. Read more about webpack configuration syntax here.

Everything is configured, so we can use ES2015 features! Here is an ES2015 template string in index.js:

// index.js
var moment = require('moment');
console.log("Hello from JavaScript!");
console.log(moment().startOf('day').fromNow());
var name = "Bob", time = "today";
console.log(`Hello ${name}, how are you ${time}?`);

For module loading, we can use an ES2015 import statement instead of require, as many codebases now do:

// index.js
import moment from 'moment';
console.log("Hello from JavaScript!");
console.log(moment().startOf('day').fromNow());
var name = "Bob", time = "today";
console.log(`Hello ${name}, how are you ${time}?`);

In this example, import looks much like require, but it is more flexible in complex situations. Since index.js has changed, run webpack again:

$ ./node_modules/.bin/webpack

Refresh index.html in the browser. When this article was written, most browsers supported all ES2015 features, so it was hard to tell whether Babel was responsible. Try an older browser such as IE9, or look for the transpiled line in bundle.js:

// bundle.js
// ...
console.log('Hello ' + name + ', how are you ' + time + '?');
// ...

Babel converted the ES2015 template string into ordinary JavaScript string concatenation for compatibility. This may not be the most impressive example, but transpilation is powerful. It can bring exciting features to JavaScript, such as async/await. Although transpiling can sometimes be tedious, it has helped improve the language considerably in recent years by letting developers experiment with future features.

We are almost finished, but the workflow still has a few rough edges. To improve performance, we should minify the output bundle, which is straightforward. We also need to rerun webpack after every JavaScript change, and that quickly gets old. Let's look at tools that solve these problems.

Using a task runner (npm scripts)

Now that our module workflow includes a build step, a task runner can automate the operations involved. In front-end development, these include minification, image optimization, tests and more.

Grunt was the most popular tool in 2013, but Gulp soon replaced it. Both use plugins that wrap other command-line tools. Today, npm's built-in scripting is often used to call these tools directly.

Let's write npm scripts to simplify using webpack. Edit package.json:

{
  "name": "modern-javascript-example",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "webpack --progress -p",
    "watch": "webpack --progress --watch"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "moment": "^2.19.1"
  },
  "devDependencies": {
    "babel-core": "^6.26.0",
    "babel-loader": "^7.1.2",
    "babel-preset-env": "^1.6.1",
    "webpack": "^3.7.1"
  }
}

We added two scripts, build and watch. To run the build script, enter:

$ npm run build

This runs webpack using the webpack.config.js configuration created earlier. The --progress option displays percentage progress, and -p minifies the code. Now run the watch script:

$ npm run watch

The --watch option reruns webpack automatically whenever a JavaScript file changes, which is convenient during development.

The package.json scripts can run webpack without the full ./node_modules/.bin/webpack path because Node.js knows where installed npm executables are located. To make things even easier, install webpack-dev-server, a simple development web server with live reloading, as a development dependency:

$ npm install webpack-dev-server --save-dev

Add an npm script to package.json:

{
  "name": "modern-javascript-example",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "webpack --progress -p",
    "watch": "webpack --progress --watch",
    "server": "webpack-dev-server --open"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "moment": "^2.19.1"
  },
  "devDependencies": {
    "babel-core": "^6.26.0",
    "babel-loader": "^7.1.2",
    "babel-preset-env": "^1.6.1",
    "webpack": "^3.7.1"
  }
}

Now start the development server:

$ npm run server

This automatically opens index.html in the browser at localhost:8080 by default. When you change JavaScript in index.js, webpack-dev-server rebuilds the bundle and refreshes the browser. It saves time by letting you focus on code instead of constantly switching between the editor and browser to check changes.

Webpack and webpack-dev-server have many more options; we have only scratched the surface. Read more here. You can also write npm scripts for other command-line tasks, such as converting Sass to CSS, compressing images or running tests.

Conclusion

We have taken a brief tour of modern JavaScript, moving from plain HTML and JavaScript to using:

  • a package manager to download third-party packages automatically;
  • a bundler to combine scripts into single files;
  • a transpiler to use future JavaScript features;
  • and a task runner to automate build operations.

There is a lot to explore, especially for beginners. Web development used to be an inviting starting point because it was easy to begin and get involved. Today it is more complicated, partly because of the variety of rapidly changing tools.

Still, things are better than they may seem. The ecosystem is settling down, and Node-based tools are increasingly accepted for front-end work. Using npm for packages, Node-style require or import for modules, and npm scripts for tasks creates a convenient, consistent workflow. It is much simpler than it was a year or two ago!

Frameworks often include tools that simplify getting started. Ember's ember-cli strongly influenced Angular's angular-cli, React's create-react-app, Vue's vue-cli and others. These tools provide what a project needs to start writing code. They are not magic: they simply configure the tools for a comfortable workflow. You may still need to customize webpack, Babel and other settings, so understanding each tool's role matters.

Modern JavaScript can be challenging because it continues to change rapidly. Although it sometimes feels like reinventing the wheel, this evolution enables innovations such as hot reloading, real-time linting and time-travel debugging. It is an exciting time to be a developer, and I hope this article helps you plan your own path.