Learn Nodejs the right way

Before You Begin

Orignally published on Plain English in 2019

When learning a new tech stack, people often search for how well it addresses their current challenges. While this can be helpful, it doesn’t fully apply to Node.js. Most Node.js resources focus on the technologies and third-party modules used with Node.js, such as Express, which seem simple at first glance. However, these third-party wrappers around Node.js can make the platform appear easier to work with than it really is.

If I think back to two years ago, I was in the same position. I didn’t realize that I was learning numerous npm libraries but not Node.js itself. When i realized it, I took a step back to learn the native parts of Node.js, and I’m really glad I did.

Node.js on its own is powerful—more powerful than many realize—but writing barebones javascript can be overwhelming when you’re just starting out. However, it deepens your understanding of how popular npm modules work behind the scenes. After all, third-party modules in the JavaScript ecosystem exists because there is a way to do it natively without them. In node.js, specifically, the runtime ships with a lot of built-in modules that can help you build a lot of things without third-party modules. We'll go over some of these later in this article, but first, let’s take a quick look at what Node.js actually is.

Node.js in a Nutshell

Simply put, Node.js is a unsandboxed JavaScript runtime environment written in C++. JavaScript cannot be compiled to a binary and cannot interact with the system to do things like opening ports or using the file system—tasks commonly seen in Node.js. The runtime solves this problem by exposing relevent C++ business logic which is hidden behind packages and functions shipped with Node. JavaScript executed by Node.js is interpreted and compiled on the fly with necessary optimizations by V8, a JavaScript engine developed by Google, the process of which is called Just-In-Time (JIT) compilation.

Node.js was originally intended to be the ultimate server-side technology. But? Nvm

Building barebones Node.js applications

Node.js comes with a suite of powerful modules that enable you to create amazing things. Many popular npm libraries and frameworks are simply wrappers around these inbuilt modules. Learning the raw Node.js ecosystem gives you a deeper understanding of how these other modules work. Let's explore a few essential yet often overlooked modules in Node.js.

The net module

The Net module is probably the most underappreciated module in Node.js. People rarely talk about it, even though they may know a lot about some of its implementations. I haven’t found many good resources on the Net module, so let me know if you know of any. It can be used to create a bare-bones TCP server. Working directly with the Net module can be confusing because it involves writing raw response headers and bodies. This is why you might prefer using a higher-level implementation, such as the HTTP module. Here’s how the Net module works:

import { createServer } from 'net';

const server = createServer();

server.listen(2002);

let connectedClients = [];

server.on('connection', (client) => {
  connectedClients.push(client);  // this keeps the client in queue until they disconnect themselevs, however you can choose to close the connection anytime you wish by calling .end method on the client instance
  let index = connectedClients.length - 1;
  client.on('close', () => {
    connectedClients.splice(index, 1);
  });
});

server.on('close', (client) => {
  console.log('A client disconnected', client);
});

server.on('error', console.error);
server.once('listening', () => {
  setInterval(() => {
    connectedClients.forEach((client) => {
      if (connectedClients.length) client.write('New message brodcasted\r\n');
    });
  }, 5000);
  console.log('Server listening');
});

This simple server sends a message to all connected clients every 5 seconds. Let’s break down what’s happening inside that file. We import the createServer function, which allows us to create a TCP server. The server listens for a couple of events—the important ones being client connections and the server starting. When a client connects, the server pushes the client object, referring to the current connection, to an array of connected clients and waits for more connections. Once the server starts listening on the specified port, it checks the connected clients' array every 5 seconds and sends a message to all connected clients if there are any.

The http Module

The HTTP module is like the soul of Node.js. Many popular npm libraries, like Express and Hapi, are built on top of it. While the HTTP module is often considered difficult to work with, it’s not! It just requires a few more lines of code compared to regular frmaeworks (for a simple application). If you’ve never built a server with the http module, here’s how to get started:

import http from 'http';
import url from 'url';

// Instead of searching for a router with '/' included a better approch would be replacing the path name with a regex to strip of the '/' at begining or end
const routers = {
  GET: {
    '/hello': (req, res) => {
      const jsonPayload = JSON.stringify({ foo: 'bar' });
      res
        .writeHead(200, {
          'Content-Length': Buffer.byteLength(jsonPayload),
          'Content-Type': 'application/json',
        })
        .end(jsonPayload);
    },
  },
  POST: {
    '/hello': (_, res) => {
      res.writeHead(200).end();
    },
  },
  404: (req, res) => {
    const notFoundMessage = `Cannot ${req.method} ${req.path}\n`;
    res
      .writeHead(400, {
        'Content-Length': Buffer.byteLength(notFoundMessage),
        'Content-Type': 'text/plain',
      })
      .end(notFoundMessage);
  },
};

const server = http.createServer((req, res) => {
  const parsedUrl = url.parse(req.url, true);
  const path = parsedUrl.pathname;
  const query = { ...parsedUrl.query };
  req.on('data', (data) => {
    if (!req.body) req.body = data.toString();
    else req.body += data.toString();
  });
  req.on('end', () => {
    // Adding properties to req object
    if (req.body)
      try {
        req.body = JSON.parse(req.body);
      } catch (error) {
        null;
      }
    req.query = query;
    req.path = path;
    console.log(req.path);
    // Checking if a router matched with requested pathname
    const matchedRouter =
      typeof routers[req.method][path] === 'undefined'
        ? routers['404']
        : routers[req.method][path];
    matchedRouter(req, res);
  });
});

server.listen(2002, () => {
  console.log('Listening on port 2002');
});

This basic server waits for incoming requests and routes them to the appropriate handler once the request stream ends. If a matching handler isn’t found, it routes the request to the 404 handler. A few takeaways include how the request body is a readable stream, with data transferred in chunks depending on the size. For small HTTP requests, this isn’t noticeable, but you’ll observe it with larger requests.

Note — you could have set the headers on the request object using the setHeader method of the request object but I did it the other way for simplicity's sake.

Worker Threads

Worker threads in Node.js are like web workers in the browser—they allow you to spin up another JavaScript execution thread in parallel with the main thread. This is particularly helpful for CPU-intensive tasks that could block the main thread. Worker threads and the main thread communicate via a message channel, where they can send and receive messages. Implementing worker threads in code is straightforward, usually involving sending and responding to messages and killing the worker when done.

// index.mjs

import { Worker } from 'worker_threads';

const worker = new Worker('./worker.js');
worker.on('message', (message) => {
  console.log(message);
  worker.terminate();
});
worker.on('error', (err) => console.error(err));
worker.on('exit', (code) => {
  console.log(`Worker exited with exit code ${code}`);
});

worker.postMessage('Hello from main');

// worker.mjs

import { parentPort } from 'worker_threads';

parentPort.on('message', (message) => {
  console.log(message);
  parentPort.postMessage('Message from the worker');
  // Some really cpu intensive task
});

This is the simplest worker you can make, the idea remains the same even when building something really complex with worker threads. And like always, it’s a good idea to check the documentation for other properties and methods on worker.

This sums up the most essential modules (opinionated) which are generally not discussed, but there are also a few classes and objects I think are crucial for understanding the Node.js ecosystem:

Buffer Objects

A buffer generally represents a portion of memory that stores some data. In Node.js, a buffer is a special global object used for raw data, pointing to a memory allocation outside the V8 heap. Buffers are heavily used in TCP streams and anything that involves direct interaction with the OS. You might have noticed the use of a buffer in the HTTP server example, where we converted incoming request body objects to strings using the toString method. Buffers are widely used in Node.js, so it’s beneficial to have some familiarity with them.

Stream Class

Streams in Node.js are exactly what they sound like—a continuous flow of something, in this case, chunks of data, usually of the Buffer type. Streams are heavily used in Node.js when working with the file system and network requests.

EventEmitter Class

The asynchronous nature of Node.js often revolves around events. The EventEmitter class helps create objects that can emit events, usually indicating a change in the object's state. Many Node.js modules use EventEmitter for this purpose. The concept here is how the asynchronous callback pattern works in Node.js. You can attach functions to listen and respond to specific events emitted by your instance.

Wrappping Up

Phew! That was a lot of information. This article is intended to convey how understanding things native to node can help people understand the ecosystem better. I’m not saying that using third-party modules makes you a bad developer (but it does), but it’s always helpfule to understand things at a lower level. I hope you learned something new today 😀

Resources and Further reading

• Node Documentation

• Substack’s Streams handbook