Understanding Node.js: Beyond the Frameworks
When learning a new tech stack, people often search for how well it addresses their current challenges. While this can be helpful, it doesn't fully apply to Node.js. Most Node.js resources focus on the technologies and third-party modules used with Node.js, such as Express, which seem simple at first glance. However, these third-party wrappers around Node.js can make the platform appear easier to work with than it really is.
Looking back two years ago, I was in the same position. I didn't realize that I was learning numerous npm libraries but not Node.js itself. When I recognized this gap, I took a step back to learn the native parts of Node.js, and I'm really glad I did.
Node.js on its own is powerful—more powerful than many realize—but writing barebones JavaScript can be overwhelming when you're just starting out. However, it deepens your understanding of how popular npm modules work behind the scenes. After all, third-party modules in the JavaScript ecosystem exist because there is a way to do it natively without them. In Node.js, specifically, the runtime ships with many built-in modules that can help you build a lot of things without third-party dependencies. We'll explore some of these later in this article, but first, let's take a quick look at what Node.js actually is.
Node.js in a Nutshell
Simply put, Node.js is an unsandboxed JavaScript runtime environment written in C++. JavaScript cannot be compiled to a binary and cannot interact with the system to do things like opening ports or using the file system—tasks commonly seen in Node.js applications. The runtime solves this problem by exposing relevant C++ business logic which is hidden behind packages and functions shipped with Node. JavaScript executed by Node.js is interpreted and compiled on the fly with necessary optimizations by V8, a JavaScript engine developed by Google, through a process called Just-In-Time (JIT) compilation.
Node.js was originally intended to be the ultimate server-side technology, and it has largely succeeded in that mission, becoming one of the most popular platforms for building backend services.
Building Barebones Node.js Applications
Node.js comes with a suite of powerful modules that enable you to create amazing things. Many popular npm libraries and frameworks are simply wrappers around these built-in modules. Learning the raw Node.js ecosystem gives you a deeper understanding of how these other modules work. Let's explore a few essential yet often overlooked modules in Node.js.
The net Module
The Net module is probably the most underappreciated module in Node.js. People rarely talk about it, even though they may know a lot about some of its implementations. Documentation and comprehensive tutorials for the Net module are relatively scarce compared to other modules. It can be used to create a bare-bones TCP server. Working directly with the Net module can be confusing because it involves writing raw response headers and bodies. This is why you might prefer using a higher-level implementation, such as the HTTP module. Here's how the Net module works:
import { createServer } from 'net';
const server = createServer();
server.listen(2002);
let connectedClients = [];
server.on('connection', (client) => {
connectedClients.push(client); // this keeps the client in queue until they disconnect themselves; however, you can choose to close the connection anytime you wish by calling .end method on the client instance
let index = connectedClients.length - 1;
client.on('close', () => {
connectedClients.splice(index, 1);
});
});
server.on('close', (client) => {
console.log('A client disconnected', client);
});
server.on('error', console.error);
server.once('listening', () => {
setInterval(() => {
connectedClients.forEach((client) => {
if (connectedClients.length) client.write('New message broadcasted\r\n');
});
}, 5000);
console.log('Server listening');
});
This simple server sends a message to all connected clients every 5 seconds. Let's break down what's happening inside that file. We import the createServer function, which allows us to create a TCP server. The server listens for a couple of events—the important ones being client connections and the server starting. When a client connects, the server pushes the client object, referring to the current connection, to an array of connected clients and waits for more connections. Once the server starts listening on the specified port, it checks the connected clients' array every 5 seconds and sends a message to all connected clients if there are any.
The http Module
The HTTP module is like the soul of Node.js. Many popular npm libraries, like Express and Hapi, are built on top of it. While the HTTP module is often considered difficult to work with, it's not! It just requires a few more lines of code compared to regular frameworks (for a simple application). If you've never built a server with the http module, here's how to get started:
import http from 'http';
import url from 'url';
// Instead of searching for a router with '/' included, a better approach would be replacing the path name with a regex to strip off the '/' at beginning or end
const routers = {
GET: {
'/hello': (req, res) => {
const jsonPayload = JSON.stringify({ foo: 'bar' });
res
.writeHead(200, {
'Content-Length': Buffer.byteLength(jsonPayload),
'Content-Type': 'application/json',
})
.end(jsonPayload);
},
},
POST: {
'/hello': (_, res) => {
res.writeHead(200).end();
},
},
404: (req, res) => {
const notFoundMessage = `Cannot ${req.method} ${req.path}\n`;
res
.writeHead(404, {
'Content-Length': Buffer.byteLength(notFoundMessage),
'Content-Type': 'text/plain',
})
.end(notFoundMessage);
},
};
const server = http.createServer((req, res) => {
const parsedUrl = url.parse(req.url, true);
const path = parsedUrl.pathname;
const query = { ...parsedUrl.query };
req.on('data', (data) => {
if (!req.body) req.body = data.toString();
else req.body += data.toString();
});
req.on('end', () => {
// Adding properties to req object
if (req.body)
try {
req.body = JSON.parse(req.body);
} catch (error) {
// Silent fail if body is not valid JSON
}
req.query = query;
req.path = path;
console.log(req.path);
// Checking if a router matched with requested pathname
const matchedRouter =
!routers[req.method] || typeof routers[req.method][path] === 'undefined'
? routers['404']
: routers[req.method][path];
matchedRouter(req, res);
});
});
server.listen(2002, () => {
console.log('Listening on port 2002');
});
This basic server waits for incoming requests and routes them to the appropriate handler once the request stream ends. If a matching handler isn't found, it routes the request to the 404 handler. A few takeaways include how the request body is a readable stream, with data transferred in chunks depending on the size. For small HTTP requests, this isn't noticeable, but you'll observe it with larger requests.
Note — you could have set the headers on the response object using the setHeader method of the response object, but I did it the other way for simplicity's sake.
Worker Threads
Worker threads in Node.js are like web workers in the browser—they allow you to spin up another JavaScript execution thread in parallel with the main thread. This is particularly helpful for CPU-intensive tasks that could block the main thread. Worker threads and the main thread communicate via a message channel, where they can send and receive messages. Implementing worker threads in code is straightforward, usually involving sending and responding to messages and terminating the worker when done.
// index.mjs
import { Worker } from 'worker_threads';
const worker = new Worker('./worker.js');
worker.on('message', (message) => {
console.log(message);
worker.terminate();
});
worker.on('error', (err) => console.error(err));
worker.on('exit', (code) => {
console.log(`Worker exited with exit code ${code}`);
});
worker.postMessage('Hello from main');
// worker.mjs
import { parentPort } from 'worker_threads';
parentPort.on('message', (message) => {
console.log(message);
parentPort.postMessage('Message from the worker');
// Some really CPU-intensive task
});
This is the simplest worker you can make; the idea remains the same even when building something really complex with worker threads. As always, it's a good idea to check the documentation for other properties and methods available on worker threads.
This covers the most essential modules (in my opinion) that are generally not discussed, but there are also a few classes and objects I think are crucial for understanding the Node.js ecosystem:
Buffer Objects
A buffer generally represents a portion of memory that stores some data. In Node.js, a buffer is a special global object used for raw data, pointing to a memory allocation outside the V8 heap. Buffers are heavily used in TCP streams and anything that involves direct interaction with the OS. You might have noticed the use of a buffer in the HTTP server example, where we converted incoming request body objects to strings using the toString method. Buffers are widely used in Node.js, so it's beneficial to have some familiarity with them.
Stream Class
Streams in Node.js are exactly what they sound like—a continuous flow of something, in this case, chunks of data, usually of the Buffer type. Streams are heavily used in Node.js when working with the file system and network requests.
EventEmitter Class
The asynchronous nature of Node.js often revolves around events. The EventEmitter class helps create objects that can emit events, usually indicating a change in the object's state. Many Node.js modules use EventEmitter for this purpose. The concept here is how the asynchronous callback pattern works in Node.js. You can attach functions to listen and respond to specific events emitted by your instance.
Wrapping Up
That was a lot of information! This article aimed to convey how understanding Node.js at its core can help developers navigate the ecosystem better. I'm not suggesting that using third-party modules is bad practice—they're valuable tools that save time and effort—but understanding what happens under the hood will make you a more effective developer. I hope you learned something new today! 😀