Node.js Stream

Stream is an abstract interface, and many objects in Node implement this interface. For example, the request object for an HTTP server request is a Stream, as is the stdout (standard output).

Node.js has four types of streams:

• Readable - Operations for reading.
• Writable - Operations for writing.
• Duplex - Operations for both reading and writing.
• Transform - Operations where data is written and then read out.

All Stream objects are instances of EventEmitter. Common events include:

• data - Triggered when data is available to read.
• end - Triggered when there is no more data to read.
• error - Triggered when an error occurs during receiving or writing.
• finish - Triggered when all data has been flushed to the underlying system.

This tutorial will introduce common stream operations.

Reading from a Stream

Create an input.txt file with the following content:

tutorialpro.org official website: www.tutorialpro.org

Create a main.js file with the following code:

var fs = require("fs");
var data = '';

// Create a readable stream
var readerStream = fs.createReadStream('input.txt');

// Set the encoding to UTF8
readerStream.setEncoding('UTF8');

// Handle stream events --> data, end, and error
readerStream.on('data', function(chunk) {
   data += chunk;
});

readerStream.on('end',function(){
   console.log(data);
});

readerStream.on('error', function(err){
   console.log(err.stack);
});

console.log("Program executed successfully");

The output of the above code is:

Program executed successfully
tutorialpro.org official website: www.tutorialpro.org

Writing to a Stream

Create a main.js file with the following code:

var fs = require("fs");
var data = 'tutorialpro.org official website: www.tutorialpro.org';

// Create a writable stream to write to the file output.txt
var writerStream = fs.createWriteStream('output.txt');

// Write data to the stream with UTF8 encoding
writerStream.write(data,'UTF8');

// Mark the end of the file
writerStream.end();

// Handle stream events --> finish, error
writerStream.on('finish', function() {
    console.log("Write completed.");
});

writerStream.on('error', function(err){
   console.log(err.stack);
});

console.log("Program executed successfully");

This program will write the data from the data variable to the output.txt file. The output of the code is:

$ node main.js 
Program executed successfully
Write completed.

Check the content of the output.txt file:

$ cat output.txt 
tutorialpro.org official website: www.tutorialpro.org

Piping Streams

As shown in the image above, we compare files to buckets of water, and the content of the files to the water. By connecting two buckets with a pipe, water flows from one bucket to another, gradually copying the contents of a large file.

The following example reads the content of a file and writes it to another file.

Set the content of input.txt as follows:

tutorialpro.org official website: www.tutorialpro.org
Piping stream operation example

Create a main.js file with the following code:

var fs = require("fs");

// Create a readable stream
var readerStream = fs.createReadStream('input.txt');

// Create a writable stream
var writerStream = fs.createWriteStream('output.txt');

// Pipe the read and write operations
// Read from input.txt and write to output.txt
readerStream.pipe(writerStream);

console.log("Program executed successfully");

readerStream.pipe(writerStream);

console.log("Program execution completed");

Code execution result is as follows:

$ node main.js 
Program execution completed

Check the content of the output.txt file:

$ cat output.txt 
tutorialpro.org official website: www.tutorialpro.org
Example of pipe stream operation

Chaining Streams

Chaining is a mechanism to connect the output of one stream to another stream and create a chain of multiple stream operations. It is typically used for piping operations.

Next, we will use piping and chaining to compress and decompress files.

Create a compress.js file with the following code:

var fs = require("fs");
var zlib = require('zlib');

// Compress the input.txt file to input.txt.gz
fs.createReadStream('input.txt')
  .pipe(zlib.createGzip())
  .pipe(fs.createWriteStream('input.txt.gz'));

console.log("File compression completed.");

Code execution result is as follows:

$ node compress.js 
File compression completed.

After performing the above operation, we can see that a compressed file input.txt.gz of input.txt is generated in the current directory.

Next, let's decompress the file. Create a decompress.js file with the following code:

var fs = require("fs");
var zlib = require('zlib');

// Decompress the input.txt.gz file to input.txt
fs.createReadStream('input.txt.gz')
  .pipe(zlib.createGunzip())
  .pipe(fs.createWriteStream('input.txt'));

console.log("File decompression completed.");

Code execution result is as follows:

$ node decompress.js 
File decompression completed.