Olayinka Omole

Build Super Fast Apps in Node.js using Redis Cache

Build Super Fast Apps in Node.js using Redis Cache

When trying to optimise our applications, one of the first things we look to is caching. Caching involves storing data in a high performance store (many times temporarily) so such data can be retrieved faster at a later time. Redis is an efficient key-value data store that has become very popular for caching. One of the things that makes Redis a good choice is the number of data structures it supports — such as strings, hashes, lists, sets, and so on. This gives us some flexibility!

In this tutorial we will build a simple application in Node.js and speed it up by using Redis to cache data. Our app will be used to retrieve historical exchange rate data from an API and display it to users. You can view the entire code for this tutorial on GitHub, and the demo here.

Prerequisites

To follow along properly, you will need an understanding of JavaScript (ES6 syntax). You will also need to have Node.js and NPM installed. You can find installation instructions for both here.

I used the RedisGreen service from Manifold, so you can also optionally create an account there and set up the Redis service!

Setting Up

We will keep our file structure very simple. To get started, create a project folder, and initialise a new project within it with this command:

1npm init -y

Tip: The -y flag helps to create a package.json file with default values.

We will be using the Express framework, so we will need to install and save it as a dependency. We will also be making use of the node.js redis client for communicating with Redis, and axios for making our API calls. To install and save all 3 as dependencies:

1npm install express redis axios

Tip: If you use npm 5, you don’t need to specify the -S or --save flag to save as a dependency in your package.json file.

Next, we can create the files needed for our app. This is the file structure we will be using:

1├── currency-converter-app
2 ├── views
3 └── index.html
4 ├── package.json
5 └── server.js

Our app view will be located in the ./views folder, while our server-side logic will reside in the ./server.js file.

Building The App Backend

For starters, in our server.js file, we will initialise an express app, and require all our needed modules:

1// ./server.js
2
3const express = require('express');
4const path = require('path');
5const axios = require('axios');
6const redis = require('redis');
7const app = express();

Defining Routes and Responses

Next, we can define the basic routes for our application:

1// ./server.js
2...
3
4const API_URL = 'http://api.fixer.io';
5
6app.get('/', (req, res) => {
7 res.sendFile('index.html', {
8 root: path.join(__dirname, 'views')
9 });
10});
11
12app.get('/rate/:date', (req, res) => {
13 const date = req.params.date;
14 const url = `${API_URL}/${date}?base=USD`;
15
16 axios.get(url).then(response => {
17 return res.json({ rates: response.data.rates });
18 }).catch(error => {
19 console.log(error);
20 });
21
22});

From the above, we can see that our application has 2 main routes:

  • / — the base route which serves our app’s main view. It displays the index.html file from the views folder using sendFile.

  • /rate/:date — this retrieves the rate for a specified date, using the Fixer API. It returns the response from the API as JSON.

To start the app:

1// ./server.js
2...
3
4const port = process.env.port || 5000;
5
6app.listen(port, () => {
7 console.log(`App listening on port ${port}!`)
8});

Building The App View

Now, we can build a basic view for our app. We will be importing Bulma (my current choice of CSS framework) to take advantage of some preset styling and make our view look better:

1<!-- ./views/index.html -->
2
3<!DOCTYPE html>
4<html>
5
6<head>
7 <meta charset="utf-8">
8 <meta name="viewport" content="width=device-width, initial-scale=1">
9 <title>Currency Converter!</title>
10 <link rel="stylesheet" href="[https://cdnjs.cloudflare.com/ajax/libs/bulma/0.6.0/css/bulma.min.css](https://cdnjs.cloudflare.com/ajax/libs/bulma/0.6.0/css/bulma.min.css)">
11</head>
12
13<body>
14 <section class="section">
15 <div class="container">
16 <h1 class="title">Currency💰Converter!</h1>
17 <p class="subtitle">Get historical data about exchange rates quickly.</p>
18
19 <div class="content">
20 <blockquote>
21 BASE RATE IS <strong>USD</strong>.
22 GET HISTORICAL RATES FOR ANY DAY SINCE 1999.
23 THANKS TO <a href="[http://fixer.io/](http://fixer.io/)">FIXER.IO</a>
24 </blockquote>
25 </div>
26
27 <div class="columns">
28 <form id="rateForm">
29
30 <div class="column">
31
32 <div class="field">
33 <div class="control">
34 <input id="rateDate" name="date" class="input" type="date" required>
35 </div>
36 </div>
37
38 <div class="field">
39 <div class="control">
40 <button class="button is-primary is-stretched">
41 Get rates
42 </button>
43 </div>
44 </div>
45
46 </div>
47
48 </form>
49 </div>
50
51 <div class="notification is-link is-hidden" id="visits-count">
52 </div>
53
54 <!-- container for results -->
55 <div class="columns is-multiline is-mobile" id="results">
56 </div>
57
58 </div>
59 </section>
60</body>
61</html>

The above code contains basic markup for our app view. Next, we will write the JavaScript code to retrieve the rates from our backend, and display the rates. We import axios once again for our API calls:

1<!-- ./views/index.html -->
2...
3
4<!-- importing axios for API calls -->
5<script src="[https://unpkg.com/axios/dist/axios.min.js](https://unpkg.com/axios/dist/axios.min.js)"></script>
6
7<script>
8 const datePicker = document.querySelector("#rateDate");
9 const form = document.querySelector("#rateForm");
10 const submitButton = document.querySelector("#rateForm button");
11
12 form.onsubmit = e => {
13 e.preventDefault()
14 submitButton.classList.add("is-loading");
15
16 const date = datePicker.value;
17 const url = `/rate/${date}`;
18
19 axios.get(url).then(response => {
20 submitButton.classList.remove("is-loading");
21 showRates(response.data.rates);
22 }).catch(error => {
23 console.log(error);
24 });
25
26 }
27
28 function showRates(rates) {
29 let html = '';
30
31 for (rate in rates) {
32 html += '<div class="column is-one-quarter"><div class="card"><div class="card-content">';
33 html += `<p class="title">${rates[rate]}</p>
34 <p class="subtitle">${rate}</p>`;
35 html += '</div></div></div>';
36 };
37
38 document.querySelector("#results").innerHTML = html;
39 }
40
41 // IIFE - Executes on page load
42 (function () {
43 // set today's date as default date
44 datePicker.valueAsDate = new Date();
45 })();
46</script>
47
48</body>
49</html>

Now, we can test our app:

1node server.js

You should see this when you visit http://localhost:5000.

Base Currency Rates AppBase Currency Rates App

Caching Data Using Redis

Next, we will connect to a Redis store and start caching historical data, so we don’t need to retrieve data from the Fixer API everytime a new request is made for the same date.

So for example, if a request is made for the exchange rates as at the 1st of June 2000 once by Person A, when Person B requests the rates for the same date, our app fetches the data from the cache instead of hitting the API. This will make the response time for our app faster.

We will also store the number of requests that have been made for a particular date in our Redis store, and increment it every time a new request for the rates for that date.

Setting Up A Redis Service Using Manifold Optional

Manifold provides a RedisGreen service that allows you set up Redis and connect to it in less than 5 minutes. You can head over to manifold.co, sign up, create a project and provision your Redis service. It is very straight forward.

After setting up the Redis service, you can copy out the REDIS_URL credential as we will be using that in the next step.

You can also set up Redis locally by downloading it from the official downloads page.

Connecting To Redis

We will be communicating with Redis using the node_redis Node.js client. To connect to Redis:

1// ./server.js
2
3...
4
5// connect to Redis
6const REDIS_URL = process.env.REDIS_URL;
7const client = redis.createClient(REDIS_URL);
8
9client.on('connect', () => {
10 console.log(`connected to redis`);
11});
12client.on('error', err => {
13 console.log(`Error: ${err}`);
14});
15
16...

We are setting the REDIS_URL gotten from the previous step as an environment variable and passing it to redis.createClient().

Note: By default, redis.createClient() will use 127.0.0.1 as hostname and 6379 as port. If your setup is different, you can connect using the redis URL, or by supplying the hostname and port. For a list of connection options, you can check the official docs here.

We also use the connect and error events during development to check if a connection was made, and see any errors that exist. For a list of available events, visit here.

Saving and Retrieving Cache Data

Now, we can store and retrieve data from our Redis store. We will update the function that retrieves our rates so we can check for data in our cache first before making the API call, and also increment the count of requests for a the rates for a particular date.

Note: All Redis commands are exposed as functions on the client object. Read more here.

Updating the function that retrieves our rates:

1// ./server.js
2
3...
4app.get('/rate/:date', (req, res) => {
5 const date = req.params.date;
6 const url = `${API_URL}/${date}?base=USD`;
7
8 const countKey = `USD:${date}:count`;
9 const ratesKey = `USD:${date}:rates`;
10
11 client.incr(countKey, (err, count) => {
12 client.hgetall(ratesKey, function(err, rates) {
13 if (rates) {
14 return res.json({ rates, count });
15 }
16
17 axios.get(url).then(response => {
18 // save the rates to the redis store
19 client.hmset(
20 ratesKey, response.data.rates, function(err, result) {
21 if (err) console.log(err);
22 });
23
24 return res.json({
25 count,
26 rates: response.data.rates
27 });
28 })
29 .catch(error => {
30 return res.json(error.response.data)
31 }));
32
33 });
34 });
35});
36
37...

Note: The node redis API is entirely asynchronous. Callbacks need to be implemented to retrieve data.

In the code above, we specify two keys which we use for data storage:

  • countKey: Tracks the number of requests made for a particular date. It is incremented on every request to the same date.

  • ratesKey: Stores the rates data for a particular date.

Some redis commands we make use of:

  • incr: Increments the count of requests for a particular date.

  • hgetall: Retrieves the cached list of rates for a particular date.

  • hmset: Sets a list of rates for a date.

Promisifying Our Redis Commands Optional

Optionally, to get out of nesting too many callbacks (avoiding callback hell), and handle all our errors more easily, we can promisify node_redis with bluebird.

First, we install the bluebird package:

1npm i bluebird

Next:

1// ./server.js
2
3...
4
5const bluebird = require("bluebird");
6
7// make node_redis promise compatible
8bluebird.promisifyAll(redis.RedisClient.prototype);
9bluebird.promisifyAll(redis.Multi.prototype);

This will add Async to all node_redis functions, so we can modify our rates route like this:

1// ./server.js
2
3...
4
5app.get('/rate/:date', (req, res) => {
6 const date = req.params.date;
7 const url = `${API_URL}/${date}?base=USD`;
8
9 const countKey = `USD:${date}:count`;
10 const ratesKey = `USD:${date}:rates`;
11
12 let count;
13 client
14 .incrAsync(countKey)
15 .then(result => {
16 count = result;
17 return count;
18 })
19 .then(() => client.hgetallAsync(ratesKey))
20 .then(rates => {
21 if (rates) {
22 return res.json({ rates, count });
23 }
24
25 axios.get(url).then(response => {
26 client
27 .hmsetAsync(ratesKey, response.data.rates)
28 .catch(e => {
29 console.log(e)
30 });
31
32 return res.json({
33 count,
34 rates: response.data.rates
35 });
36 }).catch(error => res.json(error.response.data))
37
38 })
39 .catch(e => {
40 console.log(e)
41 });
42});

Finally, we will modify the JavaScript for our view to show the count of the number of requests for each date:

1<!-- ./views/index.html -->
2...
3
4<script>
5...
6
7 const visitsDiv = document.querySelector("#visits-count");
8
9 form.onsubmit = e => {
10 e.preventDefault()
11 submitButton.classList.add("is-loading");
12 visitsDiv.classList.add("is-hidden");
13
14 const date = datePicker.value;
15 const url = `/rate/${date}`;
16
17 axios.get(url).then(response => {
18 submitButton.classList.remove("is-loading");
19
20 // call function that shows the rates
21 showRates(response.data.rates);
22
23 // call function that shows count of requests for each date
24 showVisits(response.data.count, date);
25
26 }).catch(error => {
27 console.log(error);
28 });
29 }
30
31 function showVisits(count, date) {
32 let html = `
33 The rates as at <strong>${date}</strong>
34 have been checked <strong>${count}</strong> time(s).
35 `;
36 visitsDiv.classList.remove("is-hidden");
37 visitsDiv.innerHTML = html
38 }
39
40...
41</script>
42...

Running The App!

To run the app:

1REDIS_URL=YOUR_REDIS_URL node server.js

If you’re running Redis locally, you can simply do node server.js.

Final app making use of Redis CacheFinal app making use of Redis Cache

Running Your Application with the Manifold CLI Optional

It will be useful to note that Manifold also provides a CLI tool to help developers easily manage their services. It helps reduce the steps from provisioning to deployment to a minimum! You can get started on it here.

You can find a demo of the app we built in this tutorial here.


Add your API to our catalog and reach millions of developers.


Conclusion

In this tutorial, we have learned how to speed up our Node.js applications using Redis. We have:

  1. Built a Node.js app from scratch,

  2. Provisioned a Redis server using the Manifold RedisGreen service,

  3. Connected to the Redis server, and

  4. Saved data to, and retrieved data from the Redis store using the node redis client.

If your next question is ‘so when do I cache?’ — remember when considering optimisations, if you are still thinking about if you need it or not… you probably don’t need it.

There are still a bunch of improvements we can make to our base app using Redis commands and other techniques. The entire code is hosted on GitHub, you can take a look and play around with it. Do you have any questions or comments about Redis, Node.js and Caching? Let’s discuss in the comments!

Aiven Redis
Aiven RedisThe most complete cloud Redis on the market
Try Nowarrow_right
Stratus Background
StratusUpdate

Sign up for the Stratus Update newsletter

With our monthly newsletter, we’ll keep you up to date with a curated selection of the latest cloud services, projects and best practices.
Click here to read the latest issue.