Performance
I am sure you inspect (Ctrl + Shift + i) your website for styling and debugging but what about its performance? Ever visited the "network" section of the Dev Tools? If you haven't, let me explain to you why it is essential.
Let's create a simple program to fetch data from any static public API using by creating a server
Terminal
Open your terminal and run these lines one by one
mkdir caching
cd caching
npm init -y
touch index.js
npm install express cors axios
code . #This will open vscode in out project folder
Code
index.js
const express = require('express');
const cors = require('cors');
const axios = require('axios');
const app = express();
app.use(cors());
FETCH_URL = 'https://anapioficeandfire.com/api/characters/581';
app.get('/api',async (req, res) => {
try {
const {data} = await axios.get(FETCH_URL);
console.log("data added to cache");
res.json(data);
}
} catch (error) {
console.log(error);
res.json(error);
}
});
app.listen(5000);
Output
Notice on the right corner, these are derived from the Response Head. Look at the Time value. It means it took 1850ms to fetch this API back to Postman.
Do you know the reason? Study the architecture down below
Architecture
If you compare this to our project:
Client = Postman API
Main server = index.js
external server = server at 'FETCH_URL
database = source of that data
If time for process i is t(i), then:
t(1) + t(2) + t(3) + t(4) + t(5) + t(6) + t(7) + t(8) = 1850 ms
One request takes 1850ms. But imagine multiple concurrent requests, it will create more traffic to which you would have to implement load balancing and clustering strategies, which will add more processes in the architecture and delay to the response time.
Even if you achieve that, the minimum fetch time for each request will not go down approximately 1700ms. Is there any way to counter this? Absolutely!
Caching
Cache memory is a smaller segment which is a layer above Main memory and thus its access time is very fast. It has limited storage but has way lesser access time, (10x - 100x).
Imagine this:
You run a store. All your items are in the warehouse backyard. Would you bring the item from the warehouse to the store for each customer? I guess not. So, when you bring the item for the first time, bring some more and keep the extra in a box in the store so that next time when a customer comes, you won't need to run back and forth, only once when the box is empty again. This saves your time and energy.
Similarly caching can:
Reduce latency
Reduce traffic
Reduce server load
Improve user experience
Cached Architecture
Let's Implement this using Redis
Prerequisites
Redis store everything in (key: value) pairs, similar to JSON, except values are always string. It works on Linux, and MacOS. Therefore, for Windows you need to install Windows Subsystem for Linux (WSL).
wsl --install
You will have to set the username and password after it installs.
wsl #Run this in the terminal of yor project folder to run Linux
check:
run (one by one):
sudo apt-get update
sudo apt-get install redis-server
sudo apt-get install redis
You might be asked for your password from time to time but you will get used to it.
Now open your project in VScode.
Select WSL terminal and run:
redis-server
Great! Your Redis server has been set up and you can use this to cache data. Make sure server is running when you run your application.
If you face an issue or error saying PORT is in use, before this command, run:
sudo service redis-server stop
And retry "redis-server" command.
Basic Commands (Redis)
Open another WSL terminal and run:
redis-cli
These are some basic operations for Redis. If you want to learn more, you can learn from: Redis CLI | Docs
Terminal
Open your node terminal and run:
npm i redis
Code
index.js
const express = require('express');
const cors = require('cors');
const redis = require('redis'); // add redis
const axios = require('axios');
const app = express();
app.use(cors());
const redisClient = redis.createClient(); // initiate Redis client
FETCH_URL = 'https://anapioficeandfire.com/api/characters/581'; // External url
app.get('/api',async (req, res) => {
try {
await redisClient.connect(); // connect to redis server
const check = await redisClient.exists('temp'); // check if key exists
if (check){
console.log("data exists in cache");
const data = await redisClient.get('temp');
await redisClient.disconnect(); // important step
res.json(JSON.parse(data));
}else{ // if not, fetch data like normally and store key in cache
const {data} = await axios.get(FETCH_URL);
console.log("data added to cache");
await redisClient.setEx('temp',3600,JSON.stringify(data)); // stringify, as redis on accepts string values
// with setEx you can set an expiry time for a key which here is 3600s or 1 hr
await redisClient.disconnect(); // important step
res.json(data);
}
} catch (error) {
console.log(error);
return error;
}
});
app.listen(5000);
node index.js
Output
First fetch:
Second fetch:
Awesome! We reduced our response time by 100x. This is how effective caching can be. Let's fetch the results one more time:
Fetch 3:
It reduces even further! Though it will not necessarily reduce at every request but is still very impressive. However, this is not a correct way to implement caching. Let's optimize our code for better structure.
const express = require('express');
const cors = require('cors');
const redis = require('redis');
const axios = require('axios');
const app = express();
app.use(cors());
const redisClient = redis.createClient();
FETCH_URL = 'https://anapioficeandfire.com/api/characters/581';
// create a function which returns a promise for data
// by doing this we reduced the amount of code we have to write
// in case of multiple routes
// KEY is the key for which the cache will be stored
// CALLBACK is the function which will run if the key does not exist
const getOrSetCache = async (key, callBack) => {
return new Promise(async (resolve, reject)=>{
try {
await redisClient.connect();
const data = await redisClient.get(key); // check if key exists
if (data){ // will either be some value or null
await redisClient.disconnect();
console.log("data exists in cache");
return resolve(JSON.parse(data)); // here we parsed data
// beforehand so that in route handlers, we dont have to
}
const freshData = await callBack();
// callback function if key is not found
await redisClient.setEx(key,3600,JSON.stringify(freshData));
// get data from callback and set in cache
console.log('data added to cache');
await redisClient.disconnect();
//disconnect from redis-server, important step
return resolve(freshData);
} catch (error) {
await redisClient.disconnect();
return reject(error);
}
});
}
// See how our code for route handling is reduced !!
app.get('/api' , async (req,res) => {
try {
const fetchData = await getOrSetCache('temp', async () => {
const {data} = await axios.get(FETCH_URL); // normal fetch call
return data;
});
// here KEY is 'temp' and
// there are 2 lines of a CALLBACK function
res.json(fetchData);
} catch (error) {
console.log(error);
res.json(error);
}
});
app.listen(5000);
This doesn't affect the output, but improves code structure, readability and reusability.
Summary
Learn how to enhance your website's performance by implementing caching with Redis. This guide walks you through setting up a simple server using Node.js and Express, fetching data from an external API, and optimizing response times by caching data. By following these steps, you can significantly reduce latency (100x), traffic, server load, and improve user experience.
Thanks for reading it through! Leave a like or a comment if you found it interesting or have some doubts or suggestions on the topic. I am open to discussions :)