Build Time Data Caching in Next.js

This blog introduces a solution to the problem of redundant API calls in Next.js applications. By implementing a caching mechanism, developers can pre-fill a cache folder with necessary data before the build process, reducing the number of redundant API calls from thousands to just one. The blog provides step-by-step instructions on creating and utilizing the cache folder, offering a straightforward approach to optimizing Next.js applications with shared data requests.

1. Problem

If you're using ISR, you may encounter a situation where a request is shared across multiple pages. For instance, imagine having a header and footer that are configured via a headless CMS. In this case, you would need to fetch the layout configuration's API for every route.

Consider the following workaround:

// pages/post/[slug].tsx

export const getStaticPaths: GetStaticPaths = async function ({ locales }) {
  const products = await ProductServices.list();

  const paths = products?.map(({ slug }) => ({ params: { slug } })) || [];

  return { paths, fallback: "blocking" };
};

export const getStaticProps: GetStaticProps = async function ({ params }) {
  const { slug = "" } = params;

  const [product, layoutConfig] = await Promise.all([
    ProductServices.detail({ slug }),
    LayoutServices.detail(),
  ]);

  if (!product) return { notFound: true };

  return {
    props: { product, layoutConfig },
    revalidate: Intervals.Week,
  };
};

// services/layout/detail.ts

import Http from "@utils/Http";
import type ILayoutDetail from "@types/layout";

export default async function layoutDetail() {
  const response = await Http.get<ILayoutDetail>("LAYOUT_DETAIL_API_URL_HERE");

  return response.data;
}

Until now, we have statically generated pages that make API calls during the build time. As long as a page is generated statically, the number of APIs involved doesn't matter because the end-user receives only an HTML file. However, it becomes crucial when the same API is called with the same output on all these pages. If, for example, we have 10,000 posts, we would end up calling the layout config API 10,000 times. While backend developers might implement caching mechanisms in such cases, as the Persian saying goes, "The door to the pot is open, where is the integrity of the cat?"

Therefore, we need to implement a small caching mechanism ourselves to reduce 10,000 API calls to just 1.

2. Solution

The solution is straightforward and can be divided into the following steps:

2.1. Creating the Cache Folder

First, we need to create an empty folder called cache that will contain all the caches. Ensure you add this folder to the .gitignore file.

2.2. Pre-Filling the Cache Before Build

Before running the build command, we should populate our cache with the necessary data. This can be achieved with a script written in any language. Options include Bash, Python, and Golang. While Python and Golang offer cross-platform compatibility, Bash suffices for this simple task, requiring only a terminal in Linux or Git Bash in Windows.

The following shell script fetches the layout config API and stores its response in the cache/layout-detail.json file:

# scripts/cache/layout.sh

mkdir cache

curl -v -o cache/layout-config.json https://api.xxx.com/layout-config

We also need a cache restoration script to clear the cache after the build:

# scripts/cache/restore.sh

rm -rf cache

Make sure to add this API to your package.json build script:

// package.json
{
  "scripts": {
    "build": "bash scripts/cache/layout-detail.sh && next build && bash scripts/cache/restore.sh"
  }
}

What if we have multiple caches? How can we prevent them from chaining one after another? We can create a build.sh file and chain all the cache scripts together:

# scripts/ops/build.sh

(
    bash scripts/cache/layout.sh
    bash scripts/cache/other.sh
    bash scripts/cache/another.sh
) &&
    next build &&
    bash scripts/cache/restore.sh

Processes that operate independently of each other can be executed sequentially by using the && operator. Conversely, processes that rely on each other should be chained together concurrently using the ; operator.

2.3. Utilizing Cached Responses

Now, let's leverage the generated cache. In the above example, we had a services/layout/detail.ts file responsible for fetching data from the headless CMS. We need to make some changes to it so that if a cache for that API already exists, it returns the cached response:

// services/layout/detail.ts

import fs from "fs";
import Http, { type HTTPResponse } from "@utils/Http";
import type ILayoutDetail from "@types/layout";

const CACHE_PATH = "./cache/layout-detail.json";

export default async function layoutDetail() {
  const cacheExists = fs.existsSync(CACHE_PATH);

  let response: HTTPResponse<ILayoutDetail>;

  if (cacheExists) {
    response = JSON.parse(fs.readFileSync(CACHE_PATH, "utf-8"));
  } else {
    response = await Http.get<ILayoutDetail>("LAYOUT_DETAIL_API_URL_HERE");
  }

  return response.data;
}

That's it! It's as simple as that!

Conclusion

In conclusion, implementing a caching mechanism for shared requests in a static site can greatly improve build duration and reduce the number of API calls. By pre-filling a cache folder with the necessary data before the build process, we can avoid making redundant API requests for each page. With a simple caching solution, such as the one described above, we can significantly decrease the number of API calls from thousands to just one.