Question: Multi-pod deployment + prepopulation leads to STALE cache on other pods

[tamadam]

I deployed a Next.js app using this cache handler to Kubernetes, and I’m seeing unexpected cache behavior when running multiple pods. I want to confirm if this is expected or if I’m missing something.

I’m running 3 pods of the same application in Kubernetes. Cache handler setup is based entirely on your example implementation.

Because I can’t prepopulate the cache during build time, I do it after the Next.js server starts. I configured a Kubernetes liveness probe that calls GET /api/health.

Inside that route I do:

• Load all routes to warm from an external API
• Hit my own server using fetch("http://localhost:3000/...")
• Only warm the cache once (using a Redis startup flag)

const routesToPreload = await fetch(
  my-routes-api
);

const pagesToWarm: string[] = await routesToPreload.json();

const preloadPromises = pagesToWarm.map((route) =>
  fetch(`http://localhost:3000${route}`, { next: { revalidate: 0 } })
    .then((res) => {
      if (!res.ok) {
        throw new Error(`Failed to fetch ${route}, status ${res.status}`);
      }
    })
    .catch((err) => {
      console.error(`Error loading route ${route}:`, err.message);
    })
);

await Promise.allSettled(preloadPromises);

My startup flag to ensure only one pod prepopulates:

export const startupKey = my_app_${process.env.GIT_HASH}:startup_flag;

Prepopulation logic

const redisClient = global.redisClientInstance;

if (redisClient) {
  const isInitialized = await redisClient.get(startupKey);

  if (!isInitialized) {
    console.info("Cache not initialized — prepopulating...");

    await redisClient.set(startupKey, "1");
    await prepopulateCache();

    console.info("Cache prepopulation complete.");
  } else {
    console.info("Cache already initialized, skipping.");
  }
}

Prepopulation runs only on 1 pod, which is expected.

But after that:

• If I access the app through Pod A → page is HIT
• Then access the same route through Pod B → STALE
• Then Pod C → STALE
• Then Pod A again → HIT

After each pod serves the page at least once (usually within 1 hour, my revalidate time), everything finally becomes HIT across all pods

It looks like every pod still performs a regeneration even though Redis was prepopulated.

Is this the normal behavior or how to solve this?

[BowlingX]

This is an issue within a distributed system, as there is no mutex locking mechanism in place that delegates the regeneration. I think this would be a great addition to the library (I guess could be implemented within redis (e.g. see https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/ -> https://github.com/alexpota/redlock-universal)), that only one pod will run the regeneration and all others always serve the response from the (stale) cache, while it is in "locked" state.

For the first case, you do not want to use your health checks for that, but either a job that runs after your deployment, which calls e.g. a /api/revalidate endpoint (with a token), see https://nextjs.org/docs/13/pages/building-your-application/data-fetching/incremental-static-regeneration#using-on-demand-revalidation or you use a startupProbe (see https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/)

Regarding startup probes though: You might also want to incorporate a mutex to not run them each time another pod scales up, but possibly also only on a single process to not delay startup

[AyronK]

This isn’t something we’re planning to work on soon because it would be difficult to maintain and test. You can still do it with a custom setup by putting the instrumentation code behind an endpoint that a job or orchestrator calls after deployment. That way, only one place handles the cache generation instead of every replica doing it when they start up. Or as mentioned in the previous comment decorate it with an extra lock to ensure only one process runs regeneration at a time.