Instances and shared memory

One instance, many requestsCopy Anchor Link

An app instance is one running copy of your server. Nitroship can send many requests to the same instance at the same time. It does not start a fresh container for every request.

In a Node server process, a module-level variable or a global belongs to the process, not to one request. A Map created outside your request handler can be read and changed by other requests handled by that process, including later requests. If your app starts several processes, each process has its own module-level memory.

Separate instances, including instances in different regions, do not share live memory. Updating a Map in one instance does not update the others.

Good uses for local memoryCopy Anchor Link

Use local memory for work you can safely repeat when the cache is empty:

• Expensive, non-user-specific data, such as configuration, imported public keys for verifying JWT signatures, or compiled templates.
• Database and HTTP connection pools that can open new connections when needed.

Keep an authoritative source elsewhere. A cache miss must load from that source rather than assume an earlier request has already filled the cache.

Example: reuse an imported JWT verification keyCopy Anchor Link

This Node/TypeScript module fetches a PEM public key from your trusted key service and imports it once. Concurrent requests for the same URL share the in-flight Promise, and later requests reuse the imported key. A failed load is removed so the next call can try the source again.

import { createPublicKey, type KeyObject } from "node:crypto";

const keys = new Map<string, Promise<KeyObject>>();

export function getVerificationKey(keyUrl: string): Promise<KeyObject> {
  const cached = keys.get(keyUrl);
  if (cached) return cached;

  const loading = (async () => {
    const response = await fetch(keyUrl);
    if (!response.ok) throw new Error(`Key load failed: ${response.status}`);
    return createPublicKey(await response.text());
  })().catch((error) => {
    keys.delete(keyUrl);
    throw error;
  });

  keys.set(keyUrl, loading);
  return loading;
}

Call this with a small, configured set of trusted public-key URLs, not arbitrary URLs from a request. Keep the caller's token and authentication result in request-local variables; only the reusable public key goes in this cache.

The key service remains the source of truth. This example keeps successful entries for the process lifetime. If keys can rotate, add expiration or explicit invalidation; if the set of keys can grow, add an entry limit and eviction. A cache must not keep using an outdated key just because it is already in memory.

Scaling and memory lifetimeCopy Anchor Link

Compute settings describe how many instances can run and the resources of each one:

With minInstances: 0, you have not requested a minimum number of running instances. When traffic is idle, Nitroship can freeze, checkpoint, or remove instances; the platform can also retain a frozen warm instance. Snapshot/restore may preserve memory, but it is not durable storage.

Design for an empty cache after an app-process restart, a new deploy or redeploy, scale-down, or a move to another node. A crash or out-of-memory termination can also discard process memory. Never make successful requests depend on a previous instance's cache surviving.

An unbounded cache can consume the tier's memory limit and cause an out-of-memory failure. Bound both the number of entries and their size, and leave room for request handling and connection pools.

See Managing deployments for applying compute settings, and the Compute settings API for available regions and tiers.

Do not put request state in globalsCopy Anchor Link

Never keep a current user ID, authentication context, token, request body, or response object in a module-level variable or global. Another request can replace or read it, causing one user's data to leak into another user's response.

Shared mutable objects also need care. Even in a single Node process, asynchronous handlers can interleave at await: two requests can read the same value before either writes its update. Keep request-specific values in the handler's local variables and arguments. For cross-request updates that must be correct, use database transactions or atomic operations in an appropriate shared store, not an in-memory read-modify-write sequence.

Where shared and durable state belongsCopy Anchor Link

Use an external database for records, sessions, and other authoritative application state. Use an external Redis service or another suitable shared store when all instances need the same cache, rate-limit counter, or coordination state; choose persistence and atomic operations to match the job. Store uploaded files in object storage rather than process memory.

Pass connection settings and credentials through Environment variables. A local Map can sit in front of a shared source to save work, but your application must still work when that Map is empty.