Redis

Who this is for: anyone who wants several NativeLink processes to share one low-latency store, or who needs the scheduler's pub/sub subscriptions. What you'll have at the end: a Redis-backed store with concurrency and timeouts set so it survives a burst. Time: twenty minutes.

Redis sits between the filesystem and object storage: it is shared across machines like a bucket, and roughly a millisecond away like a local disk. What it is not is cheap per byte. Use it for small, hot, shared objects (the Action Cache, scheduler state, the index side of a dedup store) and put the bulk CAS somewhere that prices storage by the gigabyte.

The recipe

stores: [
  {
    name: "AC_MAIN_STORE",
    redis_store: {
      addresses: ["redis://127.0.0.1:6379/"],
      mode: "standard",
      key_prefix: "nativelink:ac:",
    },
  },
],

addresses is the only required field. The URL carries everything Redis needs: scheme, optional credentials, host, port and database index. redis://username:password@redis-server-url:6380/99 selects port 6380 and database 99. Keep credentials out of the file by writing redis://${REDIS_USER}:${REDIS_PASSWORD}@redis:6379/ and exporting the variables; see The config file.

Standalone, sentinel or cluster

mode takes standard (the default), sentinel or cluster.

The store connects to the sentinel, asks it for the current master, and re-resolves the master when a write hits a demoted replica or the connection drops. Passing more than one address in standard or sentinel mode is rejected at startup.

key_prefix namespaces one Redis instance across several stores, and matters more here than with object storage: a Redis instance is much more likely to be shared with something else you run. Give the CAS, the AC and the scheduler distinct prefixes.

Sizing it so it doesn't time out

Redis is single-threaded per node, so the settings that matter are the ones bounding how much work you can have in flight.

max_client_permits is the most useful lever. It exists precisely to stop timeouts caused by many, many inflight actions: when the ceiling is reached, work queues instead of piling onto Redis and blowing through command_timeout_ms. Lower it if you see timeouts under load; that is almost always the right first move.

Peak memory in the client process is roughly read_chunk_size × max_chunk_uploads_per_update × concurrent uploads. Working backwards from available memory, divide by about ten to leave room for everything else the process is doing.

max_count_per_cursor (default 1500) bounds how many entries one cursor returns; it exists to reduce thundering-herd effects when many workers hit the provisioner at once.

Where Redis actually earns its place

Rather than as a bulk CAS, Redis is usually the fast or index half of a composition:

{
  name: "CAS_MAIN_STORE",
  dedup: {
    index_store: {
      redis_store: {
        addresses: ["redis://redis:6379/"],
        key_prefix: "nativelink:index:",
      },
    },
    content_store: {
      experimental_cloud_object_store: {
        provider: "aws",
        region: "us-east-1",
        bucket: "my-nativelink-cas",
        key_prefix: "cas/",
      },
    },
  },
},

The index is small, hot and shared by every process; the content is large, cold and cheap per byte. Compose stores covers the rest of these shapes.

Steps

1. Provision Redis with enough memory for what you intend to keep, and decide its eviction policy on the Redis side, because NativeLink does not manage it.

2. Add the store with addresses, the right mode, and a key_prefix that won't collide with anything else using that instance.

3. Reference credentials through the environment rather than writing them into the URL in the config file.

4. Run a build, then run it again from a second machine pointed at the same Redis, to confirm the store really is shared.

5. Load-test it and watch for command_timeout_ms errors. If you see them, lower max_client_permits before touching anything else.

When it doesn't work