Migrate from another REAPI cache

Who this is for: anyone running Buildbarn, Buildfarm, bazel-remote or a hosted REAPI cache who wants to move to NativeLink without a flag day. What you'll have at the end: NativeLink serving your builds, backed by the old cache, with a cutover you choose rather than one you're forced into. Time: an hour to set up; days to weeks of running both, at your pace.

The instinct is to copy the data across. Don't. A remote cache is not a database: every entry in it is derived, reproducible, and has a key that is a hash of its own contents. Copying terabytes to preserve something your build can regenerate is work you can skip.

The move that works instead is to put NativeLink in front of the old cache and let it read through. Builds hit NativeLink; anything it doesn't have, it fetches from the old cache and keeps. Your hit rate never drops, the new cache warms itself on real traffic, and the old one is never written to again, so if you need to go back, it is exactly as you left it.

The diode

A fast_slow store whose slow half points at the old cache, marked read-only:

stores: [
  {
    name: "OLD_CACHE",
    grpc: {
      instance_name: "",
      store_type: "cas",
      endpoints: [
        {
          address: "grpcs://old-cache.example.com:443",
          tls_config: { use_native_roots: true },
        },
      ],
      rpc_timeout_s: 0,
    },
  },
  {
    name: "CAS_MAIN_STORE",
    fast_slow: {
      fast: {
        filesystem: {
          content_path: "/data/cas/content",
          temp_path: "/data/cas/tmp",
          eviction_policy: { max_bytes: 500000000000 },
        },
      },
      slow: { ref_store: { name: "OLD_CACHE" } },
      slow_direction: "read_only",
    },
  },
],

slow_direction: "read_only" is the whole trick, and it is a documented use of the field rather than a clever misuse: the source calls this shape a diode. Reads fall through to the old cache when NativeLink misses. Writes stop at NativeLink. The old cache is a source and never a destination.

The tls_config block is required on any grpcs:// address; the use_native_roots form is right for a hosted cache with a public certificate, and ca_file is the alternative for a private CA (TLS and authentication has both).

Do the same for the action cache with a second grpc store at store_type: "ac". Action results are the entries that actually decide your hit rate; the CAS blobs they reference come along behind them.

Leave rpc_timeout_s at 0. It disables the per-RPC deadline, which matters because pulling a multi-gigabyte blob through the diode on a cold cache can take longer than any timeout you would have guessed. Dead connections are still caught by the HTTP/2 and TCP keepalives.

What crosses the boundary and what doesn't

Digests must match. If your old cache holds SHA-256 entries and your builds now ask for BLAKE3, nothing is found, not because of a bug but because the key of a blob is its hash, and you are asking a different question. The digest function is chosen by the client (Bazel's --digest_function, SHA-256 unless you changed it); NativeLink's global.default_digest_hash_function, described in Stores, only fills in when a client does not say. Keep the client, NativeLink and the old cache on the same function. This is the one incompatibility that cannot be worked around.

Instance names may need rewriting. instance_name on the grpc store replaces the instance name on every proxied call, so an old cache that expects main while your clients send an empty instance is a one-field fix rather than a client migration.

Older backends want older resource names. Modern REAPI resource names look like {instance}/blobs/{digest_function}/{hash}/{size}. Buildbarn before v0.3 and other older servers expect the original {instance}/blobs/{hash}/{size}, and sending them the modern form produces InvalidArgument: Unsupported digest function. Set use_legacy_resource_names: true when you see that.

Credentials come with you. A hosted cache that wants a token takes it via headers, and one that wants each developer's own token takes forward_headers to pass the client's credential upstream:

grpc: {
  instance_name: "",
  store_type: "cas",
  endpoints: [
    {
      address: "grpcs://old-cache.example.com:443",
      tls_config: { use_native_roots: true },
    },
  ],
  headers: { authorization: "Bearer my-static-token" },
},

Header values are sent verbatim: they are not ${VAR}-expanded like most string fields, so the token lives in the config file and the file must be protected accordingly.

Remote execution does not transfer. Workers, platform properties and queue state are per-system. A read-through migration moves your cache; the execution side is a separate build-out, and Remote execution is where it happens.

Anything vendor-specific doesn't transfer either. If you depend on a feature of your current system outside the REAPI surface (a custom HTTP interface, a proprietary authorization model, a bespoke admin API), that dependency is what you should be examining before the cache bytes.

Run both while you cut over

Nothing forces a single switch date. Give the two caches separate config groups in .bazelrc, so a build can pick:

build:oldcache --remote_cache=grpcs://old-cache.example.com:443
build:nativelink --remote_cache=grpcs://nativelink.example.com:50051
build --config=nativelink

Move CI first, because it is the highest-volume traffic and warms the new cache fastest, and because its failures are visible immediately rather than reported by a person mid-afternoon. Developers follow once CI has been steady for a week.

Steps

1. Confirm the digest function your builds use, and set NativeLink's default_digest_hash_function to match. Everything downstream depends on this being right.

2. Stand up NativeLink normally, with its own filesystem or object store, and check that a build against it works before the old cache is involved at all.

3. Add the diode: grpc stores for CAS and AC pointing at the old cache, wrapped as the slow half of fast_slow with slow_direction: "read_only".

4. Point CI at NativeLink and watch the hit rate. It should start near what the old cache gave, because misses are being served through the diode, and it should stay there.

5. Move developers, then wait. Give it long enough that a full build's worth of actions has passed through, typically a week or two of normal work.

6. Remove the diode once misses are no longer being answered by the old cache, then decommission it.

When it doesn't work