Local testing

Keep a NativeLink cluster on your own machine to test config changes, reproduce scheduler behaviour, and prove a client works, in the two shapes the repo itself uses.

Who this is for: anyone changing a config, debugging a client, or reproducing something a shared cluster did. What you'll have at the end: a local cluster in whichever of the two shapes matches what you're testing, and a reliable way to tell what actually ran where. Time: ten minutes for the single-process shape.

Before you start

A cache serving hits to your build. If you don't have one yet, start with Getting started.

Your first remote action gets you one remote build. This page is what you do afterwards: keep a cluster around as a test fixture, and know which shape to run for what you're testing.

This is not LRE

Nothing here has a toolchain opinion; it uses whatever is on your PATH, and needs no Nix. That's deliberate: it isolates protocol and config problems from toolchain problems. Toolchains and hermeticity is the other axis.

Which shape to run

You're testing	Run	Why
A client's flags, a store config, scheduler matching	Single process	Fastest loop; one log to read
Cross-process wiring, `grpc` stores, separate CAS and scheduler	The compose split	Same topology as production, and what CI exercises
Toolchain hermeticity	Neither; see Toolchains	Different problem

Shape 1: one process

--remote_cache and --remote_executor both point at 127.0.0.1:50051. 50061 exists only so the worker can register with the scheduler; Bazel never talks to it.

nativelink ./nativelink-config/examples/local_rbe_self_test.json5

INFO nativelink: Ready, listening on 0.0.0.0:50051
INFO nativelink: Ready, listening on 0.0.0.0:50061
INFO nativelink_worker::local_worker: Worker registered with scheduler, worker_id: 1f175824-...

local_rbe_self_test.json5

The interesting parts of that config, for when you start editing it:

{
  stores: [
    { name: "CAS_MAIN_STORE", filesystem: { /* ... */ } },
    { name: "AC_MAIN_STORE", filesystem: { /* ... */ } },
    {
      // The worker's fast tier and the client-facing CAS share one store,
      // so anything the worker produces is immediately visible to Bazel.
      // `fast` must be a filesystem store: the worker hardlinks out of it
      // to build each action's sandbox.
      name: "WORKER_FAST_SLOW_STORE",
      fast_slow: {
        fast: { filesystem: { /* ... */ } },
        slow: { ref_store: { name: "CAS_MAIN_STORE" } },
      },
    },
  ],
  schedulers: [{
    name: "MAIN_SCHEDULER",
    simple: { supported_platform_properties: { cpu_count: "minimum" /* ... */ } },
  }],
  workers: [{
    local: {
      worker_api_endpoint: { uri: "grpc://127.0.0.1:50061" },
      cas_fast_slow_store: "WORKER_FAST_SLOW_STORE",
      upload_action_result: { ac_store: "AC_MAIN_STORE" },
      platform_properties: { cpu_count: { values: ["1"] } /* ... */ },
    },
  }],
  servers: [
    { name: "local", /* cas · ac · bytestream · execution · capabilities on :50051 */ },
    { name: "worker_api", /* worker_api · health on :50061 */ },
  ],
}

Then:

bazel test \
  --remote_cache=grpc://127.0.0.1:50051 \
  --remote_executor=grpc://127.0.0.1:50051 \
  --remote_default_exec_properties=cpu_count=1 \
  //your:target

Set both flags, even to the same address

It's tempting to assume --remote_executor alone is enough, since the executor reads and writes the CAS anyway. It isn't. Without an explicit --remote_cache, Bazel calls Execute with an action digest it never uploaded, and the server correctly rejects it: FAILED_PRECONDITION: Action ... is missing from CAS.

Shape 2: the compose split

When you need separate processes (a CAS process and a scheduler process talking over grpc stores, which is what a real deployment looks like), the repo ships that too, and its own integration tests run against it.

cd deployment-examples/docker-compose
docker compose up

This publishes the CAS on 50051 (plus a TLS listener for the same services on 50071) and the scheduler, with the execution and AC services, on 50052. The scheduler's worker API on 50061 is not published from the scheduler container at all, so it is only reachable inside the compose network; the worker reaches it by service name.

That port split is why the repo's .bazelrc reads the way it does:

build:self_test --remote_cache=grpc://127.0.0.1:50051

build:self_execute --remote_executor=grpc://127.0.0.1:50052
build:self_execute --remote_default_exec_properties=cpu_count=1
build:self_execute --platform_suffix=self-execute

.bazelrc

bazel test --config self_test --config self_execute //:dummy_test

These configs are for the compose split, not the single-process example

self_execute points at 50052. If you're running local_rbe_self_test.json5, which puts everything on 50051, these two --config flags will not work; use the explicit flags from shape 1 instead. This mistake is common because both live in the same repo.

Production configuration explains the split topology this is a miniature of.

Verify what actually ran

Wall-clock time is a bad signal and Bazel's summary line is only a rough one. Ask the build event log directly:

bazel test \
  --config self_test --config self_execute \
  --nocache_test_results \
  --build_event_json_file=/tmp/bep.json \
  //:dummy_test

jq --slurp -r '.[] | select(.id.testResult.label=="//:dummy_test")
  | .testResult.executionInfo.strategy' /tmp/bep.json

remote

simple_remote_execution_test.sh

That is exactly what CI asserts, twice: once cold, then again after bazel clean to confirm the second run is served remotely rather than from Bazel's local disk cache.

For per-action detail rather than per-test, --execution_log_json_file records the runner for every spawn.

You did it right if

First run: Bazel's INFO: N processes: ... summary line counts the genrule and the test as remote, with no remote cache hit.
After bazel clean and a re-run: the genrule that produces dummy_test.sh shows up as remote cache hit while the test itself re-executes remotely, because --nocache_test_results is set.
jq reports remote both times.