Local Remote Execution
What LRE actually is: a way to make your local toolchain and your remote workers the same toolchain, so both sides share one cache.
Who this is for: anyone deciding whether LRE is worth adopting, and anyone who has read about it and formed the wrong model of what it does. What you'll have at the end: the actual mechanism, the guarantee it makes and its exact qualifiers, and what it does not cover. Time: fifteen minutes.
Local Remote Execution is a way of building your toolchain with Nix such that the exact same toolchain binaries are used by your local build and by your remote workers. Because the toolchain is part of every action's digest, the two sides then produce identical action digests, and a cache entry written by one is a cache hit for the other.
That is the whole idea. It is worth stating what it is not, because the name invites a different reading.
LRE does not mean running a worker on your laptop
No local NativeLink worker exists in this design, and no container runs on your machine during a build. Your local build is an ordinary local Bazel build. What LRE changes is which compiler that build invokes. The toolchain container images are, in the source's own words, "never really instantiated"; they exist to have their environment extracted.
The mechanism, precisely
The trick is absolute paths.
A Nix-built toolchain lives at a path like
/nix/store/<hash>-clang-…/bin/clang, where the hash is derived from every
input that produced it. LRE generates Bazel toolchain configurations that
hardcode those absolute store paths: not /usr/bin/cc, not a PATH lookup,
the literal store path.
Then it arranges for that exact path to exist on both sides:
On the remote side, the worker image is built with nix2container and
its environment is composed entirely of /nix/store/… paths. The image is a
carrier for that closure rather than a conventional root filesystem.
On the local side, entering the Nix development shell realizes the same
derivations into your own /nix/store. This is what the commented-out PATH
lines in the generated lre.bazelrc are for: Bazel ignores them, but Nix
needs a reason to fetch the closure, and those lines are it. The
says so directly: "all paths printed in lre.bazelrc will be available on
your local system."
What is shared between local and remote is the store paths, not the image. Understanding that explains almost every question people have about LRE's edges.
The guarantee, with its qualifiers intact
The source states the promise carefully, and the qualifiers are load-bearing:
a framework to build, distribute, and rapidly iterate on custom toolchain setups that are transparent, fully hermetic, and reproducible across machines of the same system architecture […] This lets you reuse build artifacts with virtually perfect cache hit rate.
Both emphases are the source's meaning, not an editorial softening.
Reproducibility is claimed within an architecture, not across them; an
aarch64-darwin laptop and an x86_64-linux worker are not going to share
C++ artifacts, because they are not running the same binaries. And "virtually
perfect" acknowledges that the toolchain is one input among several, and LRE
only pins the toolchain.
LRE is explicitly experimental
The repository's own documentation opens with a warning that LRE is "highly
experimental and can be challenging to set up in downstream projects", and
a note that at the time of writing it "only works on x86_64-linux". The
Rust toolchains have since grown to four systems; the C++ toolchain has
not. Treat the setup cost as real.
What is pinned, and what is not
This is where careful reading pays off.
Not the image digest. Every container-image value in the generated
platforms is name:tag, where the tag is a Nix derivation hash, never a
sha256: content digest. The upstream config generator's digest-resolution
step was deliberately patched out, with the in-source justification that
"skipping this hash is not a generally safe practice and only works for us
because our images are reproducible."
Not a URL that gets pulled. container-image is a raw string matched by
the scheduler against what a worker advertises. NativeLink does not fetch it.
Two different conventions coexist in-tree (docker://lre-cc:<tag> for C++
and a ghcr.io/... reference for Rust), and a downstream user has to match
whatever string their own workers advertise. See
platform properties.
Not the image you actually deploy. Three distinct images are in play: a generator image, the image the platform config nominally names, and the worker image that actually runs. The repository warns about this in its own deployment manifests. The relationship is maintained by convention and by tag propagation, not by any mechanism that would catch a mismatch.
Not a single shared closure. The local closure is whatever the dev shell realizes; the worker image's closure is that plus extras the worker needs. They are compatible by construction, not identical.
Coverage
| Language | Status |
|---|---|
| Rust | Full toolchains across four Nix systems, stable and nightly |
| C/C++ | Full toolchain, x86_64-linux only |
| Java | An image and a generated runtime exist, but no Bazel toolchain is registered |
| Python, Go | Not provided |
Windows is not supported. On macOS the C++ path has no local toolchain at
all (the flake module only emits the C++ platform and toolchain flags on
Linux), so an lre-cc action can only run on a Linux worker carrying the
image; you get hermeticity there, not locality.
LRE is not an offline mode
It is tempting to read "local remote execution" as "remote execution without the network". It is not. LRE pins the toolchain; whether a given action runs locally or on a remote worker is a separate question decided by platform matching. On the platforms where the toolchain exists locally, builds are local, but that is a consequence, not the guarantee.
Why Nix, specifically
Because the design needs every tool the build invokes to be identified by a hash of everything that produced it, and to live at a path that encodes that hash. That is precisely what a Nix store path is.
/usr/bin/clang cannot be pinned. Its identity is whatever your package
manager last installed, it differs across machines that both call themselves
"Ubuntu 22.04", and no amount of hashing the binary recovers the headers and
libraries it will reach for. /nix/store/<hash>-clang-… has already answered
all of those questions by existing.
Everything else about LRE follows from that one property. The container
images, the generated Bazel toolchains, the lre.bazelrc, all of it is
plumbing to get those store paths onto both sides of the build.
For Nix itself, see Develop with Nix.
When it is worth it
| Situation | LRE? |
|---|---|
| Cache hits are poor between developers and CI on the same architecture | Yes; this is the case it was built for |
| You already build with Bazel and already use Nix | Yes; the marginal cost is small |
You want C++ hermeticity on x86_64-linux | Yes |
| Your fleet is mixed-architecture and you expected cross-arch sharing | No; the guarantee does not extend there |
| You need Python or Go toolchains pinned | Not yet |
| You are looking for sandboxing or isolation | No; different problem, see correctness and hermeticity |
| You want a quick win with a low setup cost | No; the source calls the setup challenging, and means it |
Where the exact truth lives
The flake-side and Bazel-side wiring, the regeneration commands, and the
downstream-project setup are in
local-remote-execution/.
For actually running it (the commands, and what is local versus remote once
you leave x86_64-linux) see
toolchains and hermeticity,
which is in the Remote execution section.
Common questions
The same ideas as commands you can run, in the Remote execution section.