CMake with recc

Read time: ~5 minutes. Hands-on time: another ~5 if you have Docker and a package manager handy; longer on Linux if you build recc from source.

This tutorial gets you from zero to a CMake build hitting a NativeLink remote cache. Works on Linux and macOS (tested on macOS Apple Silicon; Linux x86_64 follows the exact same commands).

We use recc, a small Remote Execution v2 client originally from Bloomberg, as a CMake compiler launcher. We run it in cache-only mode: compiles still happen on your machine, only the outputs travel through NativeLink. That keeps the setup tiny while still giving you cross-machine and cross-branch caching.

Prerequisites

• Docker
• CMake 3.16+
• A C/C++ compiler (clang or gcc)

1. Start NativeLink

curl -O https://raw.githubusercontent.com/TraceMachina/nativelink/v1.6.5/nativelink-config/examples/basic_cas.json5

docker run -d --name nativelink \
  -v $(pwd)/basic_cas.json5:/config \
  -p 50051:50051 -p 50061:50061 \
  ghcr.io/tracemachina/nativelink:v1.6.5 config

The image is multi-arch (x86_64 and ARM64) as of v1.6.0, so it runs natively on Linux and Apple Silicon alike.

Confirm it came up:

docker logs nativelink 2>&1 | grep "Ready, listening on 0.0.0.0:50051"

2. Install recc

brew install recc

sudo apt-get update
sudo apt-get install -y cmake g++ gcc git googletest libabsl-dev \
  libgmock-dev libgrpc++-dev libprotobuf-dev protobuf-compiler-grpc \
  libssl-dev pkg-config uuid-dev nlohmann-json3-dev libcurl4-openssl-dev

git clone https://gitlab.com/BuildGrid/buildbox/buildbox.git
cd buildbox && mkdir build && cd build
cmake .. && make -j"$(nproc)"
sudo make install

3. Create the example

Drop these two files into a fresh project directory and cd into it. You can also copy them from

in the repo.

# CMakeLists.txt
cmake_minimum_required(VERSION 3.16)
project(hello LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
add_executable(hello main.cpp)

// main.cpp
#include <iostream>

int main() {
    std::cout << "Hello, NativeLink!\n";
    return 0;
}

4. Point recc at NativeLink

export RECC_SERVER=localhost:50051
export RECC_INSTANCE=main
export RECC_CACHE_ONLY=1
export RECC_CACHE_UPLOAD_LOCAL_BUILD=1

What each one does:

• RECC_SERVER: gRPC endpoint for the CAS / Action Cache.
• RECC_INSTANCE=main: must match an instance_name in your NativeLink config. basic_cas.json5 exposes both "" and "main"; recc defaults to "" unless it was built with a different DEFAULT_RECC_INSTANCE, so setting it explicitly removes the guesswork.
• RECC_CACHE_ONLY=1: if there's no cached result, build locally instead of waiting on a remote worker.
• RECC_CACHE_UPLOAD_LOCAL_BUILD=1: push the result of a local compile back up so the next build (or the next teammate) gets a hit. It only has an effect together with RECC_CACHE_ONLY.

5. Build it

RECC=$(command -v recc)

cmake -B build -S . \
  -DCMAKE_C_COMPILER_LAUNCHER=$RECC \
  -DCMAKE_CXX_COMPILER_LAUNCHER=$RECC

cmake --build build
./build/hello

You should see Hello, NativeLink!.

6. Confirm the cache is live

Wipe the build directory and rebuild. This time the compile output should come straight out of NativeLink:

rm -rf build

cmake -B build -S . \
  -DCMAKE_C_COMPILER_LAUNCHER=$RECC \
  -DCMAKE_CXX_COMPILER_LAUNCHER=$RECC

RECC_LOG_LEVEL=info cmake --build build

You should see an Action Cache hit line for the compile, like the captured output below:

[ 50%] Building CXX object CMakeFiles/hello.dir/main.cpp.o
[INFO] Action Cache hit for [cf20db769511312cf9c66cb3ac1b82512bf4572b42b99f9a73b13a00e90e14d3/199]
[100%] Linking CXX executable hello
[100%] Built target hello

That's NativeLink serving the compile result instead of the compiler running again. Linking still happens locally. recc only ships compile actions, not link actions, by default.

Teardown

docker stop nativelink && docker rm nativelink

FAQ

Going further

• Point RECC_SERVER at a shared NativeLink deployment to share the cache across your team.
• Drop RECC_CACHE_ONLY once you have remote workers configured to actually offload compile work. See Remote execution.
• For PCH, LTO, and other gnarly C++ build features at scale, read Reid Kleckner's writeup.