Servers and services

Who this is for: anyone deciding what a NativeLink process should expose, and on which port. What you'll know at the end: what a listener is, what each of the eleven services does, how instance_name partitions a cache, and why the worker API must not share a port with anything else. Time: thirty minutes.

A server is a listener plus a set of services

The servers array is the only entry point into a NativeLink process. Each entry binds one socket and decides what is reachable on it:

servers: [
  {
    name: "public",
    listener: {
      http: {
        socket_address: "0.0.0.0:50051",
      },
    },
    services: {
      cas: [{ instance_name: "main", cas_store: "CAS_MAIN_STORE" }],
      ac: [{ instance_name: "main", ac_store: "AC_MAIN_STORE" }],
      bytestream: [{ instance_name: "main", cas_store: "CAS_MAIN_STORE" }],
      capabilities: [{ instance_name: "main" }],
    },
  },
],

Four things are going on there, and each is worth naming.

name is documented as a label for telemetry and logs, defaulting to the server's index in the array. Set it; it's the cheapest documentation in the file.

listener currently has exactly one variant, http, which serves HTTP/HTTP2 and therefore gRPC. socket_address is parsed as a Rust SocketAddr, so it must be an IP literal plus port: 0.0.0.0:50051 to bind every IPv4 interface, [::]:50051 for IPv6, 127.0.0.1:50051 for loopback only. A hostname or a bare :port is rejected as an invalid address. TLS, HTTP/2 tuning, and compression are options on the listener, not on the services.

services is optional in the schema, but the binary refuses to start a server without it ('services' must be configured), so in practice every server entry carries one.

Each service entry names a store or a scheduler as a string, the name from the stores or schedulers array. This is where the wiring described in the section overview actually happens.

The eleven services

A cache-only deployment needs the first four. Adding execution adds execution and worker_api. The rest are opt-in.

capabilities is the one that surprises people: it carries almost no config, but Bazel queries it before doing anything else and fails the build if the call is not served, with an error about querying capabilities rather than one naming a missing service. And if capabilities has no remote_execution block, the response carries no execution_capabilities, so the client is told this endpoint caches but does not execute, which is correct for a cache-only Getting started deployment and a silent bug once you add Remote execution.

capabilities: [
  {
    instance_name: "main",
    remote_execution: { scheduler: "MAIN_SCHEDULER" },
  },
],

Instance names

Most services are declared as an array, and each element carries an instance_name. That field is a routing key: it selects which store (or scheduler) serves a request. The instance name is not part of the storage key, so two instance names pointing at different stores are fully isolated, and two pointing at the same store share every blob.

cas: [
  { instance_name: "", cas_store: "CAS_MAIN_STORE" },
  { instance_name: "main", cas_store: "CAS_MAIN_STORE" },
],

That pattern (the same store registered twice, once under the empty name and once under main) appears throughout the shipped examples, and it relies on the sharing: clients differ in what they send, some omit the instance name entirely, and registering both means either kind of client works, against one cache, without a config change on their side.

Pick one name per logical environment. main for the production cache, <repo-name> for per-repo isolation, <something>-experiments for a sandbox that shouldn't pollute the real cache, each pointed at its own store. Every client targeting a given cache must use the same one.

The port split

The worker_api service is different from every other service on this page, and the difference is a security boundary rather than a design preference.

A client calling the CAS can upload and fetch blobs. A worker connected to the worker API is handed actions to execute and reports their results as authoritative. Those are not the same permission, and the worker API has no authentication of its own to distinguish them.

The rule: worker_api gets its own listener, on its own port, with no other service sharing it. The shipped examples put the public API on :50051 and the worker API on :50061, and the second port is the one you never expose beyond the cluster:

servers: [
  {
    name: "public",
    listener: { http: { socket_address: "0.0.0.0:50051" } },
    services: {
      cas: [{ instance_name: "main", cas_store: "CAS_MAIN_STORE" }],
      ac: [{ instance_name: "main", ac_store: "AC_MAIN_STORE" }],
      bytestream: [{ instance_name: "main", cas_store: "CAS_MAIN_STORE" }],
      execution: [{
        instance_name: "main",
        cas_store: "CAS_MAIN_STORE",
        scheduler: "MAIN_SCHEDULER",
      }],
      capabilities: [{
        instance_name: "main",
        remote_execution: { scheduler: "MAIN_SCHEDULER" },
      }],
    },
  },
  {
    name: "private_workers_servers",
    listener: { http: { socket_address: "0.0.0.0:50061" } },
    services: {
      worker_api: { scheduler: "MAIN_SCHEDULER" },
      admin: {},
      health: {},
    },
  },
],

Note that worker_api takes a single object rather than an array; it has no instance_name, because it isn't part of the client-facing protocol at all.

admin and health sit on the private port in the examples for the same reason. A build event protocol endpoint (experimental_bep), when you enable it, can go on its own listener so it can be scaled and secured independently of both.

TLS, when you need it

TLS is a property of the listener:

listener: {
  http: {
    socket_address: "0.0.0.0:50071",
    tls: {
      cert_file: "/etc/nativelink/tls/server.crt",
      key_file: "/etc/nativelink/tls/server.key",
    },
  },
},

Adding client_ca_file turns that into mutual TLS (clients must present a certificate signed by that CA) and client_crl_file lets you revoke one. That's the closest thing NativeLink has to built-in client authentication, and it is a reasonable answer for a cache exposed beyond a trusted network.

FAQ