Configuration reference

This is the canonical NativeLink configuration reference for v1.5.2. It is autogenerated from the Rust config crate (nativelink-config/src) via the build-schema binary, so it can never drift from what the binary actually deserializes.

Top-level fields

The root object (CasConfig) accepts the following fields:

Configuration types

Every type reachable from the root configuration, in reading order.

NamedStoreConfig

Common fields

Plus exactly one of the following variants (the key selects the variant):

cache_metrics

Cache metrics store wraps another store and emits low-cardinality OpenTelemetry cache operation metrics for the wrapped store.

This wrapper is opt-in. Stores that are not explicitly wrapped by cache_metrics are constructed exactly as they are without this wrapper and do not pay its hot-path timing or recording cost.

Example JSON5 config:

"cache_metrics": {
  "cache_type": "cas",
  "backend": {
    "filesystem": {
      "content_path": "~/.cache/nativelink/content_path-cas",
      "temp_path": "~/.cache/nativelink/tmp_path-cas"
    }
  }
}

Type: CacheMetricsSpec

memory

Memory store will store all data in a hash map in memory.

Example JSON5 config:

"memory": {
  "eviction_policy": {
    "max_bytes": "10mb",
  }
}

Type: MemorySpec

experimental_cloud_object_store

A generic blob store that will store files on the cloud provider. This configuration will never delete files, so you are responsible for purging old files in other ways. It supports the following backends:

1. Amazon S3: S3 store will use Amazon's S3 service as a backend to store the files. This configuration can be used to share files across multiple instances. Uses system certificates for TLS verification via rustls-platform-verifier.

Example JSON5 config:

"experimental_cloud_object_store": {
  "provider": "aws",
  "region": "eu-north-1",
  "bucket": "crossplane-bucket-af79aeca9",
  "key_prefix": "test-prefix-index/",
  "retry": {
    "max_retries": 6,
    "delay": 0.3,
    "jitter": 0.5
  },
  "multipart_max_concurrent_uploads": 10
}

2. Google Cloud Storage: GCS store uses Google's GCS service as a backend to store the files. This configuration can be used to share files across multiple instances.

Example JSON5 config:

"experimental_cloud_object_store": {
  "provider": "gcs",
  "bucket": "test-bucket",
  "key_prefix": "test-prefix-index/",
  "retry": {
    "max_retries": 6,
    "delay": 0.3,
    "jitter": 0.5
  },
  "multipart_max_concurrent_uploads": 10
}

3. Azure Blob Store: Azure Blob store will use Microsoft's Azure Blob service as a backend to store the files. This configuration can be used to share files across multiple instances.

Example JSON5 config:

"experimental_cloud_object_store": {
  "provider": "azure",
  "account_name": "cloudshell1393657559",
  "container": "simple-test-container",
  "key_prefix": "folder/",
  "retry": {
      "max_retries": 6,
      "delay": 0.3,
      "jitter": 0.5
  },
  "multipart_max_concurrent_uploads": 10
}

4. NetApp ONTAP S3 NetApp ONTAP S3 store will use ONTAP's S3-compatible storage as a backend to store files. This store is specifically configured for ONTAP's S3 requirements including custom TLS configuration, credentials management, and proper vserver configuration.

   This store uses AWS environment variables for credentials:

   • AWS_ACCESS_KEY_ID
   • AWS_SECRET_ACCESS_KEY
   • AWS_DEFAULT_REGION

   Example JSON5 config:

   "experimental_cloud_object_store": {
     "provider": "ontap",
     "endpoint": "https://ontap-s3-endpoint:443",
     "vserver_name": "your-vserver",
     "bucket": "your-bucket",
     "root_certificates": "/path/to/certs.pem",  // Optional
     "key_prefix": "test-prefix/",               // Optional
     "retry": {
       "max_retries": 6,
       "delay": 0.3,
       "jitter": 0.5
     },
     "multipart_max_concurrent_uploads": 10
   }

Type: ExperimentalCloudObjectSpec

ontap_s3_existence_cache

ONTAP S3 Existence Cache provides a caching layer on top of the ONTAP S3 store to optimize repeated existence checks. It maintains an in-memory cache of object digests and periodically syncs this cache to disk for persistence.

The cache helps reduce latency for repeated calls to check object existence, while still ensuring eventual consistency with the underlying ONTAP S3 store.

Example JSON5 config:

"ontap_s3_existence_cache": {
  "index_path": "/path/to/cache/index.json",
  "sync_interval_seconds": 300,
  "backend": {
    "endpoint": "https://ontap-s3-endpoint:443",
    "vserver_name": "your-vserver",
    "bucket": "your-bucket",
    "key_prefix": "test-prefix/"
  }
}

Type: OntapS3ExistenceCacheSpec

verify

Verify store is used to apply verifications to an underlying store implementation. It is strongly encouraged to validate as much data as you can before accepting data from a client, failing to do so may cause the data in the store to be populated with invalid data causing all kinds of problems.

The suggested configuration is to have the CAS validate the hash and size and the AC validate nothing.

Example JSON5 config:

"verify": {
  "backend": {
    "memory": {
      "eviction_policy": {
        "max_bytes": "500mb"
      }
    },
  },
  "verify_size": true,
  "verify_hash": true
}

Type: VerifySpec

completeness_checking

Completeness checking store verifies if the output files & folders exist in the CAS before forwarding the request to the underlying store. Note: This store should only be used on AC stores.

Example JSON5 config:

"completeness_checking": {
  "backend": {
    "filesystem": {
      "content_path": "~/.cache/nativelink/content_path-ac",
      "temp_path": "~/.cache/nativelink/tmp_path-ac",
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  },
  "cas_store": {
    "ref_store": {
      "name": "CAS_MAIN_STORE"
    }
  }
}

Type: CompletenessCheckingSpec

compression

A compression store that will compress the data inbound and outbound. There will be a non-trivial cost to compress and decompress the data, but in many cases if the final store is a store that requires network transport and/or storage space is a concern it is often faster and more efficient to use this store before those stores.

Example JSON5 config:

"compression": {
  "compression_algorithm": {
    "lz4": {}
  },
  "backend": {
    "filesystem": {
      "content_path": "/tmp/nativelink/data/content_path-cas",
      "temp_path": "/tmp/nativelink/data/tmp_path-cas",
      "eviction_policy": {
        "max_bytes": "2gb",
      }
    }
  }
}

Type: CompressionSpec

dedup

A dedup store will take the inputs and run a rolling hash algorithm on them to slice the input into smaller parts then run a sha256 algorithm on the slice and if the object doesn't already exist, upload the slice to the content_store using a new digest of just the slice. Once all parts exist, an Action-Cache-like digest will be built and uploaded to the index_store which will contain a reference to each chunk/digest of the uploaded file. Downloading a request will first grab the index from the index_store, and forward the download content of each chunk as if it were one file.

This store is exceptionally good when the following conditions are met:

• Content is mostly the same (inserts, updates, deletes are ok)
• Content is not compressed or encrypted
• Uploading or downloading from content_store is the bottleneck.

Note: This store pairs well when used with CompressionSpec as the content_store, but never put DedupSpec as the backend of CompressionSpec as it will negate all the gains.

Note: When running .has() on this store, it will only check to see if the entry exists in the index_store and not check if the individual chunks exist in the content_store.

Example JSON5 config:

"dedup": {
  "index_store": {
    "memory": {
      "eviction_policy": {
         "max_bytes": "1GB",
      }
    }
  },
  "content_store": {
    "compression": {
      "compression_algorithm": {
        "lz4": {}
      },
      "backend": {
        "fast_slow": {
          "fast": {
            "memory": {
              "eviction_policy": {
                "max_bytes": "500MB",
              }
            }
          },
          "slow": {
            "filesystem": {
              "content_path": "/tmp/nativelink/data/content_path-content",
              "temp_path": "/tmp/nativelink/data/tmp_path-content",
              "eviction_policy": {
                "max_bytes": "2gb"
              }
            }
          }
        }
      }
    }
  }
}

Type: DedupSpec

existence_cache

Existence store will wrap around another store and cache calls to has so that subsequent has_with_results calls will be faster. This is useful for cases when you have a store that is slow to respond to has calls. Note: This store should only be used on CAS stores.

Example JSON5 config:

"existence_cache": {
  "backend": {
    "memory": {
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  },
  // Note this is the existence store policy, not the backend policy
  "eviction_policy": {
    "max_seconds": 100,
  }
}

Type: ExistenceCacheSpec

fast_slow

FastSlow store will first try to fetch the data from the fast store and then if it does not exist try the slow store. When the object does exist in the slow store, it will copy the data to the fast store while returning the data. This store should be thought of as a store that "buffers" the data to the fast store. On uploads it will mirror data to both fast and slow stores.

WARNING: If you need data to always exist in the slow store for something like remote execution, be careful because this store will never check to see if the objects exist in the slow store if it exists in the fast store (i.e., it assumes that if an object exists in the fast store it will exist in the slow store).

Example JSON5 config:

"fast_slow": {
  "fast": {
    "filesystem": {
      "content_path": "/tmp/nativelink/data/content_path-index",
      "temp_path": "/tmp/nativelink/data/tmp_path-index",
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  },
  "slow": {
    "filesystem": {
      "content_path": "/tmp/nativelink/data/content_path-index",
      "temp_path": "/tmp/nativelink/data/tmp_path-index",
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  }
}

Type: FastSlowSpec

shard

Shards the data to multiple stores. This is useful for cases when you want to distribute the load across multiple stores. The digest hash is used to determine which store to send the data to.

Example JSON5 config:

"shard": {
  "stores": [
   {
    "store": {
      "memory": {
        "eviction_policy": {
            "max_bytes": "10mb"
        },
      },
    },
    "weight": 1
  }]
}

Type: ShardSpec

filesystem

Stores the data on the filesystem. This store is designed for local persistent storage. Restarts of this program should restore the previous state, meaning anything uploaded will be persistent as long as the filesystem integrity holds.

Example JSON5 config:

"filesystem": {
  "content_path": "/tmp/nativelink/data-worker-test/content_path-cas",
  "temp_path": "/tmp/nativelink/data-worker-test/tmp_path-cas",
  "eviction_policy": {
    "max_bytes": "10gb",
  }
}

Type: FilesystemSpec

ref_store

Store used to reference a store in the root store manager. This is useful for cases when you want to share a store in different nested stores. Example, you may want to share the same memory store used for the action cache, but use a FastSlowSpec and have the fast store also share the memory store for efficiency.

Example JSON5 config:

"ref_store": {
  "name": "FS_CONTENT_STORE"
}

Type: RefSpec

size_partitioning

Uses the size field of the digest to separate which store to send the data. This is useful for cases when you'd like to put small objects in one store and large objects in another store. This should only be used if the size field is the real size of the content, in other words, don't use on AC (Action Cache) stores. Any store where you can safely use VerifySpec.verify_size = true, this store should be safe to use (i.e., CAS stores).

Example JSON5 config:

"size_partitioning": {
  "size": "128mib",
  "lower_store": {
    "memory": {
      "eviction_policy": {
        "max_bytes": "${NATIVELINK_CAS_MEMORY_CONTENT_LIMIT:-100mb}"
      }
    }
  },
  "upper_store": {
    /// This store discards data larger than 128mib.
    "noop": {}
  }
}

Type: SizePartitioningSpec

grpc

This store will pass-through calls to another GRPC store. This store is not designed to be used as a sub-store of another store, but it does satisfy the interface and will likely work.

One major GOTCHA is that some stores use a special function on this store to get the size of the underlying object, which is only reliable when this store is serving the a CAS store, not an AC store. If using this store directly without being a child of any store there are no side effects and is the most efficient way to use it.

Example JSON5 config:

"grpc": {
  "instance_name": "main",
  "endpoints": [
    {"address": "grpc://${CAS_ENDPOINT:-127.0.0.1}:50051"}
  ],
  "connections_per_endpoint": "5",
  "rpc_timeout_s": "5m",
  "store_type": "ac",
  // Static headers attached to every outgoing request to the upstream
  // remote cache. Useful for fixed service-account credentials.
  "headers": {
    "authorization": "Bearer my-static-token"
  },
  // Header names to copy from the inbound client request and forward to
  // the upstream remote cache. Use this to pass through dynamic
  // credentials such as a JWT sent by the build client.
  "forward_headers": ["authorization", "x-custom-token"]
}

Type: GrpcSpec

redis_store

Stores data in any stores compatible with Redis APIs.

Pairs well with SizePartitioning and/or FastSlow stores. Ideal for accepting small object sizes as most Redis store services have a max file upload of between 256Mb-512Mb.

Example JSON5 config:

"redis_store": {
  "addresses": [
    "redis://127.0.0.1:6379/",
  ],
  "max_client_permits": 1000,
}

Type: RedisSpec

noop

Noop store is a store that sends streams into the void and all data retrieval will return 404 (NotFound). This can be useful for cases where you may need to partition your data and part of your data needs to be discarded.

Example JSON5 config:

"noop": {}

Type: NoopSpec

experimental_mongo

Experimental MongoDB store implementation.

This store uses MongoDB as a backend for storing data. It supports both CAS (Content Addressable Storage) and scheduler data with optional change streams for real-time updates.

Example JSON5 config:

"experimental_mongo": {
    "connection_string": "mongodb://localhost:27017",
    "database": "nativelink",
    "cas_collection": "cas",
    "key_prefix": "cas:",
    "read_chunk_size": 65536,
    "max_concurrent_uploads": 10,
    "enable_change_streams": false,
    "max_requests": "100"
}

Type: ExperimentalMongoSpec

WorkerConfig

Plus exactly one of the following variants (the key selects the variant):

local

A worker type that executes jobs locally on this machine.

Type: LocalWorkerConfig

NamedSchedulerConfig

Common fields

Plus exactly one of the following variants (the key selects the variant):

simple

Type: SimpleSpec

grpc

Type: SchedulerGrpcSpec

cache_lookup

Type: CacheLookupSpec

property_modifier

Type: PropertyModifierSpec

historical_resource

Type: HistoricalResourceSpec

ServerConfig

OriginEventsSpec

GlobalConfig

CacheMetricsSpec

MemorySpec

ExperimentalCloudObjectSpec

See experimental_cloud_object_store for details

OntapS3ExistenceCacheSpec

VerifySpec

CompletenessCheckingSpec

CompressionSpec

DedupSpec

ExistenceCacheSpec

FastSlowSpec

ShardSpec

FilesystemSpec

RefSpec

SizePartitioningSpec

GrpcSpec

RedisSpec

NoopSpec

No fields.

ExperimentalMongoSpec

Configuration for ExperimentalMongoDB store.

LocalWorkerConfig

SimpleSpec

SchedulerGrpcSpec

A scheduler that forwards requests to an upstream scheduler. This is useful to use when doing some kind of local action cache or CAS away from the main cluster of workers. In general, it's more efficient to point the build at the main scheduler directly though.

CacheLookupSpec

PropertyModifierSpec

HistoricalResourceSpec

ListenerConfig

Plus exactly one of the following variants (the key selects the variant):

http

Listener for HTTP/HTTPS/HTTP2 sockets.

Type: HttpListener

ServicesConfig

IdentityHeaderSpec

OriginEventsPublisherSpec

ConfigDigestHashFunction

StoreSpec

Plus exactly one of the following variants (the key selects the variant):

cache_metrics

Cache metrics store wraps another store and emits low-cardinality OpenTelemetry cache operation metrics for the wrapped store.

This wrapper is opt-in. Stores that are not explicitly wrapped by cache_metrics are constructed exactly as they are without this wrapper and do not pay its hot-path timing or recording cost.

Example JSON5 config:

"cache_metrics": {
  "cache_type": "cas",
  "backend": {
    "filesystem": {
      "content_path": "~/.cache/nativelink/content_path-cas",
      "temp_path": "~/.cache/nativelink/tmp_path-cas"
    }
  }
}

Type: CacheMetricsSpec

memory

Memory store will store all data in a hash map in memory.

Example JSON5 config:

"memory": {
  "eviction_policy": {
    "max_bytes": "10mb",
  }
}

Type: MemorySpec

experimental_cloud_object_store

A generic blob store that will store files on the cloud provider. This configuration will never delete files, so you are responsible for purging old files in other ways. It supports the following backends:

1. Amazon S3: S3 store will use Amazon's S3 service as a backend to store the files. This configuration can be used to share files across multiple instances. Uses system certificates for TLS verification via rustls-platform-verifier.

Example JSON5 config:

"experimental_cloud_object_store": {
  "provider": "aws",
  "region": "eu-north-1",
  "bucket": "crossplane-bucket-af79aeca9",
  "key_prefix": "test-prefix-index/",
  "retry": {
    "max_retries": 6,
    "delay": 0.3,
    "jitter": 0.5
  },
  "multipart_max_concurrent_uploads": 10
}

2. Google Cloud Storage: GCS store uses Google's GCS service as a backend to store the files. This configuration can be used to share files across multiple instances.

Example JSON5 config:

"experimental_cloud_object_store": {
  "provider": "gcs",
  "bucket": "test-bucket",
  "key_prefix": "test-prefix-index/",
  "retry": {
    "max_retries": 6,
    "delay": 0.3,
    "jitter": 0.5
  },
  "multipart_max_concurrent_uploads": 10
}

3. Azure Blob Store: Azure Blob store will use Microsoft's Azure Blob service as a backend to store the files. This configuration can be used to share files across multiple instances.

Example JSON5 config:

"experimental_cloud_object_store": {
  "provider": "azure",
  "account_name": "cloudshell1393657559",
  "container": "simple-test-container",
  "key_prefix": "folder/",
  "retry": {
      "max_retries": 6,
      "delay": 0.3,
      "jitter": 0.5
  },
  "multipart_max_concurrent_uploads": 10
}

4. NetApp ONTAP S3 NetApp ONTAP S3 store will use ONTAP's S3-compatible storage as a backend to store files. This store is specifically configured for ONTAP's S3 requirements including custom TLS configuration, credentials management, and proper vserver configuration.

   This store uses AWS environment variables for credentials:

   • AWS_ACCESS_KEY_ID
   • AWS_SECRET_ACCESS_KEY
   • AWS_DEFAULT_REGION

   Example JSON5 config:

   "experimental_cloud_object_store": {
     "provider": "ontap",
     "endpoint": "https://ontap-s3-endpoint:443",
     "vserver_name": "your-vserver",
     "bucket": "your-bucket",
     "root_certificates": "/path/to/certs.pem",  // Optional
     "key_prefix": "test-prefix/",               // Optional
     "retry": {
       "max_retries": 6,
       "delay": 0.3,
       "jitter": 0.5
     },
     "multipart_max_concurrent_uploads": 10
   }

Type: ExperimentalCloudObjectSpec

ontap_s3_existence_cache

ONTAP S3 Existence Cache provides a caching layer on top of the ONTAP S3 store to optimize repeated existence checks. It maintains an in-memory cache of object digests and periodically syncs this cache to disk for persistence.

The cache helps reduce latency for repeated calls to check object existence, while still ensuring eventual consistency with the underlying ONTAP S3 store.

Example JSON5 config:

"ontap_s3_existence_cache": {
  "index_path": "/path/to/cache/index.json",
  "sync_interval_seconds": 300,
  "backend": {
    "endpoint": "https://ontap-s3-endpoint:443",
    "vserver_name": "your-vserver",
    "bucket": "your-bucket",
    "key_prefix": "test-prefix/"
  }
}

Type: OntapS3ExistenceCacheSpec

verify

Verify store is used to apply verifications to an underlying store implementation. It is strongly encouraged to validate as much data as you can before accepting data from a client, failing to do so may cause the data in the store to be populated with invalid data causing all kinds of problems.

The suggested configuration is to have the CAS validate the hash and size and the AC validate nothing.

Example JSON5 config:

"verify": {
  "backend": {
    "memory": {
      "eviction_policy": {
        "max_bytes": "500mb"
      }
    },
  },
  "verify_size": true,
  "verify_hash": true
}

Type: VerifySpec

completeness_checking

Completeness checking store verifies if the output files & folders exist in the CAS before forwarding the request to the underlying store. Note: This store should only be used on AC stores.

Example JSON5 config:

"completeness_checking": {
  "backend": {
    "filesystem": {
      "content_path": "~/.cache/nativelink/content_path-ac",
      "temp_path": "~/.cache/nativelink/tmp_path-ac",
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  },
  "cas_store": {
    "ref_store": {
      "name": "CAS_MAIN_STORE"
    }
  }
}

Type: CompletenessCheckingSpec

compression

A compression store that will compress the data inbound and outbound. There will be a non-trivial cost to compress and decompress the data, but in many cases if the final store is a store that requires network transport and/or storage space is a concern it is often faster and more efficient to use this store before those stores.

Example JSON5 config:

"compression": {
  "compression_algorithm": {
    "lz4": {}
  },
  "backend": {
    "filesystem": {
      "content_path": "/tmp/nativelink/data/content_path-cas",
      "temp_path": "/tmp/nativelink/data/tmp_path-cas",
      "eviction_policy": {
        "max_bytes": "2gb",
      }
    }
  }
}

Type: CompressionSpec

dedup

A dedup store will take the inputs and run a rolling hash algorithm on them to slice the input into smaller parts then run a sha256 algorithm on the slice and if the object doesn't already exist, upload the slice to the content_store using a new digest of just the slice. Once all parts exist, an Action-Cache-like digest will be built and uploaded to the index_store which will contain a reference to each chunk/digest of the uploaded file. Downloading a request will first grab the index from the index_store, and forward the download content of each chunk as if it were one file.

This store is exceptionally good when the following conditions are met:

• Content is mostly the same (inserts, updates, deletes are ok)
• Content is not compressed or encrypted
• Uploading or downloading from content_store is the bottleneck.

Note: This store pairs well when used with CompressionSpec as the content_store, but never put DedupSpec as the backend of CompressionSpec as it will negate all the gains.

Note: When running .has() on this store, it will only check to see if the entry exists in the index_store and not check if the individual chunks exist in the content_store.

Example JSON5 config:

"dedup": {
  "index_store": {
    "memory": {
      "eviction_policy": {
         "max_bytes": "1GB",
      }
    }
  },
  "content_store": {
    "compression": {
      "compression_algorithm": {
        "lz4": {}
      },
      "backend": {
        "fast_slow": {
          "fast": {
            "memory": {
              "eviction_policy": {
                "max_bytes": "500MB",
              }
            }
          },
          "slow": {
            "filesystem": {
              "content_path": "/tmp/nativelink/data/content_path-content",
              "temp_path": "/tmp/nativelink/data/tmp_path-content",
              "eviction_policy": {
                "max_bytes": "2gb"
              }
            }
          }
        }
      }
    }
  }
}

Type: DedupSpec

existence_cache

Existence store will wrap around another store and cache calls to has so that subsequent has_with_results calls will be faster. This is useful for cases when you have a store that is slow to respond to has calls. Note: This store should only be used on CAS stores.

Example JSON5 config:

"existence_cache": {
  "backend": {
    "memory": {
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  },
  // Note this is the existence store policy, not the backend policy
  "eviction_policy": {
    "max_seconds": 100,
  }
}

Type: ExistenceCacheSpec

fast_slow

FastSlow store will first try to fetch the data from the fast store and then if it does not exist try the slow store. When the object does exist in the slow store, it will copy the data to the fast store while returning the data. This store should be thought of as a store that "buffers" the data to the fast store. On uploads it will mirror data to both fast and slow stores.

WARNING: If you need data to always exist in the slow store for something like remote execution, be careful because this store will never check to see if the objects exist in the slow store if it exists in the fast store (i.e., it assumes that if an object exists in the fast store it will exist in the slow store).

Example JSON5 config:

"fast_slow": {
  "fast": {
    "filesystem": {
      "content_path": "/tmp/nativelink/data/content_path-index",
      "temp_path": "/tmp/nativelink/data/tmp_path-index",
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  },
  "slow": {
    "filesystem": {
      "content_path": "/tmp/nativelink/data/content_path-index",
      "temp_path": "/tmp/nativelink/data/tmp_path-index",
      "eviction_policy": {
        "max_bytes": "500mb",
      }
    }
  }
}

Type: FastSlowSpec

shard

Shards the data to multiple stores. This is useful for cases when you want to distribute the load across multiple stores. The digest hash is used to determine which store to send the data to.

Example JSON5 config:

"shard": {
  "stores": [
   {
    "store": {
      "memory": {
        "eviction_policy": {
            "max_bytes": "10mb"
        },
      },
    },
    "weight": 1
  }]
}

Type: ShardSpec

filesystem

Stores the data on the filesystem. This store is designed for local persistent storage. Restarts of this program should restore the previous state, meaning anything uploaded will be persistent as long as the filesystem integrity holds.

Example JSON5 config:

"filesystem": {
  "content_path": "/tmp/nativelink/data-worker-test/content_path-cas",
  "temp_path": "/tmp/nativelink/data-worker-test/tmp_path-cas",
  "eviction_policy": {
    "max_bytes": "10gb",
  }
}

Type: FilesystemSpec

ref_store

Store used to reference a store in the root store manager. This is useful for cases when you want to share a store in different nested stores. Example, you may want to share the same memory store used for the action cache, but use a FastSlowSpec and have the fast store also share the memory store for efficiency.

Example JSON5 config:

"ref_store": {
  "name": "FS_CONTENT_STORE"
}

Type: RefSpec

size_partitioning

Uses the size field of the digest to separate which store to send the data. This is useful for cases when you'd like to put small objects in one store and large objects in another store. This should only be used if the size field is the real size of the content, in other words, don't use on AC (Action Cache) stores. Any store where you can safely use VerifySpec.verify_size = true, this store should be safe to use (i.e., CAS stores).

Example JSON5 config:

"size_partitioning": {
  "size": "128mib",
  "lower_store": {
    "memory": {
      "eviction_policy": {
        "max_bytes": "${NATIVELINK_CAS_MEMORY_CONTENT_LIMIT:-100mb}"
      }
    }
  },
  "upper_store": {
    /// This store discards data larger than 128mib.
    "noop": {}
  }
}

Type: SizePartitioningSpec

grpc

This store will pass-through calls to another GRPC store. This store is not designed to be used as a sub-store of another store, but it does satisfy the interface and will likely work.

One major GOTCHA is that some stores use a special function on this store to get the size of the underlying object, which is only reliable when this store is serving the a CAS store, not an AC store. If using this store directly without being a child of any store there are no side effects and is the most efficient way to use it.

Example JSON5 config:

"grpc": {
  "instance_name": "main",
  "endpoints": [
    {"address": "grpc://${CAS_ENDPOINT:-127.0.0.1}:50051"}
  ],
  "connections_per_endpoint": "5",
  "rpc_timeout_s": "5m",
  "store_type": "ac",
  // Static headers attached to every outgoing request to the upstream
  // remote cache. Useful for fixed service-account credentials.
  "headers": {
    "authorization": "Bearer my-static-token"
  },
  // Header names to copy from the inbound client request and forward to
  // the upstream remote cache. Use this to pass through dynamic
  // credentials such as a JWT sent by the build client.
  "forward_headers": ["authorization", "x-custom-token"]
}

Type: GrpcSpec

redis_store

Stores data in any stores compatible with Redis APIs.

Pairs well with SizePartitioning and/or FastSlow stores. Ideal for accepting small object sizes as most Redis store services have a max file upload of between 256Mb-512Mb.

Example JSON5 config:

"redis_store": {
  "addresses": [
    "redis://127.0.0.1:6379/",
  ],
  "max_client_permits": 1000,
}

Type: RedisSpec

noop

Noop store is a store that sends streams into the void and all data retrieval will return 404 (NotFound). This can be useful for cases where you may need to partition your data and part of your data needs to be discarded.

Example JSON5 config:

"noop": {}

Type: NoopSpec

experimental_mongo

Experimental MongoDB store implementation.

This store uses MongoDB as a backend for storing data. It supports both CAS (Content Addressable Storage) and scheduler data with optional change streams for real-time updates.

Example JSON5 config:

"experimental_mongo": {
    "connection_string": "mongodb://localhost:27017",
    "database": "nativelink",
    "cas_collection": "cas",
    "key_prefix": "cas:",
    "read_chunk_size": 65536,
    "max_concurrent_uploads": 10,
    "enable_change_streams": false,
    "max_requests": "100"
}

Type: ExperimentalMongoSpec

EvictionPolicy

Eviction policy always works on LRU (Least Recently Used). Any time an entry is touched it updates the timestamp. Inserts and updates will execute the eviction policy removing any expired entries and/or the oldest entries until the store size becomes smaller than max_bytes.

Retry

Retry configuration. This configuration is exponential and each iteration a jitter as a percentage is applied of the calculated delay. For example:

Retry{
  max_retries: 7,
  delay: 0.1,
  jitter: 0.5,
}

will result in:

The total delay is additive, so this example produces 9.525 to 15.875 s of total delay for a single request.

ExperimentalOntapS3Spec

CompressionAlgorithm

Plus exactly one of the following variants (the key selects the variant):

lz4

LZ4 compression algorithm is extremely fast for compression and decompression, however does not perform very well in compression ratio. In most cases build artifacts are highly compressible, however lz4 is quite good at aborting early if the data is not deemed very compressible.

see: https://lz4.github.io/lz4/

Type: Lz4Config

StoreDirection

ShardConfig

Configuration for an individual shard of the store.

GrpcEndpoint

StoreType

RedisMode

EndpointConfig

Generic config for an endpoint and associated configs.

UploadActionResultConfig

WorkerProperty

Plus exactly one of the following variants (the key selects the variant):

values

List of static values. Note: Generally there should only ever be 1 value, but if the platform property key is PropertyType::Priority it may have more than one value.

Type: array of string

query_cmd

A dynamic configuration. The string will be executed as a command (not shell) and will be split by "\n" (new line character).

Type: string

EnvironmentSource

One of:

• property: string — The name of the platform property in the action to get the value from.
• value: string — The raw value to set.
• "from_environment" — Take the value from the local environment corresponding to the name key
• "timeout_millis" — The max amount of time in milliseconds the command is allowed to run (requested by the client).
• "side_channel_file" — A special file path will be provided that can be used to communicate with the parent process about out-of-band information. This file will be read after the command has finished executing. Based on the contents of the file, the behavior of the result may be modified.
• "action_directory" — A "root" directory for the action. This directory can be used to store temporary files that are not needed after the action has completed. This directory will be purged after the action has completed.

DirectoryCacheConfig

PropertyType

When the scheduler matches tasks to workers that are capable of running the task, this value will be used to determine how the property is treated.

One of:

• string
• "minimum" — Requires the platform property to be a u64 and when the scheduler looks for appropriate worker nodes that are capable of executing the task, the task will not run on a node that has less than this value.
• "exact" — Requires the platform property to be a string and when the scheduler looks for appropriate worker nodes that are capable of executing the task, the task will not run on a node that does not have this property set to the value with exact string match.
• "priority" — Does not restrict on this value and instead will be passed to the worker as an informational piece. In the future this will be used by the scheduler and worker to cause the scheduler to prefer certain workers over others, but not restrict them based on these values.

WorkerAllocationStrategy

When a worker is being searched for to run a job, this will be used on how to choose which worker should run the job when multiple workers are able to run the task.

ExperimentalSimpleSchedulerBackend

One of:

• "memory" — Use an in-memory store for the scheduler.
• redis: ExperimentalRedisSchedulerBackend — Use a Redis store for the scheduler.

SchedulerSpec

Plus exactly one of the following variants (the key selects the variant):

simple

Type: SimpleSpec

grpc

Type: SchedulerGrpcSpec

cache_lookup

Type: CacheLookupSpec

property_modifier

Type: PropertyModifierSpec

historical_resource

Type: HistoricalResourceSpec

PropertyModification

Plus exactly one of the following variants (the key selects the variant):

add

Add a property to the action properties.

Type: PlatformPropertyAddition

remove

Remove a named property from the action.

Type: string

replace

If a property is found, then replace it with another one.

Type: PlatformPropertyReplacement

HttpListener

CasServiceConfig

ActionCacheServiceConfig

CapabilitiesServiceConfig

ExecutionServiceConfig

ByteStreamServiceConfig

FetchServiceConfig

PushServiceConfig

WorkerApiConfig

BepConfig

AdminConfig

HealthConfig

ErrorCode

The possible error codes that might occur on an upstream request.

Lz4Config

ClientTlsConfig

UploadCacheResultsStrategy

ExperimentalRedisSchedulerBackend

PlatformPropertyAddition

PlatformPropertyReplacement

HttpCompressionConfig

Note: Compressing data in the cloud rarely has a benefit, since most cloud providers have very high bandwidth backplanes. However, for clients not inside the data center, it might be a good idea to compress data to and from the cloud. This will however come at a high CPU and performance cost. If you are making remote execution share the same CAS/AC servers as client's remote cache, you can create multiple services with different compression settings that are served on different ports. Then configure the non-cloud clients to use one port and cloud-clients to use another.

HttpServerConfig

Advanced HTTP configuration. These generally should not be set. For documentation on these settings, see the hyper documentation: See: hyper HTTP server docs

Note: All of these default to the default values from hyper unless otherwise specified.

TlsConfig

CapabilitiesRemoteExecutionConfig

HttpCompressionAlgorithm

Reading the source

If anything here disagrees with the binary, the source wins:

• stores.rs
• cas_server.rs
• schedulers.rs