Build Reproducibility and the Unstable API Trap

Supply chain security has become a central concern in modern software development, but the conversation often focuses on dependency confusion attacks and compromised package registries. There is a quieter, more mundane class of supply chain risk that we encounter regularly in infrastructure security assessments: unpinned dependencies against unstable APIs. During an engagement reviewing a Redis deployment that used custom modules, we traced a production crash to a build process that fetched Redis source headers from an unstable.tar.gz endpoint with no hash verification, no version pinning, and no reproducibility guarantee.

The module in question implemented a custom data type using the Redis Modules API. At the time the module was originally compiled, RedisModule_CreateDataType accepted individual function pointers as positional arguments for the type's lifecycle callbacks: load, save, rewrite, digest, free. Between the initial build and a subsequent rebuild triggered by an unrelated infrastructure change, Redis altered this function's signature to accept a pointer to a RedisModuleTypeMethods struct instead. The module's build system fetched the latest unstable headers, compiled without error, and produced a shared object that linked cleanly against the new API.

The ABI Incompatibility

The linker reported no errors because the function signatures were compatible at the symbol level — the function still accepted the same number of pointer-sized arguments. The incompatibility was at the ABI level: Redis expected a pointer to a struct containing function pointers at specific offsets, while the module passed individual function pointers as positional arguments. At load time, Redis initialised the module without complaint. The defect only manifested at runtime when a key containing the module's custom data type was deleted. Redis attempted to invoke the free callback through what it believed was a methods struct, dereferenced a field that did not contain a valid function pointer, and crashed with signal 11 — a null pointer dereference. The crash was intermittent because it required the specific operation of deleting a key that used the custom type.

The investigation consumed two days of production debugging. The root cause — a mismatch between the API headers used at compile time and the API expected by the running Redis server — was invisible to standard monitoring. No log message indicated an API version mismatch. No compile-time warning was emitted. The crash backtrace pointed to Redis internals, not to the module, making initial triage misleading.

Build Reproducibility as a Security Control

From a security assessment perspective, this case illustrates three distinct failures. First, the build process fetched a dependency from a URL that could return different content on each invocation, violating the principle of reproducible builds. Second, no integrity verification (hash check, signature validation) was performed on the downloaded artifact. Third, the API being consumed was explicitly marked as unstable, meaning the upstream maintainers had made no commitment to backward compatibility. Any one of these conditions is a finding in a supply chain security review; together, they created a defect that was undetectable by the build system and intermittent in production.

Our recommendation in these cases is direct: pin every dependency to a specific version or commit hash, verify integrity using cryptographic hashes stored in the repository, and never build production artifacts against an API that is explicitly documented as unstable. If an unstable API must be used, the build system should compile against a vendored copy of the headers that is updated deliberately, reviewed, and tested. The cost of maintaining a vendored copy is trivial compared to the cost of a production crash caused by an invisible ABI break.

More broadly, this finding belongs to a category we describe as "build-time supply chain risk" — distinct from the more commonly discussed runtime supply chain attacks. The threat is not a malicious actor injecting code into a dependency, but the mundane reality that an unpinned dependency will eventually change in a way that breaks your system. The security control is the same in both cases: ensure that every input to your build process is deterministic, verified, and auditable.