// HACKER NEWS — CYBERSECURITY
From Thin Air to Bootable Images: The Tine Build System
This post is part of a series covering some of the open source work we have been doing in recent months. Today we introduce and publish tine, our new Buck2-based build system.
Building an operating system with cryptographically verifiable integrity has to start with a build system with these very properties. At the same time, we are part of the greater open source community and want both to contribute and re-use as much existing work as possible. We also aim for efficient development with fast turnaround.
This roughly translates into the following requirements for our build system:
Before building our own build tool, we evaluated several options.
Given our team includes the creator and maintainer of mkosi, it was a natural first candidate to evaluate. But we quickly came to the conclusion that it has some fundamental shortcomings. It’s great at building individual images based on upstream packages, but becomes restrictive when building several weakly related images or if you need more control over the artifacts that make up an image. Building multiple images is limited to images that are intended to be shipped as part of the “main” image.
We need to build many different kinds of artifacts in a uniform and robust way, not just images. Hence the build needs to be orchestrated by a generic and flexible tool. The main build file should be a language calling into library functions like “compile a cargo crate” or “build a UKI”. mkosi is the opposite, it’s a framework: It knows how to build images, and only gives you free-form opaque hooks for the other kinds of builds. That leads to a bad experience when you want to build more than just images.
The Open Build Service (OBS) is a powerful fully-integrated build system that is primarily used by SUSE and the openSUSE project to produce all of their artifacts, anything from packages to ISOs, and many other image formats. It has strong dependency tracking and supports a dizzying array of distributions.
However, it’s also the antithesis of “minimal host requirements”. The server side of it is required, central, and non-trivial to self-host. It’s also not a generic build system, meaning any new artifact types would either have to be modelled as packages or require heavy patches to OBS. We concluded that this lack of flexibility combined with its overall architecture would make it difficult for OBS to meet our requirements.
Apache BuildStream describes the operating system image as a graph of YAML "elements", each with its own sources, dependencies and build commands. BuildStream builds each one in a bubblewrap sandbox and caches the result under a hash of everything that went into it, similar to Buck2. It is mature and used to build freedesktop-sdk, GNOME OS, and WebKitGTK.
Our concerns with BuildStream are mostly around bootstrapping and extensibility. BuildStream is a Python application with compiled extensions and other dependencies. It relies on a separate set of helper programs, plus sandboxing tools from the host. Each of those can be pinned, but through different mechanisms, and even then the result still depends on the host's Python. In practice you run it from a pinned container image instead, but then you’re still dependent on an entire container runtime you don’t control.