From Blaze to Buck2 — A Brief History of Modern Monorepo Build Systems
Our story begins in 2006 with Google’s invention of a revolutionary new build system named Blaze (which later became the open sourced Bazel). For the first time, a build system could provide extremely strong guarantees around the correctness of build artifacts in a generic way. At first glance, this didn’t seem all that exciting — however, it turned out that with this guaranteed correctness came many important advantages compared to other build systems at the time.
In particular, it meant that all build artifacts could now be cached and re-used on the same machine (or even across machines!) in a completely robust way. Developers no longer needed to clean or rebuild their code after distrusting the resulting binaries. They learned that they could just trust the build to do its job and lazily figure out exactly which artifacts actually needed to be rebuilt on their behalf.
Even better, both CI machines and developer machines could all pull from and contribute to the same cache of build artifacts, leading to huge improvements in both CI times and developer iteration times.
Blaze accomplished this by being extremely strict about how build rules were defined, which ensured that it wasn’t even possible for build engineers to break correctness. This meant that while it was often challenging to write the build logic, it would be worth it in the end because of the correctness guarantees (and, therefore, speed guarantees).
Naturally, after experiencing such build bliss, many ex-Google programmers demanded that similar tools exist elsewhere, leading to several Blaze copycats at other big tech companies. In 2010, Facebook named theirs “Buck,” and in 2011, Twitter named theirs “Pants.”
With the cat out of the bag on this tech, Google decided it was time to open source it since it seemed only inevitable that the industry would begin standardizing around one of these, and Facebook and Twitter had already open sourced their clones. However, after almost a decade of internal-only use, Blaze was so thoroughly embedded within internal Google systems that this was impossible without heavy refactoring. Eventually, though, they could do this, resulting in a new tool they named Bazel (an anagram of its predecessor) in 2015.
And for a time, this was the landscape for monorepo build systems. Assuming you weren’t working at one of these tech giants, your options were to either suffer with traditional long build times or attempt to adopt one of these open sourced systems (Buck, Bazel, or Pants), all of which had very steep learning curves and required a significant initial investment.
Companies that chose to leap with these tools often discovered that while they did completely deliver on their promises of Fast and Correct, they fell short in many other aspects that also actually matter (e.g., Extensible, Accessible, User Friendly).
As a result, companies needed dedicated developers working full time to maintain and extend their custom build configuration. This was never an issue within Google, given the swaths of Blaze engineers with years of experience available to keep things running smoothly. And yet, despite these obstacles, it was often still worth it for organizations of a certain size, given the many advantages.
It wasn’t until 2023 when we got another serious contender in this space when Facebook (now Meta) deprecated Buck in favor of a shiny new rewrite named “Buck 2” — almost exactly ten years after the original public release.
And this time, instead of being ‘Fast and Correct,’ Buck2 aimed to be ‘Fast, Correct, AND Extensible.’ And as proof of this, the core Buck2 application did not itself have any language-specific features at all. Instead, the language support was all provided as external Starlark scripts, in the exact same way a normal Buck2 user might add their own custom behaviour for their own custom build needs.
Buck2 is still lacking in terms of documentation, examples, and community, but having just completed a multi-week migration of a monorepo from Bazel to Buck2, I can say anecdotally that it’s nevertheless a very mature product (presumably due to incubating internally for years at Meta). It is as fast or faster than the alternatives while also being a lot more extensible, and most of the time, just doing the right thing that you would expect — so it is already very encouraging.
One important feature missing from all these systems, and probably the main reason they haven’t been adopted more widely outside of tech companies (e.g., open source communities), has been decent support for poly-repo/external dependency management. Those of us who have used good external dependency management tools (e.g., Cargo for Rust, Poetry for Python, npm/yarn for JavaScript, etc.) know how incredibly helpful they can be when setting up a new project. And there is no reason in theory why monorepo build systems like Bazel could not support a similar workflow compared to these tools.
Unfortunately, though, historically, there has been very little incentive for the tech companies that have authored these build systems to design these kinds of features since they all use monorepos internally. In some cases, they have added improved support as an afterthought (e.g., Bazel’s bzlmod), but as of this writing, they still have a long way to go.
Hopefully, though, progress will continue in the coming years, and we can all get closer to fast and reliable builds — even those of us who aren’t working at the big tech giants.
From Blaze to Buck2 — A Brief History of Modern Monorepo Build Systems was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.