Arch AUR CI
Complexity in software is a strange thing. No complexity equals a mess, but too much is impossible to navigate. I also understand pragmatically many Arch Linux users and probably a decent number of readers external to this space will likely wonder why in the world did you build this. And why is it even necessary?
At one point about 4 years ago I wrote my own automated build system for unofficial Arch packages...I rewrote this fairly recently to be a bit more flexible and just generally less brittle. Considering most of my technical writeups from 4 years ago are largely gone at this point I think it's a good idea to generally write this up.
Why
Arch is what one would refer to as a meta Linux distro. At least this is an opinion by some members of the community. This terms stems from the minimal nature of Arch and how a default install drops you into a terminal with just the baseline utilities. You are left to configure and figure out the rest. And this is insanely powerful because you have full control over your machine and you will learn the entire domain of how Linux and a lot of how general operating system design works (this is something that throughout my professional career has come in handy on various projects).
Beyond the purposes of education it also gives you an insane degree of low level control to build tooling for yourself as you see fit and write projects that interact at a very low level into the system (for example I have multiple daemons I have written for my system for various purposes and have even made driver\kernel modifications for things like VR). The benefits I gain on things like speed and comfort in using my machine as well as just lower hardware requirements (my laptop I bring with me is an old 8GBs of ram machine from 6 years ago while my main thin client is a virtual machine) which aligns heavily with my perspective of reusing old hardware and minimizing waste in general in life. Objectively speaking, I am roughly 3-4 times faster on my customized Linux boxes compared to say a Windows PC just due to the macros and insane amount of tooling I have so the time I spend writing custom tooling and daemons I do still come out ahead on overall.
Regardless, the above is why so much of the tooling in Arch is open to tinkering\implementation by developers. We're given a box and we have to figure out a way to manage it and we have all taken various approaches to do so. Mine just so happens to be custom daemons and leaning heavily on my infra as well as making use of existing packing where possible. Others will run docker containers on their local machine using something like distrobox or apx and others will do virtualization with something like qemu (which I have done in the past and still do for my thin client and some gaming systems). There are also the people who just load a nix shell (think NixOS which I do run on my server cluster in a few places with just the package manager)...candidly I have found nix to be a very valuable part of my infra.
My direction for a lot of my packages I need has been to make the unofficial builds work because I need fairly performance sensitive components that are effectively git builds for Linux VR, but there are a few problems with this. By default many AUR packages are source packages and as a result these can take a long time to build. There is also a fairly well documented history of supply chain attacks and this allows you to have a local\frozen version with any modifications that you know is safe. And sure you could use an AUR helper to automate pulling in the latest builds but given how frequently the AUR in recent years has had supply chain attacks I would recommend not doing that. Due to a lack of tooling I elected to build some and to be clear had their been off the shelf tooling that worked for me I would not have done this, but because there wasn't I felt the need.
How it works
This project clones a repo of PKGBUILD \ SRCINFO where each pair is located in a seperate folder per project. Effectively it is the same view as if you had cloned the AUR packages git repos and placed them next to each other. It then builds a dependency graph and compiles\installs the respective dependencies in order. It is not able to resolve a dependency if it is not present in the subfolders provided and this is intentional so as to make sure that all repos have their full dependency tree available.
Should you do this
Yes...but only if you are willing to maintain a mirror of PKGBUILDs and all the associated dependency tree. While this is a pretty low effort\low time task once it's running your initial setup could take awhile. And candidly, if all the AUR packages you use are insanely commonly\low risk then this is likely not a major concern for you. A lot of the risk in the AUR is on packages that people don't use heavily so these packages can get randomly orphaned and then adopted by a malicious actor or they are new packages that are intentionally malicious (these frequently are close in name to the correct package so they depend on typosquatting).
Because a lot of packages in Arch are also available as flatpaks which is effectively a container for desktop applications and we also have container based solutions unless what you are doing is performance sensitive, a library, or requires direct system access you can probably avoid the AUR a lot. Candidly, I would also recommend you do avoid it if you do not need it because the security risk is very real and packaging issues\bugs are very common.
More information
The project source code you can take a look at on Github. Like all my projects this is just the public mirror. For the legacy builder that started this 4 years ago you can see the source code here.