Docker
Like much of the modern software industry I do make use of docker, but in my case it is almost entirely for software that is developed by people other than myself or for things like K8s deployments. That said because my goal is fairly rapid application development I really don't want to be getting delayed by software builds so I have taken a few measures
Base images
I use a handful of base images which are all derived from debian slim. You might ask why debian...and why slim. Well while I do like the idea of smaller container sizes I also value being able to use fairly standard\simple config options that you would find in debian by itself...and also I really do not want to be fighting with the wonderful list of incompatibilities between musl and glibc. Slim simply is a smaller image than base debian.
My images all include having their apt sources changes to reference an apt-cacher-ng instance. This is to reduce bandwidth and speed up the overall process for building new images.
The base images currently base, base-nginx, and base-slim. And they are updated\rebuilt daily along with the corresponding services. While I understand in general the bulk of the binaries in a container are not a valid attack surface it is far simpler to just update everything.
Base
Includes all tooling, every language used on the network, and commonly used utilities in the stack. Languages currently in place include: C\C++, Rust, Python, Go, Java\Scala, and JS. Tooling currently in place: kubectl, talos, and misc dev tools (curl, jq, git, etc). This is used as the base for the docker builder for services and the CI image.
Slim
Just the debian slim image with the apt package cache set. Used by all service images.
Nginx
Some services are just a repackaged webapp with frontend files and others have a frontend, but the actual service is being hosted behind a reverse proxy. In both cases nginx is used for this as it has largely replaced Apache in terms of usecase (though I do recall many years ago in the past deploying it for WordPress around 11 years ago). Having a general base image just makes this a lot simpler.
Network builder
While I could use buildkit on k8s this doesn't really make any sense for me given that my builds are typically very quick and if they are lengthy I can simply compile the binary outside of the dockerbuild process. And if I did implement the the builds on k8s it would significantly complicate the docker build cache with respect to existing layers.
So instead I have opted to take the approach of just building all my docker containers on one machine. This can be very cleanly done by loading a config file that was automatically generated by my orchestration system and exposing a single REST API...simply specify the container that needs built, block the builder\return on next request that a build is running, and then unblock the builder. Additionally I have set cgroup limits to prevent the build process from consuming too much processing as is technically running on my thin client (as this is my main development system anyways and all the base images are already there). And of course this API is being called by a binary located inside of the CI pod which is dispatched by the CI runner when a new build is detected on the Gitea instance.
After an image is built the config is read and pushed to the docker registries on the network shortly before triggering a rolling restart\deploy of any services that use the image (considering almost everything is high availability with 3x replicas this is actually a very clean solution).
Internal docker registry
There are multiple ways to host a docker registry and while you could use something like harbor if this is just a local development environment\homelab you can also deploy the standard docker registry on the network...it's a very light and very simple solution. In the interest of simplicity I have gone with the later and deployed two replicas of the service behind the same domain, I just make sure every image is pushed to both. Of course for the docker daemon and Talos if you choose for this to be an insecure registry it will need handled accordingly, but adding a new image mirror is fairly trivial for Talos.