I’m trying to find a better solution to manage configuration files, both user’s dotfiles and system files in /etc. I’m running an ubuntu server where I have a bunch services with custom configurations, and systemd drop-in files, but on top of that I also have some scripts and user dotfiles that I need to track.
What I’m doing right now is that I have a folder full of symlinks in the admin user’s directory (poor username choice, btw) and I’m using bindfs to mount this directory inside a git repository, this way git won’t see them as symlinks, and will version them as regular files. The problem with doing this is that as git deletes and rewrites files, bindfs fails to track the changes and converts the symlink to regular files.
I looked into chezmoi, but that is only meant to track user dotfiles and will refuse to add a file from /etc, that is unless doing some extra work. But even so, chezmoi will not track the user:group of files, so I would still have to manage that manually.
I also looked into GNU Stow, and that would not complain about files from /etc or anywhere, but it similarly will not track permissions and I would have to manage that manually.
I see that some people are using ansible to manage dotfiles, but at that point, it would make sense to just migrate to ansible, except I don’t want to rebuild my server from scratch to use ansible. Also it looks like a lot to learn.
Is there a better solution I’m not seeing? Maybe something using git hooks?
Edit:
I ended up using pre-commit and post-merge git hooks to launch a python script. The python script reads from a yaml file where I annotate the file paths and permissions, and then copies to or from the file location to the git repository.
I used the sudoers file to allow the admin user to run this specific script with specific arguments as root without password (because the git commands are run from VS Code and not manually), which is dangerous, be careful when doing that. I have taken special care to make this secure:
- I used absolute paths for everything, to avoid allowing running from a different pwd as a way to copy different files
- The script itself is installed in a root-owned location, so an unprevileged user cannot edit it
- The configuration yaml is root-owned, so an unprevileged user cannot modify which files are copied or their permissions
- Configuration files that can grant permission are not managed by this script (the yaml, /etc/passwd, /etc/groups, polkit rules, the sudoers file, …)
I will not recommend switching to NixOS and declarative configuration. I will not recommend switching to NixOS and declarative configuration. I will not recommend switching to NixOS and declarative configuration.
…fuck. I failed the saving throw. I’m sorry.
Do look into Ansible, and the whole configuration management topic, though.
Ah yes, the obligatory NixOS recommendation post.
On a serious note though, NixOS IS well-suited for this purpose.
You don’t need to rebuild your server from scratch to use Ansible or any other configuration management tool. It helps, though, because then you can ensure you can rebuild from scratch in a fully automatic way.
You can start putting small things in control with Ansible; next time you want to make a change, do it through Ansible. If you stop making manual changes, you’ll already get some benefit- like being able to put your Ansible manifests in version control.
(I still use Puppet for configuration files, installing packages, etc. It just does some stuff better than Ansible. Still, Puppet is harder to learn, and Ansible can be more than enough. Plus, there’s stuff that Ansible can do that Puppet can’t do.)
Dotfiles are a completely separate problem, tackle them separately. Don’t use Ansible for that, use a dotfile-specific tool.
I have a simple bash script that manages folders and files with a way to route them to whatever location. Then I run the script and it does all the symlinking for me. This is what I do for systemd unit files and my own dotfiles
Chezmoi for user stuff, Ansible for system level (packages, user accounts, base services like mail and vpn)
Docker compose managed with opentofu for applications because I’m weird like that
Stow/chezmoi/your choice for dotfiles, config mgmt for system config. You don’t need to rebuild whole server to start with ansible tho, you can take over one file at a time and grow as you learn.
As you’ve found I don’t know of a tool that will cover both usecases as config mgmt for dotfiles is too much and dotfile mgrs for system config is probably out of their scope.
dotfiles and system configuration are pretty different use-cases, usually when you do system-wide stuff you want to manage not just the configuration files but also what software is installed and a bunch of other things. Ansible or something else like it is definitely the right tool for the job. And Ansible isn’t so difficult to learn, you only need to know like 5% of what it can do to be very effective.
For dotfiles my personal preference is dotbot, but there are MANY many different tools that are all good and are just different ways to accomplish roughly the same thing.
I’m trying to find an easy way to redo my VMs and configs, I’ve started learning terraform to build my VMs and create a cloud init to install all necessary things. Then I think I’ll use ansible for the remaining and put back my configs.
While I’m sure this will work well, I find this is a huge amount of work. I’m hoping someone else has some good ideas / lessons learned about that.
