The other day I was doing the usual thing of tweaking my Doom-Emacs config to avoid doing productive work. One of the bigger annoyances recently has been the fact that Emacs 28.1 was released, but it is taking some time to make it into distro repositories. So, on Fedora I fairly recently switched to a COPR repository, which carries Emacs 28 with PGTK patches for a nice combination of native-comp and Wayland compatibility. This setup combined with Doom-Emacs works and a sane person would have probably just left things there, but obviously I decided to push on a bit further.
One particular reason was that on my home systems I am running Fedora Silverblue and I have felt that my current development workflow is a bit messy and wanted to clean that up. This led me on a bit of a quest involving Emacs Flatpak, Toolbx and a hand-rolled Emacs configuration.
Fedora Silverblue in a nutshell
If you don't know what makes Silverblue special, it's worth having a quick look at the official Silverblue page or my post about it. But, in simple terms Fedora Silverblue has the following characteristics:
- System root is mostly immutable
- System software installation/updates are handled as images and applied on reboot
- For installing applications, Flatpak is the recommended default route
- Toolbx (and Distrobox) can be used to set up containerized Linux environments for developer tools, these containers will mount your home directory and certain important files for GUI programs to work as well
Development workflows on Fedora Silverblue
The development workflow story on Fedora Silverblue hasn't really been standardized yet. The distro is still quite new and differs quite a bit from other Linux distros, which means that things are a bit of a wild west for now.
There are basically a couple ways to do this though.
Single omni-Toolbox for all development activities
Possibly the easiest way to have everything working in a predictable and simple manner is to create a single Toolbox container, which has all of the development tools for every single project installed within it. This was how I organized my development environment at first.
This means that the container contains your text editor of choice, compilers, linters, build tools and all of their dependencies. The benefit is that this approach is extremely close to how you'd work on any other Linux distribution.
However, this can be argued to not be the best use of Toolbox, since you have now piled all software into one environment. This increases the potential damage of installing a software package that accidentally breaks something and also makes it harder to keep the development environment reproducible, since you'll probably end up installing all sorts of packages and updating them along the way, which makes it harder to track down problems.
Additionally Toolboxes don't integrate particularly well with the desktop environment, so creation of application launchers or keyboard shortcuts needs to be handled manually.
If these things are not a concern for you, this is probably what I would recommend for when you are getting started though. Basically all conventional Linux software development expertise and knowledge is fully applicable and the approach works without needing to worry about crossing container borders.
Toolbox-per-project
A variant of the Toolbox-oriented approach is setting up a Toolbox separately for each project you work on. This means that all of your project environments are fully separated from each other and all of them only carry tools and dependencies relevant to the project in question. Loss of a Toolbox container limits the damage to a particular project and per-project Toolboxes are fairly simple to recreate.
The problem is that now certain parts of your development workflow need to be duplicated. You potentially need to install your text editor and all of its required dependencies and tools in each environment. In practice this might not result in physical duplication, since depending on how you construct these containers, the container layer might be able to deduplicate these parts. There have also been suggestions of creating a base image, which contains all the tools you intend to use across projects and then building your project containers out of that image.
However, keeping the text editor inside the Toolbox still results in somewhat poor integration with the host system. Maintenance of different containers might also become a bit of a chore as their number grows.
Flatpak-based approach (a la GNOME Builder)
GNOME Builder has an interesting approach to software development, where you run the IDE through Flatpak and the IDE will then create sub-containers in which it builds and runs software.
This approach has the benefit of hiding the complexity of managing development environments from the user, provided that the IDE/editor you are using provides the required abstractions. In practice this means that you'll more or less need to use GNOME Builder for now. Doing something similar with other editors would probably be possible, but I don't know enough about Flatpak to know how to make that happen just yet.
In the future this might become a really simple way to solve this problem, but for now it's still a bit too early.
Addendum
I did some more investigating and found that this approach is also possible to some extent for other editors, though it takes a bit more manual work. You can install Flatpak SDKs such as the Rust build tools like this:
$ flatpak install org.freedesktop.Sdk.Extension.rust-stableAnd then load them into a Flatpak Emacs using an SDK extension flag:
$ FLATPAK_ENABLE_SDK_EXT=rust-stable flatpak run org.gnu.emacsThis causes the tools and libraries from that SDK to be made visible to the Flatpak application. There are Flatpak SDKs for a few languages like Java, Rust, Haskell, PHP and Node, but obviously support for all use-cases does not exist. But if you only plan to develop in languages that already have an SDK ready or are willing to learn how to package more SDKs, then this approach is definitely viable for things like Neovim, Emacs and VSCode Flatpaks.
Mix-and-match
You can also mix approaches such that part of the development activity happens outside Toolbox and part of it inside the Toolbox. The simplest case would be installing your text editor via Flatpak for example and then connecting from that text editor to a Toolbox for project-specific tools like compilers.
The benefit here is that you can install tools where they make most sense and where they integrate the best. So, you will get your nice app launchers and the ability to keep project environments separate.
This is also how my Emacs Flatpak workflow works.
YOLOing with rpm-ostree (not a good idea)
You can also install development tools directly on the Silverblue install using package layering with rpm-ostree. So, if you want to, you can install your editor, compilers, build tools and linters straight on the Silverblue install.
However, this isn't really how most people view you should use Silverblue. The application installation process is a bit more annoying, although experimental support for installing software without rebooting does exist nowadays. It also makes your system install kind of messy due to large numbers of overlayed packages.
I wouldn't take this route because while you aren't strictly speaking losing all of the benefits of Silverblue, you are making life quite difficult for yourself compared to just using regular Fedora Workstation for instance. You also don't get the benefits of separating your system from your application environments and expose yourself to more potential breakages.
Onwards with Emacs
So, the solution I landed on was setting up the Emacs Flatpak to work with Toolboxes such that I can install my development tools inside the Toolbox environment while maintaining a shared installation and configuration of Emacs. The Emacs Flatpak was also the easiest way to gain access to Emacs 28.1 for me.
The first hurdle was that my Doom-Emacs config is completely incompatible with Emacs Flatpak. When I tried to run the "doom" tool, it would endlessly complain about recursive loads and not actually execute successfully.
I debated my approach for a while, until I eventually decided to drop Doom-Emacs and hand-roll a vanilla configuration for myself. This was the second time that I've tried crafting an Emacs config myself, so I wasn't really sure how things would go. I've lived very comfortable with Doom for a while, but I've picked up a few things here and there, so I figured I was ready to take the plunge.
Connecting to Toolbox
The key part that would make or break the whole thing would be being able to connect to the Toolbox container from Emacs.
Emacs has a piece of software called Tramp, which allows opening files and executing commands on remote systems transparently. The first thing I looked at was toolbox-tramp, but even the README said things weren't quite working and my testing indicated the same.
However a Reddit Post I found happened to contain a small snippet of code that did just the thing I needed:
(require 'tramp)
(push
(cons
"toolbox"
'((tramp-login-program "flatpak-spawn --host toolbox")
(tramp-login-args (("enter" "-c") ("%h")))
(tramp-remote-shell "/bin/sh")
(tramp-remote-shell-args ("-i") ("-c"))))
tramp-methods)With just those few lines added to my configuration, I was able to open a project with
M-x find-file /toolbox:fedora-toolbox-36:<path-to-project>Update (2022-08-20):
The toolbox-tramp package has improved a fair bit and I have moved to using it since this blog post was published. However, there are some specifics that I had to do with it.
Firstly, since I don't use straight.el I settled on downloading the module and loading it manually. Additionally toolbox-tramp wants to use podman directly, so I had to create a wrapper script for it to call podman via 'flatpak-spawn –host':
(add-to-list 'load-path "~/.emacs.d/toolbox-tramp/")
(require 'toolbox-tramp)
;; Add ~/.var/app/org.gnu.emacs/data/local/bin to PATH for wrapper scripts
(setenv "PATH" (concat (getenv "PATH") ":" (expand-file-name "~/.var/app/org.gnu.emacs/data/local/bin")))
(setq exec-path (append exec-path (list (expand-file-name "~/.var/app/org.gnu.emacs/data/local/bin"))))In order for toolbox-tramp to work, I then placed the following script into ~/.var/app/org.gnu.emacs/data/local/bin:
#!/bin/sh
flatpak-spawn --host podman "$@"Rest of the config
Obviously the next step was actually making Emacs usable. I have been ruined forever by Vim and Doom-Emacs, so I am quite particular about how I want my buttons to work. I obviously also need certain tools and functionalities in order to work on my projects.
At the minimum, I would need:
- Evil for Vim keybindings
- Spacebar as leader key + which-key for finding my way
- an LSP client
- decent completion framework
- Magit for managing Git repos
- Basic project management
- Language support for Rust and Svelte, potentially others
- Support work "workspaces"
Evil was among the first things I installed purely because I cannot stand the default Emacs keybindings. I know the very basics enough to get by, but you won't find me using them except as long as it takes me to successfully install Evil. I also wanted to use SPC as my leader key in the same was as it's used in Doom-Emacs, because C-c and C-x are comparatively awkward for me. For this I found general.el to be quite useful, especially with its which-key support.
For my LSP client I decided to go with Eglot, since if I was going to roll my own config, I may as well steer on the light-weight side of things for now and see how it goes. I've used both lsp-mode and Eglot in the past and I don't have strong opinions either way, but in Doom-Emacs I typically used lsp-mode purely because it's the default.
For command completions, I went with vertico.el and Orderless, mainly because I quite enjoyed Vertico ever since Doom-Emacs switched from Ivy to Vertico as the default.
Magit was easy to install and evil-collection gives it slightly more Vim-ish keybindings.
For project management I've always used Projectile on Doom-Emacs and I was thinking about adding it. But I had also heard that Emacs has a built-in project management tool called project.el. I had a look around the functions it provides and it has essentially everything I used from Projectile. So, I just mapped that to "SPC p" and concluded it was good enough.
Language support for Rust and Svelte needs to be installed separately. I first tried rustic for Rust, but it seemed to have trouble finding the Cargo binary inside the Toolbox container. So, I replaced it with rust-mode and cargo.el to get basically all of the features I needed. However, I did need to tell cargo.el to look for Cargo inside "/usr/bin" for it to work over Tramp:
(setq cargo-process--custom-path-to-bin "/usr/bin/cargo")
(setq cargo-process--rustc-cmd "/usr/bin/rustc")For Svelte I just installed svelte-mode. I haven't gotten completions fully working yet and I am debating whether I want to do that via Tide or Eglot, but I am not in much rush yet. I can survive the little Svelte coding I do without completions for a while.
Workspaces took a little bit of work. I looked at the Doom-Emacs "workspaces" module to figure out how they worked, but the code in there was fairly custom. However, it lead me to persp-mode and then Perspective which I ended up using by simply binding "SPC <tab>" to the perspective-map. It works close enough to how workspaces in Doom-Emacs work that I am happy with it.
Conclusion
At the end of this quest I had managed to configure myself an Emacs that works very closely to how my Doom-Emacs config worked, except with significantly fewer packages in total. If you want to have a look at my Emacs config, you can find it in my Codeberg dotfiles.
Configuring Emacs turned out to not be nearly as difficult as I'd expected it to be. Last time I tried it, I got overwhelmed and bailed out fairly quickly, but equipped with more experience and patience, I was able to get things configured fairly well.
I did take advantage of Emacs From Scratch to get started and I intend to still go through the rest of the series to learn more tips and tricks. But I think I'm reaching a point where I have learned enough things to know fairly well how my config is working and how to build up on it. I was also quite surprised how far you can get with relatively little. My config right now is about 220 lines and it already exceeds what my NeoVim config back in the day was able to do with about the same amount of config. I have also most of my typical use-cases covered from Doom-Emacs and a good grasp of how to build up more as I need.
I've also done a bit of testing with the Toolbox-oriented workflow and it seems to be functional enough for me. I don't entirely know if I will ultimately stick with it, but at this point it will be fairly easy to take my custom Emacs config with me inside the Toolbox if needed.
For now I think I'll try to stick to Emacs Flatpak + Toolboxes for my workflow and see how it ultimately turns out. But even if it didn't, I had a good bit of fun configuring and tweaking my Emacs config and learned a few things. Next thing I'll need to do is separate out my dev environments from the omni-Toolbox to be more project-specific.