Mathias' Blog

Occasional musings and technical posts…

03 Jan 2020

Concerning LXD, snaps, and Debian

I’ve been a long-time user of LXC, going all the way back to 2014. It’s come a long way and is a great containerization technology. I use it extensively in my projects and regularly create and destroy containers as needed. The primary Linux distro I run is Debian stable, and I do my best to not pollute it with backports or non-Debian packages. This means I usually trail behind the cutting edge, but in return I have very stable systems. The first official release of Debian to ship lxc was jessie.

This post will cover my experience getting LXD 3.0 LTS running on a Debian buster system, as well as highlight concerns I have with the current path upstream LXD is taking for its packaging and distribution.

LXC to LXD

While the first stable release of LXD was in 2016, I didn’t do much with it until this past fall. The various (legacy) lxc-* commands continue to work fine, and over the past few years I had built up quite a collection of local scripts and auto-config tooling around the underlying LXC that provided everything I needed for my container usage.

However, I knew I needed to rebuild one of my physical hardware servers due to some hardware upgrades, and thought that would make an excellent opportunity to try out LXD as well. Unfortunately LXD still isn’t included in Debian, but work is being done to get it included in the next stable release. LXD has a really cool online demo that I was able to play with and verify that pretty much all of my wrappers around LXC are no longer needed. Additionally, I’m running the 3.0 LTS release, as I don’t want to be updating every couple of months.

The currently recommended way of installing LXD is to use snaps. This is a package manager designed to distribute self-contained Linux applications, similar to AppImage and Flatpak. Rather than a traditional deb or RPM package, a snap contains all its dependencies and is by default very limited in what resources it can access. This is a plus for security, and can also make distributing an application across multiple Linux distros much easier.

I setup a test system to try out snapd and LXD before rebuilding my server. Debian has snapd already packaged, and it was pretty simple to get installed, followed by the snap of LXD. Things worked well, and in testing over Thanksgiving weekend things looked promising. After achieving basic proof-of-concept usage of LXD, I started testing more complex setups that mimicked my actual usage of LXC. This is where I found some issues with the snap-approach of installing a service.

Issues with snaps

While snaps are pretty cool, there are three reasons why I ultimately decided against installing LXD via the snap route:

  • The most significant reason, and what seems to me (and others) to be a major issue with snapd is that there is no way to disable automatic updates of installed snaps. Don’t believe me? Just do a quick Google search for “snap disable updates”. This means that, at any arbitrary time, an update to an installed snap could be released, which will then cause the service, such as lxd, to restart. There are documented cases where this causes massive, unexpected downtime. snapd is advertised as being ready for production use, but until disabling auto-updates is supported, I won’t use it.

  • The directory structure used by snaps seems a little convoluted. Plus, each snap will have a bindmount setup. While not every Linux distro strictly follows the Filesystem Hierarchy Standard, I do appreciate software that follows the principle of least surprise.

  • Lastly, and this is purely my own opinion, snapd is too much of a Canonical project. There doesn’t seem to be widespread adoption of snaps beyond Ubuntu, and a lot of snapd’s capabilities seem to work best on Ubuntu. I can’t complain too much, as Canonical is paying the main developers who work on snapd. I just hope it doesn’t end up like the numerous Canonical projects that get dropped after a few years.

LXD on Debian

So, since snapd was ruled out, I had to turn back to a traditional deb-based install of LXD. Currently, it’s not possible to build LXD on Debian buster due to missing golang dependencies, but there are packages for Ubuntu Bionic (18.04LTS). The stars are aligned between this release and buster, which makes it relatively easy to use bionic packages on a Debian system.

Packages installed:

liblxc1_3.0.3-0ubuntu1~18.04.1_amd64.deb
liblxc-common_3.0.3-0ubuntu1~18.04.1_amd64.deb
lxcfs_3.0.3-0ubuntu1~18.04.1_amd64.deb
lxd_3.0.3-0ubuntu1~18.04.1_amd64.deb
lxd-client_3.0.3-0ubuntu1~18.04.1_amd64.deb
lxd-tools_3.0.3-0ubuntu1~18.04.1_amd64.deb
python3-lxc_3.0.1-0ubuntu2_amd64.deb (from Ubuntu disco to match python3.7 in Debian buster)

Warning! FrankenDebian ahead!

Obligatory warning: This flirts with FrankenDebian. If something breaks, you’ll be on your own. Your system will be in an unsupported state which will be significantly different from a standard install. You’ve been warned.

Updating and building LXD debs

After installing these packages from bionic, I had a working LXD on Debian buster. (I did have to manually pin the versions after install with apt-mark hold liblxc1 python3-lxc lxcfs to prevent apt from trying to install the versions in Debian that have a newer epoch.) Things were looking up, except that services such as MariaDB couldn’t start within containers and were erroring out with a status of (code=exited, status=226/NAMESPACE) in the systemd logs. A quick Google search found Debian bug #918839, from which it looked like this was an issue with a newer version of systemd and AppArmour. Luckily this was fixed in LXD version 3.0.4.

But 3.0.4 hasn’t been uploaded to bionic, and there’s a languishing bug on launchpad about this. The upstream project seems to only be releasing updated snaps now, and no longer updating debs.

Luckily, this being an open source project, we can build updated packages ourselves! As I’m very comfortable rebuilding debs, this didn’t seem too daunting. Briefly:

After this was done, I had a fully functioning LXD install on Debian buster. Life is good.

Conclusion

LXC is cool; LXD is even cooler. The focus by the upstream developers on only releasing snaps of new versions is concerning, as snapd is clearly not ready for use in “production” environments. However, since it’s an open source project, it’s easy to update and rebuild debs to get the latest and greatest. I’ve now got LXD 3.0 running on Debian buster, and it is indeed nicer than my wrappers. :)

Tags

debian lxc snapd

← Charitable Giving
SpaceX Dragon Launch Escape Demonstration →