The pointless art of subslots

The sub-slots feature of EAPI 5 was announced as if it was the ultimate solution to the problem of SONAME changes on library upgrades. However, the longer I see it, the more I believe that it is not really a good solution, and that it misses the actual issue targeting somewhere nearby.

The issue is likely well-known by most of the Gentoo users. Every time a library changes its ABI, it changes the SONAME (the filename programs link to) to avoid breaking existing programs. When the package is upgraded, the new version is installed under the new name, and the old one is removed. As a direct result, all applications linking to the old version become broken and need to be rebuilt.

The classic way of handling this is to run the revdep-rebuild tool. It takes a while to scan the system with it but it supposedly finds all broken executables and initiates a rebuild of them. Of course, the system is in broken state until all relevant packages are rebuilt, and sometimes they just fail to build…

As you can guess, this is far from being perfect. That’s why people tried to find a better solution, and a few solutions were actually implemented. I’d like to describe them in a quasi-chronological order.

Using slots with slot-operator deps

A perfect solution that has been long advocated by Exherbo developers. I’m not aware, though, if they ever used it themselves. I didn’t see an exact explanation of how they expect it to be done, therefore I am mostly explaining here how I think it could be done.

The idea is that every SONAME-version of the library uses a different slot. That is, every time the SONAME changes, you change slot as well. Using different slots for each SONAME means that the incompatible versions of the library can be installed in parallel until all applications are rebuilt. This has a few requirements though.

First of all, only the newest slot may install development files such as headers. This requires that every version bump is accompanied by a revision bump of the older version, dropping the development files. On each upgrade, user builds not only the new version but also rebuilds the older version.

To handle the upgrades without a small moment of breakage (and risk of longer breakage if a build fails), the package manager would need to build both packages before starting the merge process. I doubt that enforcing this is really possible right now.

Secondly, the ebuilds installing development files would need to block the older versions (in other slots) doing the same while keeping the versions lacking development files non-blocked.

To explain this better: let’s assume that we have: foo-1, foo-1-r1, foo-2, foo-2-r1, foo-3, … The -r0 versions have development files and -r1 versions don’t have them (they are just the upgrade compatibility ebuilds). Now, the blocker in foo-3 would need to block all the older -r0 versions and not -r1 ones.

In a real-life situation, there will likely be differing revision numbers as well. And I don’t know any way of handling this other than explicitly listing all blocked versions, one by one.

And in the end, reverse dependencies need to use a special slot-dependency operator which binds the dependency to the slot that was used during the package build. But it’s least of the problems, I believe.

The solution of preserved-libs

An another attempt of solving the issue was developed in portage-2.2. Although it is available in mainstream portage nowadays, it is still disabled by default due to a few bugs and the fact that some people believe it’s a hack.

The idea of preserved-libs is for the package manager to actually trace library linkage within installed programs and automatically preserve old versions of libraries as long as the relevant programs are not rebuilt to use the newer versions. As complex and as simple as that.

Preserving libraries this way doesn’t require any specific action from the package maintainer. Portage detects itself that a library with a new SONAME has been installed during an upgrade and preserves the old one. It also keeps track of all the consumers that link against the old version and remove it after the last one is rebuilt.

Of course it is not perfect. It can’t handle all kinds of incompatibilities, it won’t work outside the traditional executable-library linkage and the SONAME tracking is not perfect. But I believe this is the best solution we can have.

The nothing-new in sub-slots

Lately, a few developers who believed that preserved-libs is not supposed to go mainstream decided to implemented a different solution. After some discussion, the feature was quickly put into EAPI 5 and then started to be tested on the tree.

The problem is that it’s somehow a solution to the wrong problem. As far as I am concerned, the major issue with SONAMEs changing is that the system is broken between package rebuilds. Tangentially to this, sub-slots mostly address having to call tools like revdep-rebuild which is not a solution to the problem.

Basically all sub-slots do is forcing rebuild on a given set of reverse dependencies when the sub-slot of package changes. The rebuilds are pulled into the same dependency graph as the upgrade to be forced immediately after it.

I can agree that sub-slots have their uses. For example, xorg-server modules definitely benefit from them, and so may other cases which weren’t handled by preserved-libs already. For other cases the sub-slots are either not good enough (virtuals), redundant (regular libraries) or even broken (packages installing multiple libraries).

Aside from the xorg module benefit, I don’t see much use of sub-slots. On systems not having preserved-libs enabled, they may eventually remove the need for revdep-rebuild. On systems having preserved-libs, it can only result in needless or needlessly hurried rebuilds.

A short summary

So, we’re having two live solutions right now: one in preserved-libs, and other in sub-slots. The former addresses the issue of system being broken mid-upgrade, the latter removes (partially?) the need for calling an external tool. The former allows you to rebuild the affected packages at any convenient time, the latter forces you to do it right away.

What really worries me is that people are so opposed to preserved-libs, and at the same time accept a partial, mis-designed work called sub-slots that easily. Then advertise it without thoroughly explaining how and when to use it, and what are the problems with it. And, for example, unnecessarily rebuilding webkit-gtk regularly would be an important issue.

A particular result of that was visible when sub-slot support was introduced into app-text/poppler. That package installs a core library with quite an unstable ABI and a set of interface libraries with stable ABIs. External packages usually link with the latter.

When sub-slot support was enabled on poppler, all reverse dependencies were desired to use sub-slot matching. As a result, every poppler upgrade required needlessly rebuilding half of the system. The rev-deps were reverted but this only made people try to extend the sub-slots into a more complex and even less maintainable idea.

Is this really what we all want? Does it benefit us? And why the heck people reinvented library preservation in eclasses?!