PMS Test Suite: D-Bus → …?

The D-Bus communication method used now within PMS Test Suite shows more and more disadvantages over time. As the final evaluation’s approaching, it will be the final solution for the GSoC period of development. But after that I’ll probably jump straight to replacing it with something better.

Right now, I can list at least the following issues with my current IPC:

  • it requires the system bus to be running which may not be really useful for smaller systems,
  • …and which makes using it in a Prefix environment a painful experience,
  • …and makes running multiple instances of pmsts a random failure,
  • it limits the package to using Python 2, directly and indirectly (via GLib event loop),
  • …and the GLib Python event loop fails to propagate exceptions correctly,
  • …and it wants the PID out of subprocess.

Looking for a new solution

Before deciding which way to proceed, let’s take a look at what we exactly need to have. And we need:

  • an IPC mechanism which would work fine within limited ebuild environment (sandbox, userpriv),
  • …which would not require us to touch or prepare builddirs,
  • …which would work fine with failing (or not even being started) ebuilds as well,
  • and an event loop which would allow us to asynchronically communicate with ebuilds and wait for a single subprocess to finish,
  • …and it all has to work both with Python 2 & 3.

And it would be great if I could avoid introducing additional dependencies.


Right now, the most promising solution seems to be using asyncore Python module, and an UNIX socket. Considering that gentoopm is already able to provide us with the userpriv UIDs for all PMs, we can make the socket userpriv-aware and not world-writable.

We’d use asyncore.loop() then to handle comms, and a few secs timeout to check the subprocess for termination.

One remaining question is the socket path. We could either:

  1. just make it a well-known name,
  2. make it random and write to the eclass at generation time,
  3. make it random and pass through the environment.

The first solution has the disadvantage that only one instance of PMS test suite could be running at once. On the other hand, running more than one at once seems to be a bad idea anyway (unless they’re running a completely different test suites with unique ebuild names and separate repos). By using a common socket name, the other instance could just ping the first one and fail nicely rather than failing quite randomly.

And the third solution has the big disadvantage that we’re starting to rely on a random variable getting passed through PM. Although this is possible and most probably even will work, I don’t think it’s a good idea. And it could stop working at any point.

I’ll probably go with the first one. I guess gentoopm would have to provide us with root path too.

uam can now mount CDs and DVDs!

Today I have released uam-0.2. The new release adds a long-awaited feature — capability of mounting and unmounting ejectable media like CD and DVD disks. And it does so in a way much simpler than I expected.

But first, what is uam?

uam is my old project, dating back to the times of HAL. Although HAL is long gone, uam is still there shining. It is a simple, lightweight media automounter using udev rules only.

Unlike HAL (or udisks) it doesn’t introduce any additional daemons. It just installs a few udev rules and helper scripts. When a media device is added or removed, udev calls the scripts and they perform all the mount/umount operations as necessary. HAL/udisks not required anymore, neither do mounter daemons.

Isn’t that a very limited solution?

Of course uam can’t be as flexible as the HAL/udisks attempt. You can’t get it (easily) to do things like asking user for permission or password; well, it doesn’t even create mounted media icons on your desktop. But is that what you really want it to do?

You can tell uam is one of the plug & play apps. emerge uam, CR and newly-inserted media shall start appearing in /media. There’s a config file too. If you want to fine-tune it a little, there are a few more switches and options in /etc/udev/uam.conf. You can set mount options, mountpoint naming, device filtering…

But how does it handle CDs and DVDs without a daemon?

Before, it wasn’t possible to mount CDs without some kind of a polling daemon. HAL/udisks provided such a daemon; I was even considering adding such a daemon to uam. The other solution was to use sys-apps/pmount which allows unprivileged users to mount removable media.

None of these is any longer necessary. Nowadays, kernel can poll ejectable drives itself and report media change (and eject) events through udev. As of 0.2, uam handles those events and is able to mount CDs as well.

In order to do that, the kernel polling has to be enabled. This can be done either per-device:

echo 5000 > /sys/block/sr0/events_poll_msecs

or by setting a common polling interval as events_dfl_poll_msecs parameter to the block module:

echo 5000 > /sys/module/block/parameters/events_dfl_poll_msecs

The interval is specified in milliseconds, i.e. the above examples set it to 5 seconds. Smaller intervals result in a quicker mounting of CDs, larger result in less polling overhead.

PMS Test Suite: getting the test results

One of key problems in PMS Test Suite is getting actual test results. With the whole complexity of build process, including privilege dropping, sandbox, collision protection, auto-pretending it is not that easy to check whether a particular test succeeded without risking a lot of false positives.

The simple attempt: succeed or die!

The simplest method of all would be to assume the test is supposed to either complete and merge successfully or die. Although that will work in many cases, it has many limitations.

First of all, to make it work as expected, the actual test code has to be executed. If for some reason the test code is not executed, we end up with a false positive. Consider the test checking phase function execution order. If for some reason pkg_postinst() isn’t called at all, there is no way we could die about it.

Moreover, if a test is supposed to fail, we can’t be sure if it failed for our reason or with some random PM bug. We could try to implement some method of grabbing the failure message and parsing it but that would imply relying on a particular output format. That’s not really what I’m interested in.

On the other hand, that is most straightforward method of checking test results. It doesn’t introduce additional dependencies, is PM-safe and that’s why the most basic EbuildTestCase class of PMS Test Suite uses that. Well, to be more exact, it checks vardb before and after running the tests to see which ones were merged and which ones failed to.

Passing more complex test results

Due to the problems pointed out above, I’ve decided to introduce a more complex test result checking method. Originally, it was supposed to use files to store ebuild output but during early testing showed that that concept has a few weaknesses.

Most importantly, FEATURES=userpriv resulted in some phase functions being run as root and some other as portage user. I’ve decided that hacking permissions, sandbox and other potential obstacles to get that concept working was not worth the effort.

That’s why the current implementation uses D-Bus for communication between the actual tests and the test runner. I was a little surprised by the fact that neither Portage nor pkgcore had any trouble with letting the test code reach the system bus.

Right now, the DbusEbuildTestCase handles all necessary D-Bus integration. It creates an D-Bus object for each running test, integrates the pms-test-dbus eclass with tests and provides methods to submit and check the test results.

Not all D-Bus test cases have to actually submit any output. Simpler ones just ping the D-Bus object in pkg_setup() to let it know that the test was actually started. This avoids a case when a test is expected to die and is considered so because PM didn’t start it at all (e.g. due to insufficient permissions when emerge assumes --pretend).

systemd.eclass: use it in your packages!

Although systemd is not officially supported by Gentoo and it is not even in the tree yet, some of our packages already provide support for installing systemd units. This is because a few upstreams already supply and install systemd units in their packages.

I think it is important to support installing those units even before systemd is committed to gx86. This way, we can ensure users switching to systemd will have at least minimal working system without hacking packages around to get the necessary units. That’s why I’ve already committed the systemd.eclass.

My point here is that every package for which upstream officially supports systemd (and thus the configure script supports --with-systemdsystemunitdir) should use the systemd eclass. This way, we’re able to globally and universally control whether, where and how systemd units are installed.

Devs often refuse to use the eclass because it installs the units unconditionally. As I said earlier, and as I repeat now — that can be changed. Just take a part in the discussion that I tried to raise on the mailing list. The point of using the eclass is that we can change that and then have a way to let our users handle the change gracefully.

And please do not try to enforce your own conditionals on the eclass functions. Adding local USE=systemd is no solution. It just makes things harder for users for relatively small benefit.

PMS Test Suite: the project design

As the GSoC coding period starts tomorrow, today seems like the last good day to write a little about the project design. For that reason, I created a little diagram in dia:

PMS Test Suite design diagram

As you can see there, the project code is currently broken down into three repositories. The main code repository, called pms-test-suite.git is going to hold the core project code — main scripts, library-handling Python modules and so on. The pms-test-suite-library.git repository is going to contain the bundled test library, and pms-test-suite-overlay.git holds temporary test ebuilds and the eclass.

Looking from the left hand side, you can see a greyed out part with the test-generator script in the middle of it. This module is not a part of GSoC design goals but a possible future guideline. It is supposed to read the human-readable definitions from the Package Manager Specification, match them with helper data and create a complete test library. However, right now the project will simple use a bundled test library.

The test library will consist of modules describing various tests for PMS compliance. Each of these modules, called a test case will provide at least the following information:

  1. A human-readable test description (which will be reused in program output and ebuild DESCRIPTIONs,
  2. a list of relevant EAPIs for which ebuilds will be generated,
  3. definitions of ebuild-specific variables and phase functions necessary to perform the test,
  4. a test result checking function, possibly EAPI-conditional.

Those test cases will be reused by two other PMS Test Suite scripts. The ebuild-generator module will use them to generate the ebuilds resembling the particular tests, and the test-runner module will use them to check the test results.

The test-runner script will perform the most important task of testing a particular Package Manager. It is supposed to match the test cases with generated ebuilds (or even call ebuild-generator internally), run the Package Manager on them and collect the results. The results will be used then to determine whether the test case succeeded, failed or its result is ambiguous.

The test ebuilds will use an PM-independent method of exchanging information with the test-runner module. The internal details of this communication will be handled in the pms-test.eclass. Right now, this code uses D-Bus for this task.