Gentoo Linux in a Docker container

I have been using Docker for ebuild development for quite a while and absolutely love it, mostly because how easy it is to manipulate filesystem state with it. Work on several separate ebuilds in parallel? Just spin up several containers. Clean up once I’m done? Happens automatically when I close the container. Come back to something later? One docker commit invocation and I’m done. I could of course do something similar with virtual machines (and indeed I have to for cross-platform work) – but for native amd64 is is extremely convenient.

There is, however, one catch. By default processes running in a Docker container are fairly restricted privilege-wise and the Gentoo sandbox uses ptrace(). Result? By default, certain ebuilds (sys-libs/glibc and dev-libs/gobject-introspection , to name just two) will fail to emerge. One can of course set FEATURES=”-sandbox -usersandbox” for such ebuilds but it is an absolute no-no for both new ebuilds and any stabilisation work.

In the past working around this issue required messing with Docker security policies, which at least I found rather awkward. Fortunately since version 1.13.0 there has been a considerably easier way – simply pass

--cap-add=SYS_PTRACE

to docker-run. Done! Sandbox can now use ptrace() to its heart’s content.

Big Fat Warning: The reason why by default Docker restricts CAP_SYS_PTRACE is that a malicious program can use ptrace() to break out of the container it runs in. Do not grant this capability to containers unless you know what you are doing. Seriously.

Unfortunately the above is not the end of the story because at least as of version 1.13.0, Docker does not allow to enhance the capabilities of a docker-build job. Why is this a problem? For my own work I use a custom image which extends somewhat the official gentoo/stage3-amd64-hardened . One of the things my Dockerfile does is rsync the Portage tree and update @world so that my image contains a fully up-to-date stage3 even when the official base image does not. You can guess what happens when Docker tries to emerge an ebuild requiring the sandbox to use ptrace()… and remember, one of the packages containing such ebuilds is sys-libs/glibc . To my current knowledge the only way around this is to spin up a ptrace-enabled container using the latest good intermediate image left behind by docker-build and execute the remaining build steps manually. Not fun… Hope they will fix this some day.

 

8 thoughts on “Gentoo Linux in a Docker container”

  1. Hi,

    thanks for your blog post!
    I have experienced the same problems while trying to automate the build of some packages. I think I also had issued with ncurses at least with the version <6.
    My workaround is from the Gentoo wiki:
    https://wiki.gentoo.org/wiki/Knowledge_Base:Overriding_environment_variables_per_package
    I disable the sandboxes just for these packages. I'm sure you're aware of it, but maybe it helps someone who's in the same situation.

    Best regards
    Stefan

  2. Thanks for providing the link, it might indeed come in handy for some. Unfortunately disabling the sandbox isn’t quite suitable for Gentoo developers because the more ebuilds are emerged that way, the more the local system diverges from what most Gentoo users work on. In quite a few cases this is harmless – but still. That said, while using Docker for doing development *on* Gentoo rather than *of* Gentoo, selectively disabling the sandbox is a viable option and depending on the use case might in fact be preferable to granting container users access to ptrace().

  3. Do you deploy Gentoo-based containers? If so, does the toolchain tend to fatten the image? Also, having a toolchain seems a vulnerability, if the container is compromised.

    1. Like I mentioned earlier I personally use Docker for ebuild prototyping, meaning all my containers are short-lived and not publicly available. That said, the official _gentoo/stage3-amd64-hardened_ image is just under 900 MB in size so while considerably larger than _e.g._ Alpine it isn’t exactly huge… As for security implications of having the build toolchain installed, it is not specific to containers.

    2. Historically Docker and Gentoo based images are not a good match, unless you really don’t care about fairly huge images with a complete toolchain. Things are, in theory, a bit better now since Docker introduced nested builds:

      FROM gentoo-with-toolchain-image as builder
      RUN ROOT=/foo emerge something
      FROM some-small-footprint-image
      COPY –from builder /foo/* /

      You will still run into the caveats from this blog post though, another big limitation for Gentoo based images is the fact that you can’t mount external volumes during a Docker build.

      For now one can only resort to external tooling to resolve those issues.

  4. I was able to do an emerge -e @system after disabling sandboxing (somewhat berserk):

    RUN echo ‘FEATURES=”-sandbox -ipc-sandbox -network-sandbox -pid-sandbox -usersandbox”‘ >>/etc/portage/make.conf

    Of course, there may be other ways that ebuilds try to use ptrace. Meanwhile, I wonder whether libsandbox could be taught to not fail if ptrace is not available?

Leave a Reply

Your email address will not be published.