A new Gentoo stage4: musl + clang

I’m glad to announce that I just successfully deployed the GNU-free toolchain, which I’ve been working on so far, into a musl-based Gentoo stage4. Everything in this stage4, except for the kernel, is built by clang with non-GNU runtime libs. I’m now using this as my main Linux system, and luckily nothing breaks so far 🙂

To be more specific, the runtime environments of this stage4 is composed of the following components:

  • C runtime: musl
  • C++ runtime
    • C++ stdlib: libc++
    • C++ ABI lib: libcxxrt [1]
    • stack unwinding lib: libunwind
  • compiler: clang

Let’s take program /usr/bin/ld.gold for example, which is written in C++, and see its runtime dependencies:

$ ldd /usr/bin/ld.gold
	/lib/ld-musl-x86_64.so.1 (0x559f8b598000)
	libz.so.1 => /lib/libz.so.1 (0x7f5a5b7a9000)
	libc++.so.1 => /usr/lib/libc++.so.1 (0x7f5a5b6ea000)
	libcxxrt.so.1 => /usr/lib/libcxxrt.so.1 (0x7f5a5b4cd000)
	libunwind.so.1 => /usr/lib/libunwind.so.1 (0x7f5a5b4c4000)
	libc.so => /lib/ld-musl-x86_64.so.1 (0x559f8b598000)

Apparently none of the above libs is related to gcc. So this program is just free from gcc!

Actually I’ve already figured out how to glue clang and those non-GNU C++ runtime libs together weeks ago; but this is the first time I put it into serious use. The most challenging part is to rebuild @world with this toolchain; I’m really nervous that countless packages get broken after I issue the command emerge -e @world. Surprisingly enough, only two packages are broken.

The first broken package is iproute2, which depends on libmnl and libmnl is mis-compiled by clang. libmnl has an unpleasant past with clang [2], but its new versions are fine. So the fix is simple: just upgrade libmnl to the latest version available (use ~amd64).

The second broken package, ironically, is clang/llvm itself. It turned out the libunwind I’m using lacks a few functions essential to llvm. Actually there’re two versions of libunwind available: one is from nongnu [3] which already exists in Gentoo’s repo for a while; the other one is developed by LLVM and is almost functionally equivalent to the former. Initially I just used the nongnu version, since there’s no package for LLVM’s libunwind yet. Unfortunately, the nongnu one works fine with everything else except for llvm itself. I just had to stop being lazy and wrote an ebuild for LLVM’s libunwind. Luckily again, it has everything we need, including those functions missing in the other libunwind.

After that, while I’m using this new system, I encountered a few other broken packages. But they’re either incompatible with musl, or the gold linker (yes, it’s the /usr/bin/ld.gold I just showed you). There’s nothing wrong with clang or the C++ runtime libs.

I also found an interesting fact: the building of gcc involves several steps of bootstrapping and the final executable has no dependence on any external C++ runtime libs, though gcc itself written in C++. This means gcc still works even clang or the C++ runtime gets broken. So I don’t need to worry about breaking packages since I can still rebuild them with gcc 🙂

Now that I have a working stage4, the next step is to make gcc-config support clang, so clang can act like a real native compiler.

Stay tuned!

 

[1] libcxxrt is functionally equivalent to libc++abi, and is planned to be replaced by the latter later in this project
[2] https://bugs.chromium.org/p/chromium/issues/detail?id=548786
[3] http://www.nongnu.org/libunwind/

14 thoughts on “A new Gentoo stage4: musl + clang”

  1. I see you are using gold (originally developed by Google, now part of the GNU Project) and suppose you use other GNU binutils too.
    If you are building a completely GNU-free system, you should try using alternative set of binary utilities. Fortunately, LLVM Project provides most of them, but the linker is a problem. LLVM linker LLD is not yet in usable condition, you can try mcld: https://github.com/mclinker/mclinker

    1. Thanks for the info! It seems mcld is specifically designed for mobile devices. Perhaps we can wait lld to come into good shape, and then replace binutils with LLVM’s offerings once and for all 😀

      1. That would be very nice if everything goes well. Then the only GNU component in your system would be GNU Make (I don’t know how to replace it, projects like pymake and makepp exist and claim to support GNU syntax, but not really), but you can get rid of it when you are done compiling and make is just running the commands, not embedding a part of itself to every binary.
        You can also take a look at the ELF Toolchain (https://sourceforge.net/p/elftoolchain/wiki/Home/), but you will need BSD Make to compile it.

        1. BSDs seem to have their own implementation of Make. I don’t know if they work on Linux.

          1. BSD Make works on Linux just fine. Most distros have it as bmake, but unfortunately it is not compatible with GNU Makefile syntax.

  2. Hi!
    I’m trying to reach the same goal, I’m trying to make my desktop functional with musl+clang. You can see my current chroot /etc/portage on github, musl branch:
    https://github.com/fsvm88/gentoo-portage_etc/
    Note that lots of patches are either experimental or broken (networkmanager). Some USEs have been disabled due to complicated patching (rp-pppoe comes to mind).
    You can find me in gentoo-dev on IRC, snarfer88 🙂

  3. Pingback: Use clang as a native compiler in Gentoo – GSoC 2016 project

  4. lld is already in a good shape, enough that it will be provided as alternative linker in FreeBSD (it can link all the base system + kernel on amd64 as per today). You might want to give it a try, and report bugs in case something goes wrong =)

    1. Well, I just played with lld for a bit and found its build system to be pretty immature at this point:
      – it can’t be build out of LLVM tree
      – even if built in tree, it can’t be installed with a simple `make install`

      With these drawbacks, I’m afraid it’s difficult to create a decent Gentoo package for it. So we’ll have to wait a little longer…

      1. There is an lld ebuild in some overlay (https://gpo.zugaina.org/app-arch/lld).

        I was able to merge it using this ebuild. The ebuild seems to be a little outdated, so I had to add a single line to make it work (I don’t exactly remember what it was but it was pretty straight forward).

        I further edited the ebuild to use the official 3.9.0 rather than the trunk.

        I got a working lld and tweaked the links to ld to enable it system-wide per default. The only reason why I deleted it again was that some packages did not build with it and it was a lot of hassle to switch back and forth between gold and lld (binutils-config –linker didn’t obviously work for lld).

        So, this is a little too much for my limited knowledge of gentoo internals, BUT if someone took this overlay’s ebuild and tweaked it into good shape (and provided a way to switch between lld and gold / bfd with a single command), we might already have got most of the job done.

        I read that chromium already compiles well using lld and as dcci already said, it already links about everything on FreeBSD.

        In my opinion, gentoo is missing lld support as of now and the experimental ebuild from the overlay SEEMS to be a good place to start.

        I am absolutely willing to contribute here, I just can’t do it all alone. zlei, Maybe you want to look into this once again when you have some spare time and let me know if there’s just about anything I can do to help?

        1. Writing an ebuild for lld shouldn’t be too much trouble. It’s just that lld’s source in its current status is highly coupled with llvm – you need to download llvm’s whole source tree besides lld’s to build lld. That is probably against the philosophy endorsed by the current maintainers of llvm components in Gentoo.

          If you’re familiar with llvm’s cmake system and have the time, maybe you can try to decouple lld from llvm: to make lld buildable without llvm’s source. I think the upstream would be happy to receive a decent patch addressing this issue.

          Otherwise, we might have to wait for the upstream guys to work on the decoupling, after which we could then improve binutils-config to support lld.

    2. I tried linking kernel on amd64 with lld, but no luck. Could you please share your kernel config and tell the version of lld you are using?

      1. I just realized that the kernel mentioned was FreeBSD kernel, not Linux. At this point it is not possible to link Linux kernel with LLD (mainline kernel is not compatible with clang at all).
        I tried linking Linux with LLD and it didn’t build because of lack of `-q` option.

Comments are closed.