As this year’s Google Summer of Code has ended, it’s time for me to summarize all my work at Gentoo Linux foundation for Google Summer of Code 2022. In a nutshell, I was able to complete most of my deliverables, except the miscellaneous idea of killing the musl repository due to technical limitations.
Me using GNOME on musl
Contents
Overview of my work
The main focus of my work for GSoC 2022 was working towards getting a full GNOME desktop installed and then getting it working on Gentoo Linux, and in general, the idea was to expand support for packages and software on musl profile. So these included fixing things like fixing build-time time issues and dependencies. Much of these issues were due to packages relying on glibc-specific headers and functions.
To give some examples, I very often came across packages that used headers like error.h and execinfo.h. The error.h function is mainly used for error reporting using the error function from the aforementioned header. The execinfo.h is used to provide backtraces from programs using the function named backtrace from the execinfo.h header file. These can be avoided relatively easily by replacing the error.h header with err.h header and doing the same for the relevant function call. Also, there is the possibility of using the build system of the package (for example meson, gnu Autotools, CMake) to detect whether the header file is available on the current system and then add a definition which then can be used to use either err.h or error.h and maybe even do something clever like this:
#ifdef HAVE_ERROR_H #include <error.h> #else #include <err.h> #define error(status, errno, ...) err(status, __VA_ARGS__) #endif
For the execinfo, I mostly just avoided the backtrace section of the code. But user catcream has some standalone that can help get programs using execinfo.h and backtrace functions building on musl.
Another type of issue (very rarely) I came across was programs segfaulting due to the default stack size of musl libc which happens to be much smaller than Glibc. A great example of this issue would be the GNOME shell which just refused to start and crashed with a fail whale screen musl. A simple fix was to check if the system was using musl libc and increase the stack size using the following LDFLAGS:
-Wl,--as-needed -Wl,-z,stack-size=2097152
For the most part, I did send some of my patches for Gentoo upstream. But there are still parts of my work that I’ve yet to submit upstream.
- You can find most of my work and pull requests on GitHub.
- My GitHub profile
- My GitLab profile, that I used to submit samba and other GNOME-related patches
What I learned from GSoC
During GSoC working at Gentoo, I’ve learned and picked up a lot of good habits, mostly from my mentor/s and other developers. I’m going to try my best to put them in words below.
- First and foremost I learned about how development takes place for Gentoo, these include how packages are patched, updated, and tested. Got familiar with using and writing ebuilds.
- Secondly, I got more familiar with how programs work under the hood on Linux. Starting at what a libc is, how many types of libc are there currently, and how they differ. Why is an executable built with one libc can’t be used with another libc. Last but not least what linking is and how the two kinds of linking differ from one another.
- How to figure things out on my own. My mentor was there for me most of the time, but since he is human with a job and other responsibilities, he couldn’t be there for me all of the time. So for those situations, I had to do my own research and come up with a solution that I would later get reviewed by him. I can confidently say that these were the situations where I learned the most.
- How to articulately put down my thoughts so that other people could read and understand them. You may have guessed from reading this report till now that I struggle with putting down what’s on my mind articulately. Since I’m not interacting with my mentor, I had to articulately put down my thoughts on a particular problem or the problem itself so that my mentor and other devs could understand exactly what I’m saying.
- How to patch software/packages (a bit specific to ebuilds but this knowledge can be applied elsewhere too). Specifically, how to quickly create a patch and apply it using git. Parts of the .diff/.patch files are and aren’t relevant.
- Getting more familiar with the very powerful git. I did know the basics of git but never to the extent that I now know. From resolving merge conflicts, and rebasing to squashing commits; it is safe to say I’m now fairly confident in git.
- Observing and understanding failures, specially build time failures. There were times during the initial stages I couldn’t understand where the build has failed and couldn’t decrypt the failure message. As time went on I got more competent at reading logs and understanding build failures better. I’m far from being a pro though.
- Getting more familiar with C and C++ programming. I was by means an expert in C and C++. I knew about the basics and some other stuff, and throughout my GSoC, I got to know much more about C and C++. For example, how to override macros, how to detect what kind/version of libc the system is running … etc
- Another thing I’ve picked up is getting familiar with gdb and core dumps. I’ve had programs segfault on me and without using tools like gdb it can be very challenging to debug stuff. Getting used to debugging using gdb and core dumps has saved me countless hours using.
There were other good practices too but for the sake of this report I’ll leave them out.
My weekly work
I’ll try to put down my weekly work for the whole 12 weeks in brief and points while linking the archived emails of the actual report
Week 1
For the first two weeks, I mainly planned on focusing on tiring installing the gnome-light meta package and fixing any packages that are part of the meta package, that failed. I did come across four issues, the following packages needed patching:
- Samba: Samba was using some defines that were part of the glibc and not available on other libcs
- ppp: Similar to samba, ppp was missing a header (net/ppp_defs.h) which was a part of glibc
- librsvg: The newer librsvg was crashing on musl while trying to render to particular svg for GNOME shell
- gnome-terminal: The gnome-terminal was missing the W_EXITCODE define on musl, so I had to define it.
Week 2
For the second week I worked further on debugging the librsvg issue, but noting came fruitful came out of it, but what worked was increasing the stack size of either gjs or gnome-shell, so I went with increasing the stack size of gjs. Then I began learning about Gentoo’s package testing tools like tatt and pkg-testing-tools. Then having a few extra hours at hand I tried compiling firefox:rapid on musl, though with -clang and -pgo
Week 3
For week 3, I made sure that GNOME desktop can now be installed and used under musl profile on Gentoo. However, using Xorg didn’t work due to some race conditions (guessing), so I had to stick with Wayland.
Then as per my proposal, I started emerging MATE desktop as it was very close to GNOME. I did come across one or two packages that needed patching, so I patched them and sent them upstream. Namely the mate-power-manager and mate-sensors-applet. And then I went on testing MATE desktop for the next week.
Week 4
I tested both GNOME and MATE desktop by using them myself, and I didn’t notice anything major, yes there were some minor bugs like the gnome settings crashing upon clicking on some particular menu options.
With most of GNOME and MATE patched I started patching other tools/packages for musl, namely:
- geant
- debugedit
- google-perftools
- netcdf
- hpx
I learned how to version bump packages and worked further for upcoming midterm evaluations.
Week 5
For week 5 I focused more on fixing miscellaneous packages. Firstly I refined the debugedit patch to reduce its size. After that, while fixing the mysql-connector-c package I learned about how to use build systems to check presence of various headers on the currently running system. For example in CMake one can use the check_symbol_exists syntax to check the presence of a header, here is an example syntax:
check_symbol_exists(res_ninit "resolv.h" HAVE_RES_NINIT_FUNCTION) check_symbol_exists(res_nsearch "resolv.h" HAVE_RES_NSEARCH_FUNCTION) check_symbol_exists(res_nclose "resolv.h" HAVE_RES_NCLOSE_FUNCTION) IF (HAVE_RES_NINIT_FUNCTION) add_compile_definitions(HAVE_RES_NINIT) ENDIF(HAVE_RES_NINIT_FUNCTION) IF (HAVE_RES_NSEARCH_FUNCTION) add_compile_definitions(HAVE_RES_NSEARCH) ENDIF(HAVE_RES_NSEARCH_FUNCTION) IF (HAVE_RES_NCLOSE_FUNCTION) add_compile_definitions(HAVE_RES_NCLOSE) ENDIF(HAVE_RES_NCLOSE_FUNCTION)
With this, the CMake build systems will check the resolv.h file for
presence of res_ninit, res_nsearch, and res_nclose functions and set the
compile definitions accordingly. We can then use this definition to use
the _res_n*_ functions which are thread-safe or the non-thread-safe
alternative ones.
And similarly, I learned about checking the presence of a header file for using the meson build system while solving the libratbag package. An example syntax for meson would be:
error_exists = cc.has_header('error.h')
if error_exists
add_global_arguments('-DHAVE_ERROR_H', language : 'c')
endif
Apart from these, I tried upstreaming the Samba patch.
Week 6
For week six I moved my main workstation to the Gentoo musl profile. I did face some challenges, for example, proprietary drivers and an embedded SDK not working due to them being linked against Glibc. So I just created a chroot and a lxc container for my work.
I came across two bugs, first was xterm not opening with an error message open ttydev: I/O error and Cinnamon desktop not running. the Cinnamon bug was similar to the GNOME bug on Xorg (race condition and only one frame being rendered at a time).
I also patched a bunch of other miscellaneous packages too.
Week 7
I got my midterm evaluation result and fortunately, I passed. Sam left some suggestions for me and I tried my best to adhere to them for the rest of my GSoC journey.
My deliverables were complete, hence I started working with developer blueness to help him with the GRS project and simultaneously work on closing the bugs in the Gentoo Bugzilla musl tracker.
Not to mention, this is where I slacked off a bit and Sam helped me get started again.
Week 8
This week I mainly focused on getting firefox working with clang. Firefox seemed to have an issue with libc++ and clang on musl and failed with a few headers being not found (for example limit.h).
Unfortunately, Firefox kept me busy, and I couldn’t focus on other bugs.
Week 9
For this week I mainly helped my mentor Sam with some bugs package folly and watchman. These are packages from Facebook and Sam used them in personal projects.
For the package folly, it was expecting a define, `__ELF_NATIVE_CLASS`, on
systems other than FreeBSD, the `__ELF_NATIVE_CLASS` is also used to define
another `FOLLY_ELF_NATIVE_CLASS`. But unfortunately, on musl it was not the
case, and as a result, `FOLLY_ELF_NATIVE_CLASS` was not being defined.
For watchman, the developers used `__fswort_t` as a type for a variable
but unfortunately, this is an internal Glibc type and cannot be used on
other libc’s.
Then I worked on getting blender compiling on musl. It was honestly quite fun. I have upstreamed my blender patch and now blender can be compiled on musl without needing any patching. One can find my patch here.
From this week I took upon fixing failing tests. Sam asked me to take a look at three main packages that seemed to fail, namely:
- zsh
- Git
- Midnight Commander
I looked at all of them and then began with Midnight Commander and ZSH since these two seemed relatively easier than Git. And for git, the test seemed to fail for the ebuild only while the upstream version seemed fine.
For week 11 I mainly fixed the broken ZSH tests on musl. The zsh tests A03quoting.ztst, B03print.ztst, V09datetime.ztst,
and E02xtrace.ztst were failing. This took longer than expected for me to complete.
Mainly worked on submitting my patched upstream and played around with musl support on other architectures. Had a talk with Sam to discuss going forward and where I could help and my idea of being part of the package testing team, specifically focusing on musl profile.
With this, I wrapped up my work for Google Summer of Code 2022
Things I want to work on after GSoC
After GSoC, I mainly want to participate in package testing while mainly focusing on musl profile. I’ve seen Gentoo devs using tinderboxes with musl for package testing. Much of my inspiration comes from developer toralf who seems to play a major part in reporting bugs in Gentoo packages/ebuilds.
Apart from that, I want to continue my work on musl and GNOME, helping developers like blueness with projects such as GRS and just be part of Gentoo development as a whole.
My thoughts on the project
I am a GNOME user myself and that was a part of the reason for choosing GNOME as the primary focus of GSoC work. I had lots of fun and learned a lot while doing this project.
I find the musl libc very interesting and would very much continue my work/supporting musl bugs.
About me
If you want to know a bit about me or want to contact me, you can find relevant information here.