{"id":505,"date":"2023-06-27T04:46:22","date_gmt":"2023-06-27T04:46:22","guid":{"rendered":"https:\/\/blogs.gentoo.org\/gsoc\/?p=505"},"modified":"2023-06-27T04:50:38","modified_gmt":"2023-06-27T04:50:38","slug":"weekly-report-4-llvm-libc","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/gsoc\/2023\/06\/27\/weekly-report-4-llvm-libc\/","title":{"rendered":"Weekly report 4, LLVM libc"},"content":{"rendered":"<p>Hello! This is a combined report for both week 3 and 4.<\/p>\n<p>In these two weeks I&#8217;ve fixed several issues in LLVM libc, but quite a<br \/>\nlot of time has also been spent purely learning things. I will start<br \/>\nby going over what I&#8217;ve learned, and then refer to related issues.<\/p>\n<p><!--more--><\/p>\n<p>To start with I have gotten quite comfortable with CVise, how to use<br \/>\nit and general tricks about writing the test script for determining<br \/>\nwhether the issue is still there after reducing a source file. For<br \/>\nexample, I had an issue about a the print format macro PRId64 not<br \/>\nbeing defined on LLVM libc<\/p>\n<p>This caused an error that looked like this:<\/p>\n<p><code>\/home\/cat\/c\/llvm-project\/compiler-rt\/lib\/scudo\/standalone\/timing.h:182:22: error: expected ')'<br \/>\n   Str.append(\"%14\" PRId64 \".%\" PRId64 \"(ns) %-11s\", Integral, Fraction, \" \");<\/code><\/p>\n<p>So my first attempt of reducing was to grep for &#8220;expected &#8216;)'&#8221;. This<br \/>\nwent on to reduce the source file to simply: &#8220;(&#8220;. Maybe not the most<br \/>\ninteresting thing, but it was the &#8220;aha-moment&#8221; for me with regards to<br \/>\nCVise, because what it did with the test script became clear.<\/p>\n<p>To actually fix this issue I filed a bug<br \/>\n<a href=\"https:\/\/github.com\/llvm\/llvm-project\/issues\/63317\">https:\/\/github.com\/llvm\/llvm-project\/issues\/63317<\/a> and got told by a<br \/>\ncompiler-rt developer that the timing.cpp file is only used for<br \/>\nperformance evaluation. So the temporary fix I made was to exclude it<br \/>\nfrom the build by checking for LLVM_LIBC_INCLUDE_SCUDO=ON in CMake<br \/>\nuntil the print format macros are added to LLVM libc.<\/p>\n<p><a href=\"https:\/\/reviews.llvm.org\/D152979\">https:\/\/reviews.llvm.org\/D152979<\/a><br \/>\n<a href=\"https:\/\/github.com\/llvm\/llvm-project\/commit\/63eb7c4e6620279c63bd42d13177a94928cabb3c\">https:\/\/github.com\/llvm\/llvm-project\/commit\/63eb7c4e6620279c63bd42d13177a94928cabb3c<\/a><\/p>\n<p>The next thing I&#8217;ve learned a lot about is C++ and standard C header<br \/>\ninteroperability, or &#8220;include hell&#8221;. I learned about the differences<br \/>\nbetween C++ standard headers like &#8220;cwhatever&#8221; and &#8220;whatever.h&#8221;, also<br \/>\nwhat #include_next did, and also that compilers ship their own header<br \/>\nfiles like stddef.h and inttypes.h.<\/p>\n<p>I first ran into this when pulling new commits from master and rebuilt<br \/>\nLLVM libc, thinking that the errors were related to this. Weirdly<br \/>\nenough the original error just went away, and I couldn&#8217;t reproduce it<br \/>\nat all. But I quickly ran in to a similar issue when compiling LLVM<br \/>\nlibc in fullbuild mode on a llvm\/musl system.<\/p>\n<p>This time it was an error about wint_t not being defined:<\/p>\n<p><code>&gt; \/llvm-project\/build-libc-full\/projects\/libc\/include\/wchar.h:21:11: error: unknown type name 'wint_t'<br \/>\n&gt; int wctob(wint_t) __NOEXCEPT;<\/code><\/p>\n<p>The issue here arrises because LLVM libc&#8217;s llvm-libc-types\/wint_t.h<br \/>\ngets the wint_t type using:<\/p>\n<p><code>&gt; #define __need_wint_t<br \/>\n&gt; #include<br \/>\n&gt; #undef __need_wint_t<\/code><\/p>\n<p>This depends on internal behaviour of the stddef.h header. Because<br \/>\nthis is C++ it will include in this case libc++&#8217;s stddef.h, but this<br \/>\n#include_next&#8217;s the second stddef.h in the include search path.<\/p>\n<p>glibc uses __need_wint_t to make stddef.h define wint_t, while musl<br \/>\nuses __NEED_wint_t. No one is wrong here, as it is libc internals that<br \/>\nshould not be used by end users, instead something like wchar.h should<br \/>\nbe included. However, as this is a libc implementation too it does not<br \/>\nmake sense to include all of that stuff, so something else must be<br \/>\ndone. I then grepped the whole llvm checkout for stddef.h and realized<br \/>\nthat Clang shipped its own stddef.h too. This header, like glibc, uses<br \/>\n__need_wint_t to define wint_t, which is exactly what I want. I posted<br \/>\na bug report and got told that the internal Clang headers are to be<br \/>\nused, not the system libc&#8217;s headers, because of issues like these.<br \/>\n<code>https:\/\/github.com\/llvm\/llvm-project\/issues\/63510<\/code><\/p>\n<p>However, somehow \/usr\/include is higher up in the include order than<br \/>\n\/usr\/lib\/clang\/* even when using -ffreestanding, so I assume this is a<br \/>\nbug with the Gentoo llvm\/musl stage actually.<\/p>\n<p>&#8212;<\/p>\n<p>Another thing I have worked on is to replace some __unix__ ifdef<br \/>\nchecks in LLVM libc with __linux__. When looking through the source<br \/>\ncode there are quite a lot of places where these are mixed up<br \/>\nbizarrely enough. The most obvious ones are __unix__ check followed by<br \/>\nan #include . This is caused by &#8220;cargo cult&#8221; meaning that<br \/>\nthey once did it that way and stuck with it for no reason.<\/p>\n<p>I have fixed this here: <a href=\"https:\/\/reviews.llvm.org\/D153729#4447435\">https:\/\/reviews.llvm.org\/D153729#4447435<\/a>, but<br \/>\nI will revisit this because there&#8217;s a chance that macOS users could<br \/>\nhave used the typedefs pthread_once_t and once_flag, even though the<br \/>\nunderlying type __futex_word was supposed to be Linux only, because<br \/>\nfutexes here are Linux kernel specific. This would&#8217;ve<br \/>\npreviously not have errored out on macOS (__APPLE__) since __unix__ is<br \/>\ndefined, and __futex_word is just defined as an aligned 32 bit uint,<br \/>\nno unconditional kernel headers used that would&#8217;ve broken the build.<\/p>\n<p>I will therefore go back and define these for macOS later.<\/p>\n<p>I have also done some work on upstreaming things needed for Python<br \/>\ninto LLVM libc instead of just mashing everything into my Python<br \/>\nsource dir. The first one being the POSIX extension fdopen().<\/p>\n<p>As fopen is already implemented the hard part was not the function<br \/>\nitself, but actually figuring out where everything in LLVM libc was<br \/>\nplaced. Apart from the obvious declaration in the internal headers,<br \/>\nand corresponding source file, I also needed to make sure it was<br \/>\nusable in the libc. In total I needed to edit 7 files, like the<br \/>\nTableGen specifications, config\/$arch\/entrypoints.txt, libc<br \/>\nexposed internal header file, and of course CMakeLists.txt.<\/p>\n<p>This is not upstreamed yet but I am working on it here<br \/>\n<a href=\"https:\/\/reviews.llvm.org\/D153396\">https:\/\/reviews.llvm.org\/D153396<\/a>.<\/p>\n<p>The other thing I want to get upstreamed is the limits.h header, in my<br \/>\ncase needed for SSIZE_MAX. I have successfully made a tiny version in<br \/>\nmy libc tree that exposes some macros, and I will try to upstream what<br \/>\nI have and then work on things one by one. Similarly here, the hard<br \/>\npart was actually getting the header and macros to be exposed in the<br \/>\nlibc by editing build system code and specification files. I could<br \/>\nhave temporarily just jammed in a limits.h header file but I think<br \/>\nit&#8217;s important to get to know how LLVM libc does things\/&#8221;how the<br \/>\nboilerplate works&#8221; early in my project.<\/p>\n<p>&#8212;<\/p>\n<p>That&#8217;s all the big things, I also continued work on Python, fixed some<br \/>\nsmall stuff like a typo fix<br \/>\n(<a href=\"https:\/\/reviews.llvm.org\/rGc32ba7d5e00869de05d798ec8eb791bd1d7fb585\">https:\/\/reviews.llvm.org\/rGc32ba7d5e00869de05d798ec8eb791bd1d7fb585<\/a>),<br \/>\nadding Emacs support in llvm-common<br \/>\n(<a href=\"https:\/\/github.com\/gentoo\/gentoo\/pull\/31635\">https:\/\/github.com\/gentoo\/gentoo\/pull\/31635<\/a>)<br \/>\nand other &#8220;Gentoo but not really GSoC work&#8221;:<br \/>\n<a href=\"https:\/\/github.com\/gentoo\/gentoo\/pull\/31560\">https:\/\/github.com\/gentoo\/gentoo\/pull\/31560<\/a> (license fix, soju).<br \/>\n<a href=\"https:\/\/github.com\/gentoo\/gentoo\/pull\/30933\">https:\/\/github.com\/gentoo\/gentoo\/pull\/30933<\/a> (new package, senpai).<\/p>\n<p>&#8212;<\/p>\n<p>Next week I will work on getting Clang\/LLVM supported in<br \/>\nCrossdev. This will be done by first making sure that the hosts LLVM<br \/>\ntoolchain supports the target architecture via the LLVM_TARGETS<br \/>\nUSE-flag. Currently Clang on Gentoo can compile things like the<br \/>\nkernel, but anything that relies on runtime libraries, like libc,<br \/>\nfails due to compiler-rt not being compiled for the target triple, so<br \/>\nI will also make sure that Crossdev compiles compiler-rt for the<br \/>\nspecified target triple.<\/p>\n<p>&#8211; &#8212;<br \/>\ncatcream<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello! This is a combined report for both week 3 and 4. In these two weeks I&#8217;ve fixed several issues in LLVM libc, but quite a lot of time has also been spent purely learning things. I will start by &hellip; <a href=\"https:\/\/blogs.gentoo.org\/gsoc\/2023\/06\/27\/weekly-report-4-llvm-libc\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":177,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[17],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/posts\/505"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/users\/177"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/comments?post=505"}],"version-history":[{"count":3,"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/posts\/505\/revisions"}],"predecessor-version":[{"id":508,"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/posts\/505\/revisions\/508"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/media?parent=505"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/categories?post=505"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/gsoc\/wp-json\/wp\/v2\/tags?post=505"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}