The Answer to the Ultimate Question of QEMU chroots, argument pages, and binutils

is 46.

In a previous post I described how to patch QEMU to allow building binutils in a cross chroot. In there I increased the maximal number of argument pages to 64 because I was just after a quick fix. Today I finally bisected that, and the result is you need at least 46 for MAX_ARG_PAGES in order for binutils to build.

In bug 533882 it is discussed that LibreOffice requires an even larger number of pages. It is possible other packages also require such a large limit. Note that it may not be a good idea to increase the MAX_ARG_PAGES limit to an absurdly high number and leave it at that. A large amount of memory will be allocated in the target’s memory space and that may be a problem.

Hopefully QEMU switches to a dynamic limit someday like the kernel. In the meantime, my upcoming crossroot tool will offer a way to more easily deal with that.

Why we do not have nor want R packages in the tree

Here’s a bug and my response to it which both deserve a little bit more visibility than being buried under some random bug number. I’m actually surprised nobody complained about that before.

GNU R supports to run
> install.packages(‘ggplot2′)
in the R console as user. The library ggplot2 will then be installed in users home. Most distros like debian and the like provide a package per library.

First, thank you for pointing out that it is possible to install and maintain your own packages in your $HOME. It didn’t use to work, and the reason why it now does is a little further down but I will not spoil it.

Here’s my response:

Please, do not ever add R packages to the tree. There are thousands of them and they are mostly very badly written, to be polite. If you look at other distros you will see that they give an illusion of having some R packages, but almost all of them (if not all) are seriously lagging behind their respective upstream or simply unmaintained. The reason is that it’s a massive amount of very frustrating and pointless work.

Upstream recommends maintaining your packages yourself in your $HOME and we’ll go with that. I have sent patches a couple of years ago to fix the way this works, and now (as you can obviously see) it does work correctly on all distros, not just Gentoo. Also, real scientists usually like to lock down the exact versions of packages they use, which is not possible when they are maintained by a third party.

If you want to live on the edge then feel free to ask Benda Xu (heroxbd) for an access to the R overlay repository. It serves tens of thousands of ebuilds for R packages automatically converted from a number of sources. It mostly works, and helps in preserving a seemingly low but nonetheless functional level of mental sanity of your beloved volunteer developers.

That, or you maintain your own overlay of packages and have it added to layman.

While we are on that subject, I would like to publicly thank André Erdmann for the fantastic work he has done over the past few years. He wrote, and still occasionally updates, the magical software which runs behind the R overlay server. Thank you, André.

/bin/sh: Argument list too long

I tried building binutils-2.25 in a qemu chroot and I got the following error during the build process:

/bin/sh: Argument list too long

Google wasn’t helpful. So I looked at the code for qemu-2.2.0, which is where my static qemu binary comes from. At some point I stumbled on this line in linux-user/qemu.h:

#define MAX_ARG_PAGES 33

I changed that 33 to a 64, rebuilt, replaced the approriate static binary in my chroot, and the error went away.

Google Summer of Code 2015

TL;DR: Gentoo not selected for GSoC 2015 ‘to make way for new orgs”. All is not lost: some Gentoo projects will be available within other organizations.

As you may have already noted, the Gentoo Foundation was not selected as a mentoring organization for GSoC 2015. Many immediately started to speculate why that happened.

Today I had an opportunity to talk (on irc) to Carol Smith, from Google’s Open Source Programs Office. I asked her why we had been rejected, if they had seen any issue with our application to GSoC, and if she had comments about it. Here’s what her answer was:

yeah, i’m sorry that this is going to be disappointing
but this was just us trying to make way for new orgs this year :-(
i don’t see anything wrong with your ideas page or application, it looks good

Then I asked her the following:

one discussion we had after our rejection is if we should keep focusing on doing GSoC to attract contributors as we’ve been doing, or focus more on having projects actually be implemented, and how much you cared about it

To which she replied:

well, i’ll say that wasn’t a factor in this rejection
having said that, we in general like to see more new developers instead of the same ones year over year
we’d prefer gsoc was used to attract new members of the community
but like i said, that wasn’t a factor in your case

It’s pretty clear we haven’t done anything wrong, and that they like what we do and the way we do it. Which doesn’t mean we can’t improve, by the way. I know Carol well enough to be sure she was not dodging my questions to politely brush me aside. She says things as they are.

So, what happened then? First, the overall number of accepted organizations went down roughly 30% compared to last year. The immediate thought which comes to mind is “budget cut”. Maybe. But the team who organizes GSoC is largely the same year over year. You can’t indefinitely grow an organization at constant manpower. And last year was big.

Second, and probably the main reason why we were rejected is that this year small and/or newer organizations were favored. This was explicitly said by Carol (and I believe others) multiple times. I’m sure some of you will argue that this isn’t a good idea, but the fact is it’s their program and they run it the way they want. I will certainly not blame them. This does not mean no large organizations were selected, but that tough choices had to be made among them.

In my opinion, Carol’s lack of words to explain why we were not selected meant “not bad but not good enough”. The playing field is improving every year. We surely felt a little too confident and now have to step up our game. I have ideas for next year, these will be discussed in due time.

In the meantime, some Gentoo projects will be available within other organizations. I will not talk about what hasn’t been announced yet, but I can certainly make this one official:
glee: Gentoo-based Linux appliances on Minnowboard
If you’re interested, feel free to contact me directly.

Google Summer of Code 2015

This is a quick informational message about GSoC 2015.

The Gentoo Foundation is in the process of applying to GSoC 2015 as an organization. This is the 10th year we’ll participate to this very successful and exciting program.

Right now, we need you to propose project ideas. You do not need to be a developer to propose an idea. First, open this link in a new tab/window. Change the title My_new_idea in the URL to the actual title, load the page again, fill in all the information and save the article. Then, edit the ideas page and include a link to it. If you need any help with this, or advice regarding the description or your idea, come talk to us in #gentoo-soc on Freenode.


Tagaini Jisho

I haven’t been using Tagaini Jisho for long, but so far I’m impressed. Tagaini Jisho is a kanji dictionary and study tool. Here’s what the author has to say about it:

Tagaini Jisho is a free, open-source Japanese dictionary and kanji lookup tool that is available for Windows, MacOS X and Linux and aims at becoming your Japanese study assistant. It allows you to quickly search for entries and mark those that you wish to study, along with tags and personal notes. It also let you train entries you are studying and follows your progression in remembering them. Finally, it makes it easy to review entries you did not remember by listing them on screen or printing them on a small booklet.

Tagaini Jisho also features complete stroke order animations for more than 6000 kanji.

You can find the author’s website at

I have pushed version 1.0.2 to the tree. Note that there is a version of sqlite which is bundled with the source tarball. I tried to unbundle it but it wasn’t trivial (to me, at least), so I left it there for the time being. If you figure out how to properly unbundle sqlite, feel free to commit the changes yourself. As usual with my packages, don’t waste time asking if you can do it, just do it.

Forking ebuilds

Here’s a response to an email thread I sent recently. This was on a private alias but I’m not exposing the context or quoting anybody, so I’m not leaking anything but my own opinion which has no reason to be secret.

GLEP39 explicitly states that projects can be competing. I don’t see how you can exclude competing ebuilds from that since nothing prevents anybody from starting a project dedicated to maintaining an ebuild.

So, if you want to prevent devs from pushing competing ebuilds to the tree you have to change GLEP 39 first. No arguing or “hey all, hear my opinion” emails on whatever list will be able to change that.

Some are against forking ebuilds and object duplicating effort and lack of manpower. I will bluntly declare those people shortsighted. Territoriality is exactly what prevents us from getting more manpower. I’m interested in improving package X but developer A who maintains it is an ass and won’t yield on anything. At best I’ll just fork it in an overlay (with all the issues that having a package in an overlay entail, i.e. no QA, it’ll die pretty quickly, etc…), at worst I’m moving to Arch, or Exherbo, or else… What have we gained by not duplicating effort? We have gained negative manpower.

As long as forked ebuilds can cohabit peacefully in the tree using say a virtual (note: not talking about the devs here but about the packages) we should see them as progress. Gentoo is about choice. Let consumers, i.e. users and devs depending on the ebuild in various ways, have that choice. They’ll quickly make it known which one is best, at which point the failing ebuild will just die by itself. Let me say it again: Gentoo is about choice.

If it ever happened that devs of forked ebuilds could not cohabit peacefully on our lists or channels, then I would consider that a deliberate intention of not cooperating. As with any deliberate transgression of our rules if I were devrel lead right now I would simply retire all involved developers on the spot without warning. Note the use of the word “deliberate” here. It is important we allow devs to make mistakes, even encourage it. But we are adults. If one of us knowingly chooses to not play by the rules he or she should not be allowed to play. “Do not be an ass” is one of those rules. We’ve been there before with great success and it looks like we are going to have to go there again soon.

There you have it. You can start sending me your hate mail in 3… 2… 1…

Managing R packages

We still don’t have proper integration of CRAN and BIOC in Gentoo. I have proposed a GSoC project to remedy that. Feel free to contact me about it.

In the meantime if you need to install CRAN or BIOC packages you probably use install.packages() in R. This resolves R dependencies and installs all you need automatically. You are on your own about system dependencies though, but this is what CRAN integration in Portage is about. Upstream recommends you su root when installing R packages from within R. This is not always possible and I hope you will agree with me this is not the right thing to do. At best you will make your package manager cry due to collisions. At worst, well, it can be way worse. The recommended way should be for users to manage their R packages in their personal library. If you don’t have one, a personal library will be created for you the first time you use install.packages() or update.packages(). So there is nothing special to do and it mostly works.

One problem arises when a package you are installing depends on a version of a recommended package newer than the one you have. Recommended packages are some of those which are installed by Portage when you emerge R, thus they are owned by root. Non-root users can’t update them in the main library, which they don’t have access to, but they can do so in their personal library. However, there is a bug in R which will let you update recommended packages in your personal library, but only if you didn’t have one when you started the process. If you had one, as it’s most likely the case, the install will just fail. Frustrating.

About four weeks ago I silently added two patches to dev-lang/R-2.14.2 in order to fix this behavior. So far nobody complained. Yesterday I bumped R to 2.15.0 and the patches still apply cleanly. Please test and report back.

I have filed a bug upstream about these patches. Let’s see what they make out of them. The real solution to this problem though, at least as far as Gentoo is concerned, is to have Portage manage your R packages. And I’m sure it will come someday.


It’s Council election time again. So don’t forget to vote before Sunday at the latest, after church or pastis depending on your religion.

To my great surprise I was nominated again this year. To my even greater surprise I was encouraged by another developer, whose opinion matters to me a lot, to accept my nomination.

I was so not expecting to be nominated that the surprise made me consider accepting for a while. Those who know about my personal situation know how foolish that would be. It kept me thinking until the deadline though, which is why I didn’t formally decline on time. I consider this impolite so please accept this apology.

And since nobody is asking for my opinion here it is. What is important is that you vote, whoever you vote for. But it’s better if you vote right, and this year it’s quite easy.

Whoever is running but can’t be bothered to plan and write a manifesto doesn’t deserve your vote. That leaves 6 candidates.

Whoever doesn’t write a serious manifesto (by serious I mean “not a joke”, I’m not even discussing the actual contents) doesn’t deserve to be taken seriously. That leaves 4 candidates. Please note here that we only have 4 genuinely interested candidates for 7 seats.

Whoever has a technical idea is a good candidate to lead a project but not to the Council. Whoever wants to push a GLEP is a good candidate to become that GLEP’s champion (see workflow) and doesn’t need an election. Claiming “I want a pink pony!” (e.g. git) doesn’t count as an idea. That leaves 2 proposals on the table, and one of them is rather weak.

The good news is that the other one is a lot more than just not weak. Donnie, because that’s who we are talking about, shows he has a rather clear vision of how to go forward. I’m not going to say that it’s the only way forward and that I agree with everything in his manifesto, although I do agree with most of it, and I will let you form your own opinion. As with every council platform it is quite ambitious but it lists practical ideas that do set a direction. When there’s a pile of shit in front of you a long drawn-out plan is not necessary. What is needed is simply to roll-up your sleeves, quit talking and get working on a few simple but efficient actions to get the ball rolling. And Donnie is the man for that.

Sadly, he is the only one I can see.

An email to a friend

Here is an excerpt from an email I have sent to a friend some time ago. Who he is (obviously a Gentoo developer) and what the context was is of no importance. I thought some of you may find it inspirational.

[…] What I did that day wasn’t and still isn’t in any rulebook. It might even have been considered by some as interventionism, and I’m sure some complained about it in my back. But it had to be done and was done in a nice and jolly manner. We all cheered, drank and laughed.

My point is the following. Stop hiding behind rules and processes. Whatever it is, do it for the hell of it, because it has to be done and is what is good for Gentoo. Do it because it’s fun. Do it because you’ll end up helping people and pleasing them. But do it, don’t hide like you were hiding that day […]