{"id":443,"date":"2016-05-16T08:34:36","date_gmt":"2016-05-16T06:34:36","guid":{"rendered":"https:\/\/blogs.gentoo.org\/mgorny\/?p=443"},"modified":"2016-05-16T08:41:12","modified_gmt":"2016-05-16T06:41:12","slug":"how-linguas-are-thrice-wrong","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/mgorny\/2016\/05\/16\/how-linguas-are-thrice-wrong\/","title":{"rendered":"How LINGUAS are thrice wrong!"},"content":{"rendered":"

The LINGUAS<\/em> environment variable serves two purposes in\u00a0Gentoo. On one hand, it’s the\u00a0USE_EXPAND flag group for USE flags controlling installation of localizations. On the\u00a0other, it’s a\u00a0gettext-specfic environment variable controlling installation of localizations in\u00a0some of build systems supporting gettext. Fun fact is, both uses are simply wrong<\/em>.<\/p>\n

<\/p>\n

Why LINGUAS as an\u00a0environment variable is wrong?<\/h2>\n

Let’s start with the\u00a0upstream-blessed LINGUAS environment variable. If set, it limits localization files installed by autotools+gettext-based build systems (and\u00a0some more) to the\u00a0subset matching specified locales. At\u00a0first, it may sound like a\u00a0useful feature. However, it is an\u00a0implicit<\/em> feature, and\u00a0therefore one causing a\u00a0lot of\u00a0confusion for the\u00a0package manager.<\/p>\n

Long story short, in\u00a0this context the\u00a0package manager does not know anything about LINGUAS. It’s just a\u00a0random environment variable, that has some value and\u00a0possibly may be saved somewhere in\u00a0package metadata. However, this value can actually affect the\u00a0installed files in a\u00a0hardly predictable way. So, even if package managers actually added some special meaning to LINGUAS (which would be non-PMS compliant), it still would not be good enough.<\/p>\n

What does this practically mean? It means that if I set LINGUAS to some value on my system, then most of the\u00a0binary packages produced by it suddenly have files stripped, as compared to non-LINGUAS builds. If I installed the\u00a0binary package on some other system, it would match the\u00a0LINGUAS of build host rather than the\u00a0install host. And\u00a0this means the\u00a0binary packages are simply incorrect.<\/p>\n

Even worse, any change to LINGUAS can not be propagated correctly. Even if the\u00a0package manager decided to rebuild packages based on\u00a0changes in\u00a0LINGUAS, it has no way of knowing which locales were supported by a\u00a0package, and\u00a0if LINGUAS was used at\u00a0all. So you end up rebuilding all installed packages, just in\u00a0case.<\/p>\n

Why LINGUAS USE flags are wrong?<\/h2>\n

So, how do we solve all those problems? Of course, we introduce explicit LINGUAS flags. This way, the\u00a0developer is expected to list all supported locales in\u00a0IUSE, the\u00a0package manager can determine the\u00a0enabled localizations and\u00a0match binary packages correctly. All seems fine. Except, there are two problems.<\/p>\n

The\u00a0first problem is that it is cumbersome. Figuring out supported localizations and\u00a0adding a\u00a0dozen flags on a\u00a0number of\u00a0packages is time-consuming. What’s even worse, those flags need to be\u00a0maintained once added. Which means you have to check supported localizations for changes on\u00a0every version bump. Not all developers do that.<\/p>\n

The\u00a0second problem is that it is\u2026 a\u00a0QA violation, most of the\u00a0time. We already have quite a\u00a0clear policy that USE flags are not supposed to control installation of\u00a0small files with no explicit dependencies \u2014 and\u00a0most of\u00a0the\u00a0localization files are exactly that!<\/p>\n

Let me remind you why we have that policy. There are two reasons: rebuilds and\u00a0binary packages.<\/p>\n

Rebuilds are bad because every time you change LINGUAS, you end up rebuilding relevant packages, and\u00a0those can be huge. You may think it uncommon \u2014 but just imagine you’ve finally finished building your new shiny Gentoo install, and\u00a0noticed that you forgot to enable the\u00a0localization. And\u00a0guess what! You have to build a\u00a0number of\u00a0packages, again.<\/p>\n

Binary packages are even worse since they are tied to a\u00a0specific USE flag combination. If you build a\u00a0binary package with specific LINGUAS, it can only be installed on hosts with exactly the\u00a0same LINGUAS. While it would be trivial to strip localizations from installed binary package, you have to build a\u00a0fresh one. And\u00a0with dozen lingua-flags\u2026 you end up having thousands of\u00a0possible binary package variants, if not more.<\/p>\n

Why EAPI\u00a05 makes things worse\u2026 or\u00a0better?<\/h2>\n

Reusing the\u00a0LINGUAS name for the\u00a0USE_EXPAND group looked like a\u00a0good idea. After all, the\u00a0value would end up in\u00a0ebuild environment for use by the\u00a0build system, and\u00a0in\u00a0most of the\u00a0affected packages, LINGUAS worked out of the\u00a0box with no ebuild changes! Except that\u2026 it wasn’t really guaranteed to before EAPI\u00a05.<\/p>\n

In\u00a0earlier EAPIs, LINGUAS could contain pretty much anything, since no special behavior was reserved for it. However, starting with EAPI\u00a05 the\u00a0package manager guarantees that it will only contain those values that correspond to enabled flags. This is a\u00a0good thing, after all, since it finally makes LINGUAS work reliably. It has one side effect though.<\/p>\n

Since LINGUAS is reduced to enabled USE flags, and\u00a0enabled USE flags can only contain defined USE flags\u2026 it means that in\u00a0any ebuild missing LINGUAS flags, LINGUAS should be effectively empty (yes, I know Portage does not do that currently, and\u00a0it is a\u00a0bug in\u00a0Portage). To make things worse, this means set to an\u00a0empty value<\/em> rather than unset. In\u00a0other words, disabling localization completely.<\/p>\n

This way, a\u00a0small implicit QA issue of\u00a0implicitly affecting installed localization files turned out into a\u00a0bigger issue of\u00a0suddenly stopping to install localizations. Which in\u00a0turn can’t be fixed without introducing proper set of\u00a0LINGUAS everywhere, causing other kind of\u00a0QA issues and\u00a0additional maintenance burden.<\/p>\n

What would be the\u00a0good solution, again?<\/h2>\n

First of\u00a0all, kill LINGUAS. Either make it unset for good (and\u00a0this won’t be easy since PMS kinda implies making all PM-defined variables read-only), or disable any special behavior associated with it. Make the\u00a0build system compile and\u00a0install all localizations.<\/p>\n

Then, use INSTALL_MASK. It’s designed to handle this. It strips files from installed systems while preserving them in\u00a0binary packages. Which means your binary packages are now more portable, and\u00a0every system you install them to will get correct localizations stripped. Isn’t that way better than rebuilding things?<\/p>\n

Now, is that going to happen? I doubt it. People are rather going to focus on claiming that buggy Portage behavior was good, that QA issues are fine as long as I can strip some small files from my system in the\u00a0\u2018obvious\u2019 way, that the\u00a0specification should be changed to allow a\u00a0corner case\u2026<\/p>\n","protected":false},"excerpt":{"rendered":"

The LINGUAS environment variable serves two purposes in\u00a0Gentoo. On one hand, it’s the\u00a0USE_EXPAND flag group for USE flags controlling installation of localizations. On the\u00a0other, it’s a\u00a0gettext-specfic environment variable controlling installation of localizations in\u00a0some of build systems supporting gettext. Fun fact is, both uses are simply wrong.<\/p>\n","protected":false},"author":137,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[3],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/443"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/users\/137"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/comments?post=443"}],"version-history":[{"count":3,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/443\/revisions"}],"predecessor-version":[{"id":446,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/443\/revisions\/446"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/media?parent=443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/categories?post=443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/tags?post=443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}