During my work on\u00a0Gentoo, I have seen many types of dependency pitfalls that developers fell in. Sad to say, their number is increasing with new EAPI features \u2014 we are constantly increasing new ways into failures rather than working on simplifying things. I can’t say the learning curve is getting much steeper but it is considerably easier to make a\u00a0mistake.<\/p>\n
In\u00a0this article, I would like to point out a\u00a0few common misunderstandings and\u00a0pitfalls regarding slots, slot operators and\u00a0any-of (|| ()<\/kbd>) deps. All of\u00a0those constructs are used to express dependencies that can be usually be satisfied by\u00a0multiple packages or\u00a0package versions that can be installed in\u00a0parallel, and\u00a0missing this point is\u00a0often the\u00a0cause of\u00a0trouble.<\/p>\n <\/p>\n One of\u00a0the\u00a0most common mistakes is to assume that multiple package dependency specifications listed in\u00a0one package are going to be combined somehow. However, there is no such guarantee, and\u00a0when a\u00a0package becomes slotted this fact actually becomes significant.<\/p>\n Of course, some package managers take various precautions to prevent the\u00a0following issues. However, such precautions not only can not be relied upon but may also violate the\u00a0PMS.<\/p>\n For example, consider the\u00a0following dependency specification:<\/p>\n It is a\u00a0common way of\u00a0expressing version ranges (in\u00a0this case, versions 2*, 3* and\u00a04* are acceptable). However, if app-misc\/foo is slotted and\u00a0there are versions satisfying the\u00a0dependencies in\u00a0different slots, there is no guarantee<\/em> that the\u00a0dependency could not be satisfied by installing foo-1 (satisfies <foo-5) and\u00a0foo-6 (satisfies >=foo-2) in\u00a0two slots!<\/p>\n Similarly, consider:<\/p>\n This one is often used to apply multiple sets of\u00a0USE flags to a\u00a0single package. Once again, if the\u00a0package is slotted, there is no guarantee<\/em> that the\u00a0dependency specifications will not be satisfied by\u00a0installing two slots with different USE flag configurations.<\/p>\n However, those problems mostly apply to fully slotted packages such as\u00a0sys-libs\/db<\/kbd> where multiple slots are actually meaningfully usable by a\u00a0package. With the\u00a0more common use of\u00a0multiple slots to provide incompatible versions of the\u00a0package (e.g. binary compatibility slots), there is a\u00a0more important problem: that even a\u00a0single package dependency can match the\u00a0wrong slot.<\/p>\n For non-truly multi-slotted packages, the\u00a0solution to all those problems is simple: always specify the\u00a0correct slot. For truly multi-slotted packages, there is no easy solution.<\/p>\n For example, a\u00a0version range has to be expressed using an\u00a0any-of dep:<\/p>\n Multiple sets of\u00a0USE flags? Well, if you really insist, you can combine them for each matching slot separately\u2026<\/p>\n A\u00a0similar problem applies to the\u00a0use of the\u00a0EAPI 5 \u2018equals\u2019 slot operator. The\u00a0PMS<\/a> notes that:<\/p>\n [\u2026]<\/p>\n To implement the equals slot operator, the package manager will need to store the slot\/sub-slot pair of the best installed version of the matching package. [\u2026]<\/p>\n \tPMS, 8.2.6.3 Slot Dependencies<\/a><\/cite>\n<\/p><\/blockquote>\n The significant part is that the\u00a0slot and\u00a0subslot is recorded for the\u00a0best package version matched by\u00a0the\u00a0specification containing the\u00a0operator. So again, if\u00a0the\u00a0operator is used on multiple dependencies that can match multiple slots, multiple slots can actually be recorded.<\/p>\n Again, this becomes really significant in\u00a0truly slotted packages:<\/p>\n While one may expect the\u00a0code to record the\u00a0slot of\u00a0sys-libs\/db<\/kbd> used by the\u00a0package, this may actually record any newer version that is installed while the\u00a0package is being built. In\u00a0other words, this may implicitly bind to db-6* (and\u00a0pull it in too).<\/p>\n For this to work, you need to ensure that the\u00a0dependency with the\u00a0slot operator can not match any version newer than the\u00a0two requested:<\/p>\n In\u00a0this case, the\u00a0dependency with the\u00a0operator could still match earlier versions. However, the\u00a0other dependency enforces (as long as it’s in\u00a0DEPEND<\/kbd>) that at\u00a0least one of\u00a0the\u00a0two versions specified is installed at build-time, and\u00a0therefore is used by the\u00a0operator as\u00a0the\u00a0best version matching it.<\/p>\n The\u00a0above block can easily be extended by a\u00a0single set of\u00a0USE dependencies (being applied to all the\u00a0package dependencies including the\u00a0one with slot operator). For multiple conditional sets of\u00a0USE dependencies, finding a\u00a0correct solution becomes harder\u2026<\/p>\n Since I have already started using the\u00a0any-of dependencies in\u00a0the\u00a0examples, I should point out yet another problem. Many of\u00a0Gentoo developers do not understand how any-of dependencies work, and\u00a0make wrong assumptions about them.<\/p>\n In an\u00a0any-of group, at\u00a0least one immediate child element must be matched. A blocker is considered to be matched if its associated package dependency specification is not matched.<\/p>\n \tPMS, 8.2.3 Any-of Dependency Specifications<\/a><\/cite>\n<\/p><\/blockquote>\n So, PMS guarantees that if at\u00a0least one of\u00a0the\u00a0immediate child elements (package dependencies, nested blocks) of the\u00a0any-of block, the\u00a0dependency is considered satisfied. This is the\u00a0only guarantee PMS gives you. The\u00a0two common mistakes are to assume that the\u00a0order is significant and\u00a0that any kind of\u00a0binding between packages installed at build time and\u00a0at run time is provided.<\/p>\n Consider an\u00a0any-of dependency specification like the\u00a0following:<\/p>\n In\u00a0this case, it is guaranteed that at\u00a0least one of\u00a0the listed packages is installed at\u00a0the\u00a0point appropriate for the\u00a0dependency class. If\u00a0none of the\u00a0packages are installed already, it is customary to assume the\u00a0Package Manager will prefer the\u00a0first one \u2014 while this is not specified and\u00a0may depend on satisfiability of the\u00a0dependencies, it is a\u00a0reasonable assumption to make.<\/p>\n If multiple packages are installed, it is undefined which one is actually going to be used. In\u00a0fact, the\u00a0package may even provide the\u00a0user with explicit run time choice of the\u00a0dependency used, or use multiple of\u00a0them. Assuming that A will be preferred over B, and\u00a0B over C is simply wrong.<\/p>\n Furthermore, if one of\u00a0the\u00a0packages is uninstalled, while one of\u00a0the\u00a0remaining ones is either already installed or\u00a0being installed, the\u00a0dependency is still considered satisfied. It is wrong to assume that in\u00a0any case the\u00a0Package Manager will bind to the\u00a0package used at install time, or cause rebuilds when switching between the\u00a0packages.<\/p>\n Finally, I am reaching the\u00a0point of\u00a0lately recurring debates. Let me make it clear: our current policy states that under no circumstances may :=<\/kbd> appear anywhere inside any-of dependency blocks<\/em>.<\/p>\n Why? Because it is meaningless, it is contradictory. It is not even undefined behavior, it is a\u00a0case where requirements put for the\u00a0slot operator can not be satisfied. To explain this, let me recall the\u00a0points made in\u00a0the\u00a0preceding sections.<\/p>\n First of\u00a0all, the\u00a0implementation of the\u00a0\u2018equals\u2019 slot operator requires the\u00a0Package Manager to explicitly bind the\u00a0slot\/subslot of\u00a0the\u00a0dependency to the\u00a0installed version. This can only happen if the\u00a0dependency is installed \u2014 and\u00a0an\u00a0any-of block only guarantees that one of\u00a0them will actually be installed. Therefore, an\u00a0any-of block may trigger a\u00a0case when PMS-enforced requirements can not be satisfied.<\/p>\n Secondly, the\u00a0definition of an\u00a0any-of block allows replacing one of\u00a0the\u00a0installed packages with another at\u00a0run time, while the\u00a0slot operator disallows changing the\u00a0slot\/subslot of\u00a0one of\u00a0the\u00a0packages. The\u00a0two requested behaviors are contradictory and\u00a0do not make sense. Why bind to a\u00a0specific version of\u00a0one package, while any version of the\u00a0other package is allowed?<\/p>\n Thirdly, the\u00a0definition of an\u00a0any-of block does not specify any particular order\/preference of packages. If\u00a0the\u00a0listed packages do not block one another, you could end up having multiple of\u00a0them installed, and\u00a0bound to specific slots\/subslots. Therefore, the\u00a0Package Manager should allow you to replace A:1 with B:2 but not with B:1 nor with A:2. We’re reaching insanity now.<\/p>\n Now, all of the\u00a0above is purely theoretical. The\u00a0Package Manager can do pretty much anything given invalid input, and\u00a0that is why many developers wrongly assume that slot operators work inside any-of. The\u00a0truth is: they do not, the\u00a0developer just did not test all the\u00a0cases correctly. The\u00a0Portage behavior varies from allowing replacements with no rebuilds, to requiring both of mutually exclusive packages to be installed simultaneously.<\/p>\n","protected":false},"excerpt":{"rendered":" During my work on\u00a0Gentoo, I have seen many types of dependency pitfalls that developers fell in. Sad to say, their number is increasing with new EAPI features \u2014 we are constantly increasing new ways into failures rather than working on simplifying things. I can’t say the learning curve is getting much steeper but it is … Continue reading “Dependency pitfalls regarding slots, slot ops and\u00a0any-of deps”<\/span><\/a><\/p>\n","protected":false},"author":137,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[11],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/466"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/users\/137"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/comments?post=466"}],"version-history":[{"count":19,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/466\/revisions"}],"predecessor-version":[{"id":485,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/466\/revisions\/485"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/media?parent=466"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/categories?post=466"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/tags?post=466"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Separate package dependencies are not combined into a\u00a0single slot<\/h2>\n
>=app-misc\/foo-2\r\n<app-misc\/foo-5<\/code><\/pre>\napp-misc\/foo[foo]\r\nbar? ( app-misc\/foo[baz] )<\/code><\/pre>\n|| (\r\n =sys-libs\/db-5*\r\n =sys-libs\/db-4*\r\n)<\/code><\/pre>\n|| (\r\n ( sys-libs\/db:5.3 tools? ( sys-libs\/db:5.3[cxx] ) ) \r\n ( sys-libs\/db:5.1 tools? ( sys-libs\/db:5.1[cxx] ) ) \r\n\t\u2026\r\n)<\/code><\/pre>\nThe\u00a0\u2018equals\u2019 slot operator and\u00a0multiple slots<\/h2>\n
\n
\n
|| (\r\n =sys-libs\/db-5*\r\n =sys-libs\/db-4*\r\n)\r\nsys-libs\/db:=<\/code><\/pre>\n|| (\r\n =sys-libs\/db-5*\r\n =sys-libs\/db-4*\r\n)\r\n<sys-libs\/db-6:=<\/code><\/pre>\nThe\u00a0meaning of\u00a0any-of dependencies<\/h2>\n
\n
|| (\r\n A\r\n B\r\n C\r\n)<\/code><\/pre>\nThe\u00a0\u2018equals\u2019 slot operator in\u00a0any-of dependencies<\/h2>\n