I got this idea about writing a Portage metadata cache backend based on extended file attributes. We are talking about file metadata after all and the key=value format fits the cache quite fine. I have it working now. On the road I hit a couple of interesting issues. The cache can have arbitrary long lines but all file systems I tested have a limit on how long the values can be. I decided to just split the values into multiple attributes when they are too long. I also found out that ext4 and btrfs use the wrong errno to signal the value being too long. man xattr_set says it should be E2BIG but both of those file systems return ENOSPC. I opened an upstream kernel bug about this to see what they think:
\nhttp:\/\/bugzilla.kernel.org\/show_bug.cgi?id=12793<\/p>\n
This is what it looks like currently:<\/p>\n
\r\nbetelgeuse@pena \/mnt\/test\/dev-java\/java-config $ getfattr -d java-config-2.1.7.ebuild | head\r\n# file: java-config-2.1.7.ebuild\r\nuser.CDEPEND=\"1:\"\r\nuser.DEFINED_PHASES=\"1:compile install postinst postrm unpack\"\r\nuser.DEPEND=\"1:dev-lang\/python >=sys-apps\/sed-4 virtual\/python\"\r\nuser.DESCRIPTION=\"1:Java environment configuration tool\"\r\nuser.EAPI=\"1:0\"\r\nuser.HOMEPAGE=\"1:http:\/\/www.gentoo.org\/proj\/en\/java\/\"\r\nuser.INHERITED=\"1:\"\r\nuser.IUSE=\"1:\"\r\nuser.KEYWORDS=\"1:~alpha ~amd64 ~arm ~ia64 ~ppc ~ppc64 ~x86 ~x86-fbsd\"\r\n<\/pre>\nAs for performance the current implementation seems to perform about the same for emerge -uDpv world as the default cache.
\nThese results are with a warm file system cache.<\/p>\nResults on btrfs\/xattrs:
\nreal 0m13.194s
\nuser 0m10.906s
\nsys 0m1.811s
\nbtrfs\/default:
\nreal 0m12.101s
\nuser 0m10.847s
\nsys 0m0.980s<\/p>\nxfs does a little better because it has a longer limit for attribute values. I guess that most of the time is spend in doing something else than cache lookups but will try to profile later. The code isn’t committed anywhere outside my portage trunk git svn checkout yet but will try to see if this is something zmedico accepts to Portage trunk. Probably not going to be a documented option any time soon though.<\/p>\n","protected":false},"excerpt":{"rendered":"
I got this idea about writing a Portage metadata cache backend based on extended file attributes. We are talking about file metadata after all and the key=value format fits the cache quite fine. I have it working now. On the road I hit a couple of interesting issues. The cache can have arbitrary long lines … Continue reading Metadata Cache Backend Based on Extended Attributes<\/span><\/a><\/p>\n","protected":false},"author":57,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/posts\/32"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/users\/57"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/comments?post=32"}],"version-history":[{"count":1,"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/posts\/32\/revisions"}],"predecessor-version":[{"id":61,"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/posts\/32\/revisions\/61"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/media?parent=32"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/categories?post=32"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/betelgeuse\/wp-json\/wp\/v2\/tags?post=32"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}