Retroactively fixing the world
PyPA maintains two standards regarding packaging artifact filenames:
I have decided to give a few popular PEP 517 backends a go and see whether they follow the standards.
Continue reading “Naming standards compliance of PEP517 backends”
For a long time, the dev-python category in Gentoo did not follow any specific naming policy. Usually we went for what made the ebuild easier — the GitHub project name, if we happened to be using GitHub archives as distfiles, or PyPI project name when using source distributions from PyPI. However, this was inconvenient for users who had a hard time finding specific packages. Historically, we even had cases of developers independently adding a second copy of the same package with different name.
This is why I eventually started researching the standards for Python package naming, and drafting a new policy. The package name policy can now be found in the Gentoo Python Guide. In this post, I’d like to summarize the research that led to forming it, and the problems that we are to face yet.
Continue reading “The inconsistencies around Python package naming and the new policy”
Here’s a bunch of handy commands that I’ve conceived to semi-automatically remove old versions of packages that do not have stable keywords (and therefore are not subject to post-stabilization cleanups that I do normally).
Continue reading “Handy commands to clean up old ~arch-only packages”
It is a pretty common knowledge that ordering of members within archive can affect the compression ratio. I’ve done some quick testing and the results somewhat surprised me. Firstly, it turned out that the simplest lexical sorting by name (path) gave the best result. Secondly, because it turned out that the difference between that and sorting by size was as large as 8%.
Note that this is a pretty specific source archive, so results may vary. Test details and commands in the remainder of the post.
| Sort order | Size in bytes | Compared to best |
|---|---|---|
| name | 108 011 756 | 100.00% |
| suffix | 108 573 612 | 100.52% |
| size (smallest first) | 116 797 440 | 108.13% |
| size (largest first) | 116 645 940 | 108.00% |
| suffix + size | 111 709 128 | 103.42% |
The conclusion? Sorting can affect compression ratio more than I have anticipated. However, all the “obvious” optimizations have made the result worse than plain lexical sorting. Perhaps it’s just the matter of well-organized source code keeping similar files in the same directories. Perhaps there is a way to optimize it even more (and beat sorting by name). One interesting option would be to group files by bucket sizes, and then sort by name.
Special thanks to Adrien Nader and Lasse Collin from #tukaani for inspiring me to do this.
The upcoming clang 16 release features substantial improvements to configuration file support. Notably, it adds support for specifying multiple files and better default locations. This enabled Gentoo to finally replace the default-* flags used on sys-devel/clang, effectively empowering our users with the ability to change defaults without rebuilding whole clang.
This change has also been partially backported to clang 15.0.2 in Gentoo, and (unless major problems are reported) will be part of the stable clang 15.x release (currently planned for upcoming 15.0.3).
In this post, I’d like to shortly describe the new configuration file features, how much of them have been backported to 15.x in Gentoo and how defaults are going to be selected from now on.
Continue reading “Clang in Gentoo now sets default runtimes via config file”