Why proactively clean Python 2 up?

It seems a recurring complaint that we’re too aggressive on cleaning Python 2 up from packages. Why remove it if (package’s) upstream still supports py2? Why remove it when it still works? Why remove it when somebody’s ready to put some work to keep it working?

I’m pretty sure that you’re aware that Python 2 has finally reached its end-of-life. It’s past its last release, and the current version is most likely vulnerable. We know we can’t remove it entirely just yet (but the clock is ticking!), so why remove its support here and there instead of keeping it some more?
Continue reading “Why proactively clean Python 2 up?”

Gentoo Python Guide

Gentoo provides one of the best frameworks for providing Python support in packages among operating systems. This includes support for running multiple versions of Python (while most other distributions avoid going beyond simultaneous support for Python 2 and one version of Python 3), alternative implementations of Python, reliable tests, deep QA checks. While we aim to keep things simple, this is not always possible.

At the same time, the available documentation is limited and not always up-to-date. Both the built-in eclass documentation and Python project wiki page provide bits of documentation but they are mostly in reference form and not very suitable for beginners nor people who do not actively follow the developments within the ecosystem. This results in suboptimal ebuilds, improper dependencies, missing tests.

Gentoo Python Guide aims to fill the gap by providing a good, complete, by-topic (rather than reference-style) documentation for the ecosystem in Gentoo and the relevant eclasses. Combined with examples, it should help you write good ebuilds and solve common problems as simply as possible.

Gentoo Python Guide sources are available on GitHub. Suggestions and improvements are welcome.

No more PYTHON_TARGETS in single-r1

Since its inception in 2012, python-single-r1 has been haunting users with two sets of USE flags: PYTHON_TARGETS and PYTHON_SINGLE_TARGET. While this initially seemed a necessary part of the grand design, today I know we could have done better. Today this chymera is disappearing for real, and python-single-r1 are going to use PYTHON_SINGLE_TARGET flags only.

I would like to take this opportunity to explain why the eclass has been designed this way in the first place, and what has been done to change that.
Continue reading “No more PYTHON_TARGETS in single-r1”

Handling PEP 517 (pyproject.toml) packages in Gentoo

So far, the majority of Python packages have either used distutils, or a build system built upon it. Most frequently, this was setuptools. All those solutions provided a setup.py script with a semi-standard interface, and we were able to handle them reliably within distutils-r1.eclass. PEP 517 changed that.

Instead of a setup script, packages now only need to supply a declarative project information in pyproject.toml file (fun fact: TOML parser is not even part of Python stdlib yet). The build system used is specified as a combination of a package requirement and a backend object to use. The backends are expected to provide a very narrow API: it’s limited to building wheel packages and source distribution tarballs.

The new build systems built around this concept are troublesome to Gentoo. They are more focused on being standalone package managers than build systems. They lack the APIs matching our needs. They have large dependency trees, including circular dependencies. Hence, we’ve decided to try an alternate route.
Continue reading “Handling PEP 517 (pyproject.toml) packages in Gentoo”

A distribution kernel for Gentoo

The traditional Gentoo way of getting a kernel is to install the sources, and then configure and build one yourself. For those who didn’t want to go through the tedious process of configuring it manually, an alternative route of using genkernel was provided. However, neither of those variants was able to really provide the equivalent of kernels provided by binary distributions.

I have manually configured the kernels for my private systems long time ago. Today, I wouldn’t really have bothered. In fact, I realized that for some time I’m really hesitant to even upgrade them because of the effort needed to update configuration. The worst part is, whenever a new kernel does not boot, I have to ask myself: is it a real bug, or is it my fault for configuring it wrong?

I’m not alone in this. Recently Михаил Коляда (zlogene) has talked to me about providing binary kernels for Gentoo. While I have not strictly implemented what he had in mind, he inspired me to start working on a distribution kernel. The goal was to create a kernel package that users can install to get a working kernel with minimal effort, and that would be upgraded automatically as part of regular @world upgrades.

Continue reading “A distribution kernel for Gentoo”