Why proactively clean Python 2 up?

It seems a recurring complaint that we’re too aggressive on cleaning Python 2 up from packages. Why remove it if (package’s) upstream still supports py2? Why remove it when it still works? Why remove it when somebody’s ready to put some work to keep it working?

I’m pretty sure that you’re aware that Python 2 has finally reached its end-of-life. It’s past its last release, and the current version is most likely vulnerable. We know we can’t remove it entirely just yet (but the clock is ticking!), so why remove its support here and there instead of keeping it some more?

This is best explained on the example of dev-python/twisted — but dev-python/pillow is also quite similar. Twisted upstream removed support for Python 2 at version 20. This means that we ended up having to keep two versions of Twisted — 19 that still supports Python 2, and 20 that does not. What does that means for our users?

Firstly, they can’t normally upgrade Twisted if at least one of its reverse dependencies supports Python 2 and is installed. What’s important is that the user does not have to meaningfully need or use Python 2 in that reverse dependency. It is entirely sufficient that it supports Python 2 and the user is using default PYTHON_TARGETS.

Of course, you could argue that changing the default PYTHON_TARGETS would resolve the problem without having to proactively remove Python 2 from Twisted revdeps. Today, I’m not sure which of the two options is better. However, back when cleanup started changing default PT would involve a lot of pain for the users. We’d have to reenable 2.7 via package.use for many packages (but which ones?) or the users would have to reenable it themselves. But that’s really tangential now.

Secondly, when upstream stops supporting the old version, the maintenance cost rises quickly. Since we don’t allow mixing two versions easily (and I don’t really want to go down that path), a single version must provide all implementations that the union of its reverse dependencies requires. This meant that I had to put significant effort fixing Python 3.8 and 3.9 support in Twisted 19.

Thirdly, old versions tend to end up becoming vulnerable. This is now the case both with Twisted and Pillow! In both cases, we can’t clean up vulnerable versions yet because they still have unresolved Python 2 reverse dependencies. We have a pretty descriptive phrase for this kind of situation in Polish: «to wake up with your hand in the potty».

What’s my point here? Removing Python 2 proactively means removing it at our leisure. We start with packages that don’t need it (because they fully support Python 3), we unlock the removal in their dependencies, we clean these dependencies… and when one of the upstreams decides to remove it, we don’t have to do anything because we’ve already done that and resolved all the issues. And we don’t have to worry about having to quickly clean up the depgraph and remove vulnerable versions or perform non-trivial backports.

6 thoughts on “Why proactively clean Python 2 up?”

  1. My question is, when will the big browser packages port to python3 build-deps?

    I’ve had everything python related at python3 only for months now; python2 itself is the only python2 package I have on the system, but I can’t remove it because of the firefox/chromium/qtwebengine build-time deps.

    They sure like their four or six-week upgrade cycles but they can’t port their python2 build scripts to something actually still supported even given an entire decade? And we’re supposed to trust them for timely security updates?

    1. Did you ever see the big companies do anything for puny users such as us? I still consider it miraculous that they’ve improved error handling for files without read permission a bit.

  2. A big thanks to you and the whole gentoo team.

    It’s long overdue to remove Python 2 and the approach is the only viable route imho.

    It’s a common story that a software isn’t allowed to die – just because no one wants to migrate.

    Pull the plug in the distribution / packaging – and let dead software die.

    As soon as patching up the dead software has become the burden of those who didn’t want to migrate things suddenly get moving fast…

    We’re talking about a common time frame, too – 5 years and more waiting for upgrade is really common in most software companies.

    Let it run till it’s dead and cannot be reanimated seems to be the motto, sadly.

    While some might argue that supporting something as long as possible is viable – it’s not.

  3. The one compromise I might suggest would be to leave the -9999 live ebuilds for projects that are actively working on migrating to Python 3. Some of them work and just don’t have an official release yet.

  4. I am worried about spidermonkey. It does not have stable API. All releases have python2 build dep. With gjs and polkit hard depending on spidermonkey and takes months to move to latest release, this could cause big delay unless they backport the build system.

  5. Well, it is one thing to remove an extant Python 2.7 dependency, when there is also Python 3 support—I have already moved almost everything over to Python 3.8, while simultaneously blocking Python 3.9, even when it required switching to live(-9999) versions or deploying simple patches (such as for lirc)…
    … and quite another removing any Python integration because the last supported version is Python 2.7.
    Honestly, I would like to see more of the former and less of the latter.

Leave a Reply to Michał Górny Cancel reply

Your email address will not be published.