New tools to help with package cleanups

Did you ever have had Croaker shout at you because you removed an old version that just happened to be still required by some other package? Did you have to run your cleanups past (slow-ish) CI just to avoid that? If you did, I have just released app-portage/mgorny-dev-scripts, version 6 that has a tool just for that!

check-revdep to check depgraph of reverse dependencies

If you have used mgorny-dev-tools before, then you may know already about the rdep tool that prints reverse dependency information collected from qa-reports.gentoo.org. Now I’ve put it into a trivial pipeline with pkgcheck, and made check-revdep. The idea is really trivial: it fetches list of reverse dependencies from the server, filters it through qatom (from app-portage/portage-utils) and passes to pkgcheck scan -c VisibilityCheck.

So you do something like:


$ cd dev-python/unidecode
$ git rm unidecode-0.04.21.ebuild 
rm 'dev-python/unidecode/unidecode-0.04.21.ebuild'
$ check-revdep 
== rdep of dev-python/unidecode ==
== ddep of dev-python/unidecode ==
== bdep of dev-python/unidecode ==
== pdep of dev-python/unidecode ==
cat: /tmp/pindex/dev-python/unidecode: No such file or directory
dev-python/awesome-slugify
  NonexistentDeps: version 1.6.5: RDEPEND: nonexistent package: <dev-python/unidecode-0.05
  NonsolvableDepsInDev: version 1.6.5: nonsolvable depset(rdepend) keyword(~amd64) dev profile (default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+) (33 total): solutions: [ <dev-python/unidecode-0.05 ]
  NonsolvableDepsInStable: version 1.6.5: nonsolvable depset(rdepend) keyword(~amd64) stable profile (default/linux/amd64/17.0) (38 total): solutions: [ <dev-python/unidecode-0.05 ]
[...]
$ git restore --staged --worktree .

…and you know you can’t clean it up.

Warning: the tooling uses data from qa-reports that is updated periodically. If the data is not up-to-date (read: someone just added a dependency on your package), check-revdep may miss something.

Enable cache to speed things up

The rdep also supports using a local cache to avoid fetching everything from the server (= 4 requests per package). To populate the cache using the current data from server, just run:

$ rdep-fetch-cache 
--2020-09-02 09:00:05--  https://qa-reports.gentoo.org/output/genrdeps/rdeps.tar.xz
Resolving qa-reports.gentoo.org (qa-reports.gentoo.org)... 140.211.166.190, 2001:470:ea4a:1:230:48ff:fef8:9fdc
Connecting to qa-reports.gentoo.org (qa-reports.gentoo.org)|140.211.166.190|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1197176 (1,1M) [application/x-xz]
Saving to: ‘STDOUT’

-                                    100%[=====================================================================>]   1,14M   782KB/s    in 1,5s    

2020-09-02 09:00:08 (782 KB/s) - written to stdout [1197176/1197176]

The script will fetch the data as a tarball from server, and unpack to /tmp/*index.

pkgcheck also has its own caching, so successive checks will run faster if packages don’t change.

Combining rdep with other tools

You can also pass rdep output through to other tools, such as eshowkw (app-portage/gentoolkit) or gpy-showimpls (app-portage/gpyutils). The recommended pipeline is:

$ rdep $(pkg) | grep -v '^\[B' | xargs qatom -C -F '%{CATEGORY}/%{PN}' | sort -u | xargs gpy-showimpls | less
== rdep of dev-python/unidecode ==
== ddep of dev-python/unidecode ==
== bdep of dev-python/unidecode ==
== pdep of dev-python/unidecode ==
cat: /tmp/pindex/dev-python/unidecode: No such file or directory
app-misc/khard:0
          0.13.0: S             3.7 3.8
          0.17.0: ~             3.7 3.8 3.9
app-text/pelican:0
           3.7.1: S   #     3.6
           4.0.0: ~   #     3.6
           4.0.1: ~   #     3.6
           4.1.2: ~   #     3.6
           4.2.0: S         3.6 3.7
            9999:           3.6 3.7
dev-python/awesome-slugify:0
           1.6.5: ~         3.6 3.7 3.8
dev-python/pretty-yaml:0
          20.4.0: S         3.6 3.7 3.8 3.9
dev-python/python-slugify:0
           1.2.6: S   #     3.6 3.7 3.8 3.9
           4.0.1: S         3.6 3.7 3.8 3.9
media-sound/beets:0
        1.4.9-r2: ~ s       3.6 3.7 3.8
            9999:   s       3.6 3.7 3.8
www-apps/nikola:0
          7.8.15: S         3.6
       7.8.15-r1: ~   #     3.6
           8.0.4: ~   #     3.6 3.7 3.8
           8.1.0: ~   #     3.6 3.7 3.8
           8.1.1: ~   #     3.6 3.7 3.8
        8.1.1-r1: ~         3.6 3.7 3.8

Getting redundant versions from pkgcheck

Another nice trick is to have pkgcheck scan for redundant versions, and output it in format convenient for machine use.

For example, I often use:

git grep -l mgorny@ '**/metadata.xml' | cut -d/ -f1-2 | uniq | xargs pkgcheck scan -c RedundantVersionCheck -R FormatReporter --format '( cd {category}/{package} && eshowkw -C )'| sort -u | bash - |& less

that gives eshowkw output for all packages maintained by me that have potentially redundant versions. Plus, in another terminal:

$ git grep -l mgorny@ '**/metadata.xml' | cut -d/ -f1-2 | uniq | xargs pkgcheck scan -c RedundantVersionCheck -R FormatReporter --format 'git rm {category}/{package}/${package}-{version}.ebuild' | less

that gives convenient commands to copy-paste-execute.

Leave a Reply

Your email address will not be published. Required fields are marked *