{"id":1630,"date":"2021-08-16T15:07:45","date_gmt":"2021-08-16T13:07:45","guid":{"rendered":"http:\/\/blogs.gentoo.org\/mgorny\/?p=1630"},"modified":"2021-08-16T15:07:45","modified_gmt":"2021-08-16T13:07:45","slug":"the-stablereq-workflow-for-python-packages","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/mgorny\/2021\/08\/16\/the-stablereq-workflow-for-python-packages\/","title":{"rendered":"The stablereq workflow for Python packages"},"content":{"rendered":"

I have been taking care of periodic mass stabilization of Python packages in Gentoo for some time already. Per Guilherme Amadio<\/a>‘s suggestion, I’d like to share the workflow I use for this. I think it could be helpful to others dealing with large sets of heterogeneous packages.<\/p>\n

The workflow requires:<\/p>\n

– app-portage\/mgorny-dev-scripts<\/a>, v10
\n– dev-util\/pkgcheck<\/p>\n

<\/p>\n

Grabbing candidate list from pkgcheck<\/h2>\n

One of the features of pkgcheck<\/a> is that it can report ebuilds that haven’t been changed in 30 days and therefore are due for stabilization. This isn’t perfect but in the end, it gets the job done.<\/p>\n

I start by opening two terminals side-by-side and entering the clone of ::gentoo on both. On one of them, I run:<\/p>\n

stablereq-eshowkw 'dev-python\/*'<\/code><\/pre>\n
On the other, I do:<\/p>\n
stablereq-find-pkg-bugs 'dev-python\/*'\r\nstablereq-make-list 'dev-python\/*'<\/code><\/pre>\n

\n  Screenshot of desktop with the described three windows open
\n<\/a><\/p>\n
This gets me three things:<\/p>\n
1. An open Bugzilla search for all stabilization candidates.
\n2. A script to call file-stablereq for all stabilization candidates open in the editor.
\n3. eshowkw output for all stabilization candidates in the other terminal.<\/p>\n
The three scripts pass their arguments through to pkgcheck.  Instead of passing package specifications directly, you can use a simple pipeline to grab all packages with a specific maintainer:<\/p>\n
git grep -l python@gentoo.org '**\/metadata.xml' | cut -d\/ -f1-2 | xargs stablereq-eshowkw<\/code><\/pre>\n
Filtering the candidate list<\/h2>\n
The candidate list given by pkgcheck is pretty rough.  Now it’s time to mangle it a bit.<\/p>\n
For a start, I go through the eshowkw list to see if the packages have any newer versions that can be stabilized.  Roughly speaking, I ignore all packages that have only one stabilization candidate and I check the rest.<\/p>\n
Checking usually means looking at git log<\/kbd> and\/or pkgdiff<\/kbd> to see if a newer version would not be a better stabilization candidate.  I update the list in the editor accordingly, either changing the desired version or removing some packages altogether (e.g. because they are release candidates or to go straight for a newer version later).<\/p>\n
I close the eshowkw results then and do the next round of filtering via Bugzilla search.  I look at the Bugzilla search for bugs affecting the stabilization candidates.  Once again, I update the list accordingly.  Most importantly, this means removing packages that have their stablereq filed already.  This is also a good opportunity to resolve obsolete bugs.<\/p>\n
I close the search result tabs but leave the browser open (e.g. with an empty tab) for the next step.<\/p>\n
Filing the stablereqs<\/h2>\n
Now I save the list into a file, and run it via shell.  This generally involves a lot of file-stablereq<\/kbd> calls that open lots of browser tabs with pre-filled stablereqs.  I suppose it would be much better to use Bugzilla API to file bugs directly but I’ve never gotten around to implement that.<\/p>\n
I use bug-assign-user-js<\/a> to assign the bugs, then submit them.  With some skill, you can do it pretty fast.  Point the mouse at the ‘A’ box for the package, click, shift-tab, enter, ctrl-tab, repeat.<\/p>\n
If everything went correctly, you get a lot of new bugs filed.  Now it’s a good time to look into your e-mail client and mark the mails for newly filed bugs read, before NATTkA starts processing them.<\/p>\n
Post-processing the bugs<\/h2>\n
The last step is to go through bug mail resulting from NATTkA operations.<\/p>\n
If sanity check fails, it is necessary to either add dependencies on other bugs already filed, add additional packages to the package list or file additional stablereqs.<\/p>\n
For more complex problems, app-portage\/nattka<\/a> 0.2.15 provides a nattka make-package-list -s CATEGORY\/PACKAGE-VERSION<\/kbd> subcommand that can prepare a package list with dependencies.  However, note that it unconditionally takes newest versions available, so you will need to verify the result and replace versions whenever necessary.<\/p>\n
Additionally, I generally look at ALLARCHES<\/kbd> keyword being added to bugs.  If a bug is missing it, I verify whether the package is suitable, and add <stabilize-allarches\/><\/kbd> to its metadata.xml<\/kbd>.<\/p>\n","protected":false},"excerpt":{"rendered":"
I have been taking care of periodic mass stabilization of Python packages in Gentoo for some time already. Per Guilherme Amadio‘s suggestion, I’d like to share the workflow I use for this. I think it could be helpful to others dealing with large sets of heterogeneous packages. The workflow requires: – app-portage\/mgorny-dev-scripts, v10 – dev-util\/pkgcheck<\/p>\n","protected":false},"author":137,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[3],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/1630"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/users\/137"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/comments?post=1630"}],"version-history":[{"count":9,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/1630\/revisions"}],"predecessor-version":[{"id":1640,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/1630\/revisions\/1640"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/media?parent=1630"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/categories?post=1630"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/tags?post=1630"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}