Moving commits between independent git histories

PyPy is an alternative Python implementation. While it does replace a large part of the interpreter, a large part of the standard library is shared with CPython. As a result, PyPy is frequently affected by the same vulnerabilities as CPython, and we have to backport security fixes to it.

Backporting security fixes inside CPython is relatively easy. All main Python branches are in a single repository, so it’s just a matter of cherry-picking the commits. Normally, you can easily move patches between two related git repositories using git-style patches but this isn’t going to work for two repositories with unrelated histories.

Does this mean manually patching PyPy and rewriting commit messages by hand? Luckily, there’s a relatively simple git am trick that can help you avoid that.

Roughly, the idea is to:

1. Create a git-format-patch of the change to backport.

2. Attempt to apply the change via git am — it will fail and your repository will be left in middle of an am session.

3. Apply the change via patch.

4. git add the changes.

5. Finally, call git am --continue to finish, wrapping your changes in the original commit metadata.

For example, let’s try backporting CVE-2021-23336 (parameter cloaking) fix:

First, grab the relevant patch from the CPython repository:

$ git format-patch -1 d0d4d30882fe3ab9b1badbecf5d15d94326fd13e

Then, inside the local clone of a random PyPy git mirror:

$ git am -3 ~/git/cpython/0001-3.7-bpo-42967-only-use-as-a-query-string-separator-G.patch
Applying: bpo-42967: only use '&' as a query string separator (GH-24297) (GH-24531)
error: sha1 information is lacking or useless (Doc/library/cgi.rst).
error: could not build fake ancestor
Patch failed at 0001 bpo-42967: only use '&' as a query string separator (GH-24297) (GH-24531)
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Now enter the directory that stdlib resides in, and apply the patch manually, skipping any missing files:

$ patch -p2 < ~/git/cpython/0001-3.7-bpo-42967-only-use-as-a-query-string-separator-G.patch
can't find file to patch at input line 39
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
|From d0d4d30882fe3ab9b1badbecf5d15d94326fd13e Mon Sep 17 00:00:00 2001
|From: Senthil Kumaran 
|Date: Mon, 15 Feb 2021 10:34:14 -0800
|Subject: [PATCH] [3.7] bpo-42967: only use '&' as a query string separator
| (GH-24297)  (GH-24531)
|MIME-Version: 1.0
|Content-Type: text/plain; charset=UTF-8
|Content-Transfer-Encoding: 8bit
|bpo-42967: [security] Address a web cache-poisoning issue reported in
|urllib.parse will only us "&" as query string separator by default
|instead of both ";" and "&" as allowed in earlier versions. An optional
|argument seperator with default value "&" is added to specify the
|Co-authored-by: Éric Araujo 
|Co-authored-by: Ken Jin 
|Co-authored-by: Adam Goldschmidt 
|(cherry picked from commit fcbe0cb04d35189401c0c880ebfb4311e952d776)
| Doc/library/cgi.rst                           |  9 ++-
| Doc/library/urllib.parse.rst                  | 23 ++++++-
| Doc/whatsnew/3.6.rst                          | 13 ++++
| Doc/whatsnew/3.7.rst                          | 13 ++++
| Lib/                                    | 23 ++++---
| Lib/test/                          | 29 ++++++--
| Lib/test/                     | 68 +++++++++++++------
| Lib/urllib/                           | 19 ++++--
| .../2021-02-14-15-59-16.bpo-42967.YApqDS.rst  |  1 +
| 9 files changed, 152 insertions(+), 46 deletions(-)
| create mode 100644 Misc/NEWS.d/next/Security/2021-02-14-15-59-16.bpo-42967.YApqDS.rst
|diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst
|index 0b1aead9dd..f0ec7e8cc6 100644
|--- a/Doc/library/cgi.rst
|+++ b/Doc/library/cgi.rst
File to patch: 
Skip this patch? [y] 
Skipping patch.
3 out of 3 hunks ignored
patching file
patching file test/
patching file test/
patching file urllib/
patching file NEWS.d/next/Security/2021-02-14-15-59-16.bpo-42967.YApqDS.rst

Adjust the changes as appropriate:

$ rm -r NEWS.d/
$ git status
HEAD detached from release-pypy3.7-v7.3.3
You are in the middle of an am session.
  (fix conflicts and then run "git am --continue")
  (use "git am --skip" to skip this patch)
  (use "git am --abort" to restore the original branch)

Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git restore ..." to discard changes in working directory)
	modified:   test/
	modified:   test/
	modified:   urllib/

no changes added to commit (use "git add" and/or "git commit -a")
$ git add test/ test/ urllib/

And finally let git am commit the changes for you:

$ git am --continue
Applying: bpo-42967: only use '&' as a query string separator (GH-24297) (GH-24531)

Leave a Reply

Your email address will not be published. Required fields are marked *