gh-116741: Upgrade libexpat to 2.6.2 #117296

sethmlarson · 2024-03-27T18:51:25Z

Issue: Please upgrade bundled Expat to 2.6.2 (e.g. for the fix to CVE-2024-28757) #116741

hartwork

@sethmlarson looks good, my verification script likes it. Thank you! 👍

# Copyright (c) 2022-2024 Sebastian Pipping <sebastian@pipping.org>
# Licensed under the Apache License version 2.0

FROM alpine
RUN apk add --update \
            diffutils \
            git \
            sed \
        && \
    git clone --depth 1 https://github.com/python/cpython cpython-main \
        && \
    ( cd cpython-main && git rev-parse HEAD ) \
        && \
    git clone --depth 1 --branch libexpat-2.6.2 https://github.com/sethmlarson/cpython cpython-pr \
        && \
    ( cd cpython-pr && git rev-parse HEAD ) \
        && \
    git config --global advice.detachedHead false \
        && \
    git clone --depth 1 --branch R_2_6_0 https://github.com/libexpat/libexpat libexpat_2_6_0 \
        && \
    git clone --depth 1 --branch R_2_6_2 https://github.com/libexpat/libexpat libexpat_2_6_2 \
        && \
    diff -r -u libexpat_2_6_0/expat/lib/ cpython-main/Modules/expat/ | tee 2-6-0.diff \
        && \
    diff -r -u libexpat_2_6_2/expat/lib/ cpython-pr/Modules/expat/ | tee 2-6-2.diff \
        && \
    sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-6-0.diff 2-6-2.diff \
        && \
    diff -u 2-6-0.diff 2-6-2.diff \
        && \
    echo 'Diff is good.'

hartwork · 2024-04-14T10:21:19Z

@sethmlarson ready to merge?

hartwork · 2024-04-22T20:57:30Z

@sethmlarson any news?

sethmlarson · 2024-04-22T21:01:42Z

@hartwork I can't merge PRs on my own, I'll find a core developer to merge.

gpshead

These changes appear to match libexpat/libexpat@R_2_6_0...R_2_6_2.

We should probably script updating this.

miss-islington-app · 2024-04-23T01:15:11Z

Thanks @sethmlarson for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9, 3.10, 3.11, 3.12.
🐍🍒⛏🤖

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

miss-islington-app · 2024-04-23T01:15:36Z

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.11

bedevere-app · 2024-04-23T01:15:38Z

GH-118166 is a backport of this pull request to the 3.12 branch.

miss-islington-app · 2024-04-23T01:15:51Z

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.10

miss-islington-app · 2024-04-23T01:16:05Z

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.9

miss-islington-app · 2024-04-23T01:16:19Z

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.8

gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app · 2024-04-23T17:01:12Z

GH-118185 is a backport of this pull request to the 3.11 branch.

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app · 2024-04-23T17:02:10Z

GH-118186 is a backport of this pull request to the 3.10 branch.

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app · 2024-04-23T17:02:47Z

GH-118187 is a backport of this pull request to the 3.9 branch.

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app · 2024-04-23T17:04:19Z

GH-118188 is a backport of this pull request to the 3.8 branch.

sethmlarson · 2024-04-23T17:08:28Z

Created the backports:

hartwork · 2024-04-23T17:09:39Z

@sethmlarson thanks a lot! 👍

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee)

bedevere-bot · 2024-04-23T19:58:47Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.11 has failed when building commit 8133285.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/962/builds/1513) and take a look at the build logs.
Check if the failure is related to this commit (8133285) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/962/builds/1513

Failed tests:

test.test_multiprocessing_spawn.test_processes
test.test_multiprocessing_fork.test_processes
test.test_multiprocessing_forkserver.test_processes

Failed subtests:

test_empty_authkey - test.test_multiprocessing_forkserver.test_processes.WithProcessesTestListener.test_empty_authkey
test_empty_authkey - test.test_multiprocessing_fork.test_processes.WithProcessesTestListener.test_empty_authkey
test_empty_authkey - test.test_multiprocessing_spawn.test_processes.WithProcessesTestListener.test_empty_authkey

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/_test_multiprocessing.py", line 3483, in test_empty_authkey
    with listener.accept() as d:
         ^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 482, in accept
    deliver_challenge(c, self._authkey)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 756, in deliver_challenge
    digest = hmac.new(authkey, message, 'md5').digest()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 184, in new
    return HMAC(key, msg, digestmod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 67, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: no reason supplied


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/_test_multiprocessing.py", line 3474, in run
    client = self.connection.Client(addr, authkey=authkey)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 525, in Client
    answer_challenge(c, authkey)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 772, in answer_challenge
    digest = hmac.new(authkey, message, 'md5').digest()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 184, in new
    return HMAC(key, msg, digestmod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 67, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: no reason supplied
ERROR

(cherry picked from commit c9829ee)

Upgrade libexpat to 2.6.2

88969be

sethmlarson added the type-security A security issue label Mar 27, 2024

bedevere-app bot added the awaiting review label Mar 27, 2024

bedevere-app bot mentioned this pull request Mar 27, 2024

Please upgrade bundled Expat to 2.6.2 (e.g. for the fix to CVE-2024-28757) #116741

Closed

hartwork approved these changes Mar 27, 2024

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Mar 27, 2024

gpshead approved these changes Apr 23, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Apr 23, 2024

gpshead added needs backport to 3.8 needs backport to 3.9 only security fixes needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Apr 23, 2024

gpshead merged commit c9829ee into python:main Apr 23, 2024
46 checks passed

bedevere-app bot removed the awaiting merge label Apr 23, 2024

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Apr 23, 2024

pythongh-116741: Upgrade libexpat to 2.6.2 (pythonGH-117296)

45e4357

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

miss-islington-app bot assigned gpshead Apr 23, 2024

bedevere-app bot removed the needs backport to 3.12 only security fixes label Apr 23, 2024

sethmlarson deleted the libexpat-2.6.2 branch April 23, 2024 16:59

bedevere-app bot removed the needs backport to 3.11 only security fixes label Apr 23, 2024

sethmlarson added a commit to sethmlarson/cpython that referenced this pull request Apr 23, 2024

[3.11] pythongh-116741: Upgrade libexpat to 2.6.2 (pythonGH-117296)

672820f

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

sethmlarson added a commit to sethmlarson/cpython that referenced this pull request Apr 23, 2024

[3.10] pythongh-116741: Upgrade libexpat to 2.6.2 (pythonGH-117296)

02e79dc

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app bot removed the needs backport to 3.10 only security fixes label Apr 23, 2024

sethmlarson added a commit to sethmlarson/cpython that referenced this pull request Apr 23, 2024

[3.9] pythongh-116741: Upgrade libexpat to 2.6.2 (pythonGH-117296)

2befdf2

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app bot removed the needs backport to 3.9 only security fixes label Apr 23, 2024

sethmlarson added a commit to sethmlarson/cpython that referenced this pull request Apr 23, 2024

[3.8] pythongh-116741: Upgrade libexpat to 2.6.2 (pythonGH-117296)

4ecde3b

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee) Co-authored-by: Seth Michael Larson <seth@python.org>

bedevere-app bot removed the needs backport to 3.8 label Apr 23, 2024

gpshead pushed a commit that referenced this pull request Apr 23, 2024

[3.11] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) (#118185)

8133285

Upgrade libexpat to 2.6.2 (cherry picked from commit c9829ee)

ambv pushed a commit that referenced this pull request May 7, 2024

[3.10] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) (GH-118186)

2ec7018

(cherry picked from commit c9829ee)

ambv pushed a commit that referenced this pull request May 7, 2024

[3.9] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) (GH-118187)

7db40cd

(cherry picked from commit c9829ee)

ambv pushed a commit that referenced this pull request May 7, 2024

[3.8] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) (GH-118188)

f791cda

(cherry picked from commit c9829ee)

hartwork mentioned this pull request Sep 4, 2024

Please upgrade bundled Expat to 2.6.3 (e.g. for the fixes to CVE-2024-45490, CVE-2024-45491 and CVE-2024-45492) #123678

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-116741: Upgrade libexpat to 2.6.2 #117296

gh-116741: Upgrade libexpat to 2.6.2 #117296

sethmlarson commented Mar 27, 2024 •

edited by bedevere-app bot

Loading

hartwork left a comment

hartwork commented Apr 14, 2024

hartwork commented Apr 22, 2024

sethmlarson commented Apr 22, 2024

gpshead left a comment

miss-islington-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

sethmlarson commented Apr 23, 2024

hartwork commented Apr 23, 2024

bedevere-bot commented Apr 23, 2024

gh-116741: Upgrade libexpat to 2.6.2 #117296

gh-116741: Upgrade libexpat to 2.6.2 #117296

Conversation

sethmlarson commented Mar 27, 2024 • edited by bedevere-app bot Loading

hartwork left a comment

Choose a reason for hiding this comment

hartwork commented Apr 14, 2024

hartwork commented Apr 22, 2024

sethmlarson commented Apr 22, 2024

gpshead left a comment

Choose a reason for hiding this comment

miss-islington-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

miss-islington-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

bedevere-app bot commented Apr 23, 2024

sethmlarson commented Apr 23, 2024

hartwork commented Apr 23, 2024

bedevere-bot commented Apr 23, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

sethmlarson commented Mar 27, 2024 •

edited by bedevere-app bot

Loading