gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales

[serhiy-storchaka]

Arbitrary bytes round-trip through surrogateescape only in UTF-8 and single-byte encodings, not in a stateful multibyte encoding such as EUC-JP. Skip the run_default and run_no_utf8_mode variants when their encoding is not lossless.

• Issue: test_invalid_utf8_arg fails in non-UTF-8 multibyte locales #154582

[miss-islington-app]

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker b1e530ebf3b68e884ab37882d690d95e5d82aef5 3.14

[bedevere-app]

GH-154601 is a backport of this pull request to the 3.15 branch.

[miss-islington-app]

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker b1e530ebf3b68e884ab37882d690d95e5d82aef5 3.13

[bedevere-app]

GH-154604 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-154605 is a backport of this pull request to the 3.13 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable 3.13 (tier-2) has failed when building commit 3871923.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1388/builds/1296) and take a look at the build logs.
4. Check if the failure is related to this commit (3871923) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1388/builds/1296

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to '3871923d65c30cb8cd8306820a6f235547c0bcb4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 3871923d65c30 [3.13] gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales (GH-154584) (GH-154605)
Switched to and reset branch '3.13'

In file included from ../Include/Python.h:14,
                 from ../Modules/_testexternalinspection.c:53:
./pyconfig.h:1985:9: warning: ‘_POSIX_C_SOURCE’ redefined
 1985 | #define _POSIX_C_SOURCE 200809L
      |         ^~~~~~~~~~~~~~~
In file included from /usr/include/bits/libc-header-start.h:33,
                 from /usr/include/stdint.h:26,
                 from /usr/lib/gcc/ppc64le-redhat-linux/16/include/stdint.h:11,
                 from /usr/include/elf.h:24,
                 from ../Modules/_testexternalinspection.c:4:
/usr/include/features.h:319:10: note: this is the location of the previous definition
  319 | # define _POSIX_C_SOURCE        202405L
      |          ^~~~~~~~~~~~~~~
./pyconfig.h:2006:9: warning: ‘_XOPEN_SOURCE’ redefined
 2006 | #define _XOPEN_SOURCE 700
      |         ^~~~~~~~~~~~~
/usr/include/features.h:234:10: note: this is the location of the previous definition
  234 | # define _XOPEN_SOURCE  800
      |          ^~~~~~~~~~~~~
/usr/bin/ld.bfd: final link failed: No space left on device
collect2: error: ld returned 1 exit status
make: *** [Makefile:1065: libpython3.13d.so] Error 1

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang 3.13 (tier-3) has failed when building commit 3871923.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1395/builds/1289) and take a look at the build logs.
4. Check if the failure is related to this commit (3871923) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1395/builds/1289

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to '3871923d65c30cb8cd8306820a6f235547c0bcb4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 3871923d65c3 [3.13] gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales (GH-154584) (GH-154605)
Switched to and reset branch '3.13'

In file included from ./Modules/_testexternalinspection.c:53:
In file included from ./Include/Python.h:14:
./pyconfig.h:1985:9: warning: '_POSIX_C_SOURCE' macro redefined [-Wmacro-redefined]
 1985 | #define _POSIX_C_SOURCE 200809L
      |         ^
/usr/include/features.h:319:10: note: previous definition is here
  319 | # define _POSIX_C_SOURCE        202405L
      |          ^
In file included from ./Modules/_testexternalinspection.c:53:
In file included from ./Include/Python.h:14:
./pyconfig.h:2006:9: warning: '_XOPEN_SOURCE' macro redefined [-Wmacro-redefined]
 2006 | #define _XOPEN_SOURCE 700
      |         ^
/usr/include/features.h:234:10: note: previous definition is here
  234 | # define _XOPEN_SOURCE  800
      |          ^
2 warnings generated.
ar: libpython3.13d.a: error reading Objects/dictobject.o: No space left on device
make: *** [Makefile:1061: libpython3.13d.a] Error 1

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang Installed 3.13 (tier-3) has failed when building commit 3871923.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1429/builds/1287) and take a look at the build logs.
4. Check if the failure is related to this commit (3871923) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1429/builds/1287

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to '3871923d65c30cb8cd8306820a6f235547c0bcb4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 3871923d65c3 [3.13] gh-154582: Fix test_invalid_utf8_arg in non-UTF-8 multibyte locales (GH-154584) (GH-154605)
Switched to and reset branch '3.13'

rror: unable to open output file 'Python/errors.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Python/errors.o] Error 1
make: *** Waiting for unfinished jobs....
rror: unable to open output file 'Python/future.o': 'No space left on device'
1 error generated.
rror: unable to open output file 'Python/frozenmain.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Python/future.o] Error 1
make: *** [Makefile:3038: Python/frozenmain.o] Error 1
rror: unable to open output file 'Objects/obmalloc.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Objects/obmalloc.o] Error 1
rror: unable to open output file 'Python/flowgraph.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Python/flowgraph.o] Error 1
rror: unable to open output file 'Python/compile.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Python/compile.o] Error 1
rror: unable to open output file 'Objects/typeobject.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Objects/typeobject.o] Error 1
rror: unable to open output file 'Python/Python-ast.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Python/Python-ast.o] Error 1
rror: unable to open output file 'Python/ceval.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Python/ceval.o] Error 1
rror: unable to open output file 'Objects/unicodeobject.o': 'No space left on device'
1 error generated.
make: *** [Makefile:3038: Objects/unicodeobject.o] Error 1

chmod: cannot access 'target/': No such file or directory

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:3128: clean-retain-profile] Error 1 (ignored)