pdb prompt is nonresponsive in Python 3.13.0 from python-build-standalone

[bluthej]

With Python 3.13.0 installed with uv (i.e. from python-build-standalone), the pdb prompt is completely nonresponsive when triggered either by the --pdb flag or by a breakpoint.

This can be reproduced by the following test in test_foo.py:

def test_bar():
    raise ValueError

and running:

pytest test_foo.py --pdb

I tested the same code with the official source release from python.org for 3.13.0 on the same computer and the same code works just fine, so it seems it's somehow related to the build by python-build-standalone. Note that putting a breakpoint in a regular Python script and running it in the same interpreter does work, so this seems to be an issue with the combination with pytest.

OS: Arch Linux.

Pytest version

❯ pytest --version
pytest 8.3.3

`pip list` output

❯ pip list
Package   Version
--------- -------
iniconfig 2.0.0
packaging 24.1
pip       24.2
pluggy    1.5.0
pytest    8.3.3

[The-Compiler]

I've never used uv before, but this was a good opportunity to give it a try:

• uv python install
• uv venv
• uv pip install -e .

All that took about a second, which is indeed pretty damn crazy 🤯

And indeed I can reproduce with that. A breakpoint() in a normal Python file (outside of pytest) seems to work fine. And disabling pytest's output capturing with -s seems to fix things too!

[webknjaz]

Can this be related to the readline quirk documented @ https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#backspace-key-doesn-t-work-in-python-repl ?

[The-Compiler]

Hmm, I don't think so. I don't see such an error message, and for what it's worth, setting TERMINFO_DIRS=/etc/terminfo:/lib/terminfo:/usr/share/terminfo doesn't change anything either.

[The-Compiler]

Oh, this one seems like a likely culprit, however:

Use of libedit on Linux

Python 3.10+ Linux distributions link against libedit (as opposed to readline) by default, as libedit is supported on 3.10+ outside of macOS.

Most Python builds on Linux will link against readline because readline is the dominant library on Linux.

Some functionality may behave subtly differently as a result of our choice to link libedit by default. (We choose libedit by default to avoid GPL licensing requirements of readline.)

Perhaps something similar to #1281 could be going on with libedit (or their usage of it) too?

[The-Compiler]

Yep, it's that indeed. Built a vanilla CPython 3.13.0 with ./configure --with-readline=editline and it behaves in the exact same way. It would also explain why @jaraco saw this on macOS over in #12497 (comment) because from what I understand, CPython uses it on macOS by default.

Tried CPython v3.12.7 built the same way, that works. Will now bisect between the two...

[The-Compiler]

Well, this was a bit annoying as two build fixes for readline were in between:

cpython$ git log --oneline v3.12.7..v3.13.0 -- Modules/readline.c
417dd3aca7b gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_SetGIL (GH-118645)
c2627d6eea9 gh-116322: Add Py_mod_gil module slot (#116882)
!! 8515fd79fef gh-117845: Detect libedit hook function signature in configure (#117870)
e7e1116a781 gh-105323: Remove `WITH_APPLE_EDITLINE` to use the same declaration for all editline (gh-112513)
c2982380f82 gh-112510: Add `readline.backend` for the backend readline uses (GH-112511)
2df26d83486 gh-112105: Make completer delims work on libedit (gh-112106)
154f099e611 gh-112292 : Catch import error conditions with readline hooks (gh-112313)
!! f4cb0d27cc0 gh-109191: Fix build with newer editline (gh-110239)
501939c9c14 gh-105323: Update readline module to detect apple editline variant (gh-108665)
e7de0c5901b gh-108765: Python.h no longer includes <sys/time.h> (#108775)
b32d4cad15f gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
d228825e088 gh-106320: Remove _PyOS_ReadlineTState API (#107034)
c9ce983ae1a gh-106320: Remove private pylifecycle.h functions (#106400)
a9305b5e80e gh-105240: add missing function prototypes (#105241)
9ab587b7146 gh-89886: Rely on HAVE_SYS_TIME_H (#105058)

but with a bit of git cherry-pick to get them to whatever commit I was testing, I was able to figure out the culprit:

commit 01e7405da400e8997f8964d06cc414045e144681 (HEAD)
Author: Tian Gao <[email protected]>
Date:   Mon Mar 25 08:18:09 2024 -0700

    gh-112948: Make pdb completion similar to repl completion (#112950)

 Lib/pdb.py                                                              | 43 ++++++++++++++++++++++++++++---------------
 Lib/test/test_pdb.py                                                    | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 Misc/NEWS.d/next/Library/2023-12-11-00-51-51.gh-issue-112948.k-OKp5.rst |  1 +
 3 files changed, 80 insertions(+), 15 deletions(-)
 create mode 100644 Misc/NEWS.d/next/Library/2023-12-11-00-51-51.gh-issue-112948.k-OKp5.rst

that is:

• Make pdb completion similar to repl completion python/cpython#112948
• gh-112948: Make pdb completion similar to repl completion python/cpython#112950

which notably seems to use rlcompleter now in pdb.

Not sure where to go from here. If we want to open a CPython bug about this, we probably need a more minimal reproducer? But I have no clue whether the culprit is CPython or pytest here.

[bluthej]

@The-Compiler wow that's some impressive investigative skills right there! I certainly wouldn't have been able to figure this all out so fast...

All that took about a second, which is indeed pretty damn crazy 🤯

Yeah it's pretty sweet ^^