GH-38323: [CI][Python] Use system gdb on test-conda-python#38324
Conversation
gdb on conda-forge frequently conflicts with the newest Python version (example for Python 3.12: conda-forge/gdb-feedstock#66) We needn't install gdb from conda-forge, though, as the Python interpreter used for gdb is entirely independent from the Python interpreter under test.
|
@github-actions crossbow submit test-conda-python* |
This comment was marked as outdated.
This comment was marked as outdated.
|
Hmm, things are not so simple. All builds pass except when the Python version under test is the same as the one that the system gdb is linked against, in which case the tests fail. |
This comment was marked as outdated.
This comment was marked as outdated.
|
Revision: cc0ad1f Submitted crossbow builds: ursacomputing/crossbow @ actions-add7835c90
|
| # If the Python version being tested is the same as the Python used by the system gdb, | ||
| # we need to install the conda-forge gdb instead. | ||
| RUN mamba install -q -y \ | ||
| --file arrow/ci/conda_env_python.txt \ | ||
| gdb \ | ||
| $([ "$python" == "3.10" ] && echo "gdb") \ |
There was a problem hiding this comment.
Will this be broken when we change the base Ubuntu version?
arrow/ci/docker/conda.dockerfile
Line 19 in 63840f2
Can we detect the Python version that is used by system GDB?
gdb --batch --eval-command 'python import sys; print(sys.version_info.major, sys.version_info.minor, sep=".")'?
There was a problem hiding this comment.
Hmm, I'm not sure how to do that in a Dockerfile. Can you help me?
kou
left a comment
There was a problem hiding this comment.
All builds pass except when the Python version under test is the same as the one that the system gdb is linked against, in which case the tests fail.
Do you know why?
Could you update the description before we merge this?
| # If the Python version being tested is the same as the Python used by the system gdb, | ||
| # we need to install the conda-forge gdb instead. | ||
| RUN mamba install -q -y \ | ||
| --file arrow/ci/conda_env_python.txt \ | ||
| gdb \ | ||
| $([ "$python" == "3.10" ] && echo "gdb") \ |
There was a problem hiding this comment.
Will this be broken when we change the base Ubuntu version?
arrow/ci/docker/conda.dockerfile
Line 19 in 63840f2
Can we detect the Python version that is used by system GDB?
gdb --batch --eval-command 'python import sys; print(sys.version_info.major, sys.version_info.minor, sep=".")'?
I honestly don't understand why. But it only happens when testing Python 3.10 (which is also the Python version installed under Ubuntu)... |
| RUN mamba install -q -y \ | ||
| --file arrow/ci/conda_env_python.txt \ | ||
| gdb \ | ||
| $([ "$python" == "3.10" ] && echo "gdb") \ |
There was a problem hiding this comment.
Can we try this?
| $([ "$python" == "3.10" ] && echo "gdb") \ | |
| $([ "$python" = $(gdb --batch --eval-command 'python import sys; print(sys.version_info.major, sys.version_info.minor, sep=".")') ] && echo "gdb") \ |
|
@github-actions crossbow submit test-conda-python* |
|
Revision: 53a1887 Submitted crossbow builds: ursacomputing/crossbow @ actions-8f00837e89
|
|
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 883a439. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them. |
…che#38324) ### Rationale for this change gdb on conda-forge frequently conflicts with the newest Python version (example for Python 3.12: conda-forge/gdb-feedstock#66) ### What changes are included in this PR? Use the system-provided gdb instead of trying to install it from conda-forge. This works because the Python interpreter used for gdb is entirely independent from the Python interpreter under test, and gdb itself can debug any binary, regardless of binutils version. However, there is a complication: if the system Python version and the conda-forge Python version are the same, then for some reason `pyarrow/tests/test_gdb.py` would fail with the system gdb. In that case we still install the conda-forge gdb. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * Closes: apache#38323 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
…che#38324) ### Rationale for this change gdb on conda-forge frequently conflicts with the newest Python version (example for Python 3.12: conda-forge/gdb-feedstock#66) ### What changes are included in this PR? Use the system-provided gdb instead of trying to install it from conda-forge. This works because the Python interpreter used for gdb is entirely independent from the Python interpreter under test, and gdb itself can debug any binary, regardless of binutils version. However, there is a complication: if the system Python version and the conda-forge Python version are the same, then for some reason `pyarrow/tests/test_gdb.py` would fail with the system gdb. In that case we still install the conda-forge gdb. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * Closes: apache#38323 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
Rationale for this change
gdb on conda-forge frequently conflicts with the newest Python version (example for Python 3.12: conda-forge/gdb-feedstock#66)
What changes are included in this PR?
Use the system-provided gdb instead of trying to install it from conda-forge.
This works because the Python interpreter used for gdb is entirely independent from the Python interpreter under test, and gdb itself can debug any binary, regardless of binutils version.
However, there is a complication: if the system Python version and the conda-forge Python version are the same, then for some reason
pyarrow/tests/test_gdb.pywould fail with the system gdb. In that case we still install the conda-forge gdb.Are these changes tested?
Yes.
Are there any user-facing changes?
No.