upstream is moving to python 3.12 #2926

Closed
opened 2024-02-24 17:10:57 +01:00 by RayOfLight · 19 comments

hass upstream discontinues python3.11 starting on april 2024

hass upstream discontinues python3.11 starting on april 2024
onkelbeh added this to the 2024.2.3 milestone 2024-02-25 09:59:49 +01:00
onkelbeh self-assigned this 2024-02-25 09:59:53 +01:00
Owner

Just updated PYTHON_COMPAT in [master 4a3023514] homeassistant-2024.2.3, this is not fully tested yet.

I'll update here as soon as my production system is updated.

Just updated PYTHON_COMPAT in [master 4a3023514] homeassistant-2024.2.3, this is not fully tested yet. I'll update here as soon as my production system is updated.
onkelbeh modified the milestone from 2024.2.3 to 2024.4.0 2024-02-25 11:41:55 +01:00
Owner

Did a successfull (clean) install on a testbox with both 3.11.8_p1 & 3.12.2_p1 installed.
This box also runs samba & netatalk, so PYTHON_SINGLE_TARGET currently cannot be changed.

PYTHON_TARGETS="python3_12 python3_11"
PYTHON_SINGLE_TARGET="python3_11"

Had to 'enhance' some packages from the main repo.

Specials, to be installed manually, prior the big run:

  • dev-python/dtlssocket (with cython<3)
  • dev-python/cryptography-41.0.7
  • dev-python/chacha20poly1305-reuseable-0.12.0

I have not yet tested pysignalclirestapi, which will probably not work anymore because it still depends on dev-python/future, I requested bumping it.

Did a successfull (clean) install on a testbox with both `3.11.8_p1` & `3.12.2_p1` installed. This box also runs samba & netatalk, so PYTHON_SINGLE_TARGET currently cannot be changed. ``` PYTHON_TARGETS="python3_12 python3_11" PYTHON_SINGLE_TARGET="python3_11" ``` Had to 'enhance' some packages from the main repo. Specials, to be installed manually, prior the big run: - dev-python/dtlssocket (with cython<3) - dev-python/cryptography-41.0.7 - dev-python/chacha20poly1305-reuseable-0.12.0 I have not yet tested pysignalclirestapi, which will probably not work anymore because it still depends on dev-python/future, I requested bumping it.
Owner

[master 1eefb5c0d] homeassistant-2024.2.4 should now compile with 3.12 (after some fixes to the buildscript)

[master 1eefb5c0d] homeassistant-2024.2.4 should now compile with 3.12 (after some fixes to the buildscript)
Owner

will add 2024.2.5 soon. 2024.3 perhaps tomorrow.

will add 2024.2.5 soon. 2024.3 perhaps tomorrow.
Owner

'speedtestdotnet' has currently no support for Python 3.12:

grafik

I'm sure there are more.

'speedtestdotnet' has currently no support for Python 3.12: ![grafik](/attachments/9312f6ae-9ae4-4cb7-8a6b-c53a26d4387d) I'm sure there are more.
146 KiB
Owner

Here's some more fun for this 'issue', now with a changed PYTHON_SINGLE_TARGET:

[ebuild   R    ] media-video/ffmpeg-6.0.1-r3:0/58.60.60::gentoo  USE="alsa amr amrenc bluray bs2b bzip2 cdio chromaprint chromium codec2 cpudetection dav1d encode fdk flite fontconfig fribidi gcrypt gme gmp gnutls gpl gsm hardcoded-tables iconv iec61883 ieee1394 jpeg2k kvazaar ladspa libaom libaribb24 libass libcaca libdrm libilbc librtmp libsoxr libtesseract libv4l libxml2 lv2 lzma modplug mp3 network openh264 opus oss pic postproc rav1e rubberband sdl snappy sndio speex srt ssh static-libs svg svt-av1 theora threads truetype twolame v4l vaapi vdpau verify-sig vidstab vorbis vpx webp x264 x265 xvid zeromq zimg zlib zvbi -X (-amf) (-appkit) (-cuda) -debug -doc -frei0r -jack* -jpegxl -lcms -libplacebo (-mipsdspr1) (-mipsdspr2) (-mipsfpu) (-mmal) -nvenc -openal -opencl -opengl -openssl -pulseaudio (-qsv) -samba -test (-vmaf) -vulkan" CPU_FLAGS_X86="mmx mmxext sse sse2 -3dnow -3dnowext -aes -avx -avx2 -fma3 -fma4 -sse3 -sse4_1 -sse4_2 -ssse3 -xop" FFTOOLS="aviocat cws2fws ffescape ffeval ffhash fourcc2pixfmt graph2dot ismindex pktdumper qt-faststart sidxindex trasher" 0 KiB
[nomerge       ] net-fs/netatalk-3.1.18:0/18.0::gentoo  USE="(acl) pam shadow ssl -cracklib -dbus -debug -kerberos -ldap -pgp -quota -tcpd -tracker -zeroconf"
[ebuild   R    ] sys-apps/usbutils-017::gentoo  USE="-python" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 165 KiB
[ebuild   R    ] app-misc/pax-utils-1.3.7::gentoo  USE="-caps -man -python (-seccomp) -test" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 112 KiB
[ebuild   R    ] app-text/asciidoc-10.2.0::gentoo  PYTHON_SINGLE_TARGET="python3_12* (-pypy3) -python3_10 -python3_11*" 0 KiB
[ebuild   R    ] net-analyzer/net-snmp-5.9.3-r3:0/40::gentoo  USE="bzip2 ipv6 pcre perl ssl zlib -X -doc -elf -kmem -lm-sensors -mfd-rewrites -minimal -mysql -netlink -pcap -pci -python -rpm (-selinux) -smux -tcpd -ucd-compat -valgrind" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 6495 KiB
[ebuild   R    ] media-libs/alsa-lib-1.2.10-r2::gentoo  USE="-alisp -debug -doc -python" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 1082 KiB
[ebuild   R    ] net-misc/rsync-3.2.7-r4::gentoo  USE="acl iconv ssl xattr -examples -lz4 -rrsync -stunnel -system-zlib -verify-sig -xxhash -zstd" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 1123 KiB

Here's some more fun for this 'issue', now with a changed PYTHON_SINGLE_TARGET: ``` [ebuild R ] media-video/ffmpeg-6.0.1-r3:0/58.60.60::gentoo USE="alsa amr amrenc bluray bs2b bzip2 cdio chromaprint chromium codec2 cpudetection dav1d encode fdk flite fontconfig fribidi gcrypt gme gmp gnutls gpl gsm hardcoded-tables iconv iec61883 ieee1394 jpeg2k kvazaar ladspa libaom libaribb24 libass libcaca libdrm libilbc librtmp libsoxr libtesseract libv4l libxml2 lv2 lzma modplug mp3 network openh264 opus oss pic postproc rav1e rubberband sdl snappy sndio speex srt ssh static-libs svg svt-av1 theora threads truetype twolame v4l vaapi vdpau verify-sig vidstab vorbis vpx webp x264 x265 xvid zeromq zimg zlib zvbi -X (-amf) (-appkit) (-cuda) -debug -doc -frei0r -jack* -jpegxl -lcms -libplacebo (-mipsdspr1) (-mipsdspr2) (-mipsfpu) (-mmal) -nvenc -openal -opencl -opengl -openssl -pulseaudio (-qsv) -samba -test (-vmaf) -vulkan" CPU_FLAGS_X86="mmx mmxext sse sse2 -3dnow -3dnowext -aes -avx -avx2 -fma3 -fma4 -sse3 -sse4_1 -sse4_2 -ssse3 -xop" FFTOOLS="aviocat cws2fws ffescape ffeval ffhash fourcc2pixfmt graph2dot ismindex pktdumper qt-faststart sidxindex trasher" 0 KiB [nomerge ] net-fs/netatalk-3.1.18:0/18.0::gentoo USE="(acl) pam shadow ssl -cracklib -dbus -debug -kerberos -ldap -pgp -quota -tcpd -tracker -zeroconf" [ebuild R ] sys-apps/usbutils-017::gentoo USE="-python" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 165 KiB [ebuild R ] app-misc/pax-utils-1.3.7::gentoo USE="-caps -man -python (-seccomp) -test" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 112 KiB [ebuild R ] app-text/asciidoc-10.2.0::gentoo PYTHON_SINGLE_TARGET="python3_12* (-pypy3) -python3_10 -python3_11*" 0 KiB [ebuild R ] net-analyzer/net-snmp-5.9.3-r3:0/40::gentoo USE="bzip2 ipv6 pcre perl ssl zlib -X -doc -elf -kmem -lm-sensors -mfd-rewrites -minimal -mysql -netlink -pcap -pci -python -rpm (-selinux) -smux -tcpd -ucd-compat -valgrind" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 6495 KiB [ebuild R ] media-libs/alsa-lib-1.2.10-r2::gentoo USE="-alisp -debug -doc -python" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 1082 KiB [ebuild R ] net-misc/rsync-3.2.7-r4::gentoo USE="acl iconv ssl xattr -examples -lz4 -rrsync -stunnel -system-zlib -verify-sig -xxhash -zstd" PYTHON_SINGLE_TARGET="-python3_10 -python3_11*" 1123 KiB ```
Owner

and I had to remove net-analyzer/nmap from my core package.

and I had to remove `net-analyzer/nmap` from my core package.
onkelbeh modified the milestone from 2024.4.0 to 2024.4.1 2024-04-08 19:36:10 +02:00

Updated recently to 2024.4.2 and removed Python 3.11 from the system. Everything is working as expected with Python 3.12 only a few gentoo repo packages needed to be tweaked.

e.g. avahi

USE="-python"
PYTHON_COMPAT=( python3_{10..12} ) -> PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11"

Updated recently to 2024.4.2 and removed Python 3.11 from the system. Everything is working as expected with Python 3.12 only a few gentoo repo packages needed to be tweaked. e.g. avahi USE="-python" PYTHON_COMPAT=( python3_{10..12} ) -> PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11"

I tried to update but you are not providing ~dev-utils/pandas-2.1.4 and it is no longer in the tree. This blocks me from rebuilding with 3.12 target enabled.

For HA I would really like to know what 3.12 features they are using to justify this change.

I tried to update but you are not providing ~dev-utils/pandas-2.1.4 and it is no longer in the tree. This blocks me from rebuilding with 3.12 target enabled. For HA I would really like to know what 3.12 features they are using to justify this change.
Owner

Yep, this was dropped in 3b1c143561, re-added in 9d92431cd

Yep, this was dropped in https://github.com/gentoo/gentoo/commit/3b1c143561bbc880652245e99ea4d555a6a92f00, re-added in 9d92431cd

One more: dev-python/python-socketio. It is no longer in Portage but it is in GURU, however GURU does not have any version <5.0. This cannot be satisfied:

        >=dev-python/python-socketio-4.6.0[${PYTHON_USEDEP}]
       <dev-python/python-socketio-5.0[${PYTHON_USEDEP}]

>python-socketio-5.0 would block your =python-engineio-3.14.2 ebuild.

One more: `dev-python/python-socketio`. It is no longer in Portage but it is in GURU, however GURU does not have any version <5.0. This cannot be satisfied: ``` >=dev-python/python-socketio-4.6.0[${PYTHON_USEDEP}] <dev-python/python-socketio-5.0[${PYTHON_USEDEP}] ``` `>python-socketio-5.0` would block your `=python-engineio-3.14.2` ebuild.
Owner

OK. I'll look into that.
Yesterday I was (for the first time) able to update a box with

PYTHON_TARGETS="python3_12"
PYTHON_SINGLE_TARGET="python3_12"

I needed some betas to add Python 3.12 support:

=app-admin/restart-services-1.3 ~amd64
=app-admin/lib_users-0.15-r1 ~amd64

Had not time yet to investigate deeper.

OK. I'll look into that. Yesterday I was (for the first time) able to update a box with ```bash PYTHON_TARGETS="python3_12" PYTHON_SINGLE_TARGET="python3_12" ``` I needed some betas to add Python 3.12 support: ``` =app-admin/restart-services-1.3 ~amd64 =app-admin/lib_users-0.15-r1 ~amd64 ``` Had not time yet to investigate deeper.
Owner

cc9894459 dev-python/python-socketio: re-add 4.6.0-r2 from archive

cc9894459 dev-python/python-socketio: re-add 4.6.0-r2 from archive

I've upgraded my whole ~amd64 box with app-misc/homeassistant-2024-4-4.

I forget about the python-3.12 requirement and homeassistant didn't start with logs telling it needs python 3.12.
So I switched the whole system to python-3.12 with:

PYTHON_TARGETS="python3_11 python3_12"
PYTHON_SINGLE_TARGET="python3_12"

Everything works perfectly now.

So it seems that the homeassistant ebuild should force python3_12 dependency by removing python3_11 in PYTHON_COMPAT starting from homeassistant-2024.4.0 that it is the first homeassistant release to drop python 3.11 support upstream.

I've upgraded my whole ~amd64 box with app-misc/homeassistant-2024-4-4. I forget about the python-3.12 requirement and homeassistant didn't start with logs telling it needs python 3.12. So I switched the whole system to python-3.12 with: ``` PYTHON_TARGETS="python3_11 python3_12" PYTHON_SINGLE_TARGET="python3_12" ``` Everything works perfectly now. So it seems that the homeassistant ebuild should force python3_12 dependency by removing python3_11 in PYTHON_COMPAT starting from homeassistant-2024.4.0 that it is the first homeassistant release to drop python 3.11 support upstream.
Owner

You can safely remove PYTHON_TARGETS="python3_11" and then

EIX_LIMIT=0 eix -I# --installed-with-use python_targets_python3_11 | xargs emerge -1tv

should remove half of the code (and build time).
Currently it is not possible to remove dev-lang/python:3.11 itself, because there are fixed deps from at least glib and fontconfig.

This is from my production box:
grafik

My make.conf:

FEATURES="binpkg-multi-instance buildpkg"
EMERGE_OPTS="--jobs=2 --load-average=1 --binpkg-respect-use=y"

PYTHON_TARGETS="python3_12"
PYTHON_SINGLE_TARGET="python3_12"

I had to let some unstables in, but I did not clean my package.accept_keywords for longer, so I do not post it here :-)

You can safely remove PYTHON_TARGETS="python3_11" and then ``` EIX_LIMIT=0 eix -I# --installed-with-use python_targets_python3_11 | xargs emerge -1tv ``` should remove half of the code (and build time). Currently it is not possible to remove `dev-lang/python:3.11` itself, because there are fixed deps from at least `glib` and `fontconfig`. This is from my production box: ![grafik](/attachments/da2fc423-8af1-4a3b-a304-f587f3106904) My make.conf: ``` FEATURES="binpkg-multi-instance buildpkg" EMERGE_OPTS="--jobs=2 --load-average=1 --binpkg-respect-use=y" PYTHON_TARGETS="python3_12" PYTHON_SINGLE_TARGET="python3_12" ``` I had to let some unstables in, but I did not clean my package.accept_keywords for longer, so I do not post it here :-)
255 KiB
Owner

I did not yet remove the 3.11 PYTHON_COMPAT from the main Ebuild yet to avoid issues during build time for late migrators.

I did not yet remove the 3.11 PYTHON_COMPAT from the main Ebuild yet to avoid issues during build time for late migrators.
Owner

PYTHON_COMPAT=( python3_12 ) is set since 2024.5,
most parts of README.md are updated, too.

PYTHON_COMPAT=( python3_12 ) is set since 2024.5, most parts of README.md are updated, too.

Anyway the last gentoo enews told:

"Python 3.12 to become the default on 2024-06-01

Anyway the last gentoo enews told: > "Python 3.12 to become the default on 2024-06-01
Owner

Yep, I already removed Python 3.12 on several boxes.
grafik
last package missing was media-libs/fontconfig-2.15, even nmap works again.

Yep, I already removed Python 3.12 on several boxes. ![grafik](/attachments/7b68a6fe-c997-4da9-bddc-d5debd0bb073) last package missing was `media-libs/fontconfig-2.15`, even nmap works again.
256 KiB
onkelbeh added the
Python 3.12
label 2024-05-26 11:43:11 +02:00
Sign in to join this conversation.
No Label
Python 3.12
5 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: onkelbeh/HomeAssistantRepository#2926
No description provided.