Has this project a future? #1777

New Issue

jump · 2023-08-25T14:37:20+02:00

jump commented

2023-08-25 14:37:20 +02:00

In the past this overlay made me a lot of fun because I like full control over the software that I use. Thank you very much onkelbeh for your hard work on this fantastic project.

Nowadays it's a little different. I have a custom integration that needs a newer version of HA and the Gentoo Python repository is not standing still either. It's a kind of patching and masking game to keep HA 2023.04.06 alive.

I hope everything is OK with you and your family onkelbeh?

Can you please make a statement in the README.md how the future is going to be with this overlay?

Things like this would be of interest (suspended until, release cycles, problems in the pipe, donations, etc.)

In the past this overlay made me a lot of fun because I like full control over the software that I use. Thank you very much onkelbeh for your hard work on this fantastic project. Nowadays it's a little different. I have a custom integration that needs a newer version of HA and the Gentoo Python repository is not standing still either. It's a kind of patching and masking game to keep HA 2023.04.06 alive. I hope everything is OK with you and your family onkelbeh? Can you please make a statement in the README.md how the future is going to be with this overlay? Things like this would be of interest (suspended until, release cycles, problems in the pipe, donations, etc.)

jump referenced this issue

2023-08-25 14:52:33 +02:00

Can't emerge homeassistant-min due to missing dev-python/aiohttp version. #1776

inode64 commented

2023-09-04 12:58:45 +02:00

That's why I proposed simplifying the packages by trying to use the official gentoo python system #1775

jump closed this issue

2023-09-16 14:37:08 +02:00

RayOfLight commented

2023-09-17 17:20:20 +02:00

@jump Reasons for closure?

I keep coming back to this bug, two months without any Andreas commit, Is he alive?

Did you find a solution? Maybe a debian box with docker?

@jump Reasons for closure? I keep coming back to this bug, two months without any Andreas commit, Is he alive? Did you find a solution? Maybe a debian box with docker?

jump reopened this issue

2023-09-17 20:38:55 +02:00

jump commented

2023-09-17 20:45:01 +02:00

We need to be patient. I don't know what hapened to onkelbeh. But something went south. I'm pretty sure.

Tatsh commented

2023-09-19 11:17:17 +02:00

I am planning to put versions of these ebuilds in my overlay. There will likely be a lot less integrations because I only use a few. Like here I will accept requests.

It will be a bit of work but after the initial work I have an automatic updater (livecheck) and I always update everything weekly.

I am planning to put versions of these ebuilds in [my overlay](https://github.com/Tatsh/tatsh-overlay). There will likely be a lot less integrations because I only use a few. Like here I will accept requests. It will be a bit of work but after the initial work I have an automatic updater ([livecheck](https://github.com/Tatsh/livecheck)) and I always update everything weekly.

👍 1

inode64 commented

2023-09-19 16:49:52 +02:00

I will try to help as much as I can, given that I have little time now. I am currently maintaining my repository at https://github.com/inode64/inode64-overlay. I also preferred that it continue in this repository and it is also possible to simplify the work by using the ebuilds from the main gentoo repository, look at the issue #1775

Ghost commented

2023-09-24 15:58:56 +02:00

I have been monitoring this "bug" for quite a while as well. Today I decided to try creating my own ebuild to get homeassistant to the latest version 2023.9.2 and it seems to have worked fine.

I don't consider it all that important to have every single dependency for homeassistant install to the system-wide packages directory so I had already made small modifications to my install to make sure it installs everything it needs to a virtual environment. I do let the virtual environment use system-wide packages if they are found, so it shouldn't actually be causing many duplicates of python packages being installed.

I don't remember all the required steps but basically it should be just removing the --skip-pip flag when starting hass and then making sure there's a virtual environment installed to /etc/homeassistant/.

I then removed pretty much every single use flag and dependency except "<dev-python/pip-23.1" in the ebuild, renamed it "-2023.9.2.ebuild" and installed it through a local overlay.

After this hass started installing the required dependencies to its virtual environment and then started up fine except some small problem where it was unable to initialize "Cloud" which also made "mobile_app" & "default_config" fail. An issue on github said this was because typing-extensions should be at least 4.6.3, which I already had (4.7.1 from offical gentoo repo), but I just tried installing the latest version to the virtual environment, restarting homeassistant and then those errors disappeared. Not sure if that was what solved it or if homeassistant just needed another restart for some reason... Other than this I am now up and running on 2023.9.2 from 2023.5.4 with seemingly no issues.

Just some notes if someone else also wants to try this or if someone else wants to maintain this on an overlay.

I have been monitoring this "bug" for quite a while as well. Today I decided to try creating my own ebuild to get homeassistant to the latest version 2023.9.2 and it seems to have worked fine. I don't consider it all that important to have every single dependency for homeassistant install to the system-wide packages directory so I had already made small modifications to my install to make sure it installs everything it needs to a virtual environment. I do let the virtual environment use system-wide packages if they are found, so it shouldn't actually be causing many duplicates of python packages being installed. I don't remember all the required steps but basically it should be just removing the --skip-pip flag when starting hass and then making sure there's a virtual environment installed to /etc/homeassistant/. I then removed pretty much every single use flag and dependency except "<dev-python/pip-23.1" in the ebuild, renamed it "-2023.9.2.ebuild" and installed it through a local overlay. After this hass started installing the required dependencies to its virtual environment and then started up fine except some small problem where it was unable to initialize "Cloud" which also made "mobile_app" & "default_config" fail. An issue on github said this was because typing-extensions should be at least 4.6.3, which I already had (4.7.1 from offical gentoo repo), but I just tried installing the latest version to the virtual environment, restarting homeassistant and then those errors disappeared. Not sure if that was what solved it or if homeassistant just needed another restart for some reason... Other than this I am now up and running on 2023.9.2 from 2023.5.4 with seemingly no issues. Just some notes if someone else also wants to try this or if someone else wants to maintain this on an overlay.

fariouche commented

2023-09-24 16:46:24 +02:00

Hello @AnonymousRetard , thanks for sharing
I'm also allowing HA to install extentions via pip...
From what I remember, HA need a minimum set of python packages to start (not only pip).
Do you have an ebuild to share with us?

Maybe we can have something hybrid, like have HA + gentoo's available python (I don't know how much packages are already in gentoo, maybe it's not worth the trouble if the number of gentoo packages is very low)

Hello @AnonymousRetard , thanks for sharing I'm also allowing HA to install extentions via pip... From what I remember, HA need a minimum set of python packages to start (not only pip). Do you have an ebuild to share with us? Maybe we can have something hybrid, like have HA + gentoo's available python (I don't know how much packages are already in gentoo, maybe it's not worth the trouble if the number of gentoo packages is very low)

👍 1

Ghost commented

2023-09-24 18:33:06 +02:00

Sure, here's a link: https://pastebin.com/hAhSrUcP

It probably has a bunch of problems for machines other than mine. I don't really know how to make ebuilds, but small hacks and generating new digest files I can handle.

The things I remember changing are:

Removing almost all dependencies but I left everything regarding systemd and zwave-js since I use those two and I know the zwave-js service is running outside of homeassistant (this package which is from the tatsh overlay is probably not pulled in by the part I left in there though.. not entirely sure).
Removing most things regarding testing since I don't really care about that. The "test" use-flag still exists though which I'm not sure why. I don't have it selected.
Found something about python 3.9 which I removed since it seemed inconsistent and I'm on 3.11 now (and so is official homeassistant).
Removed sqlite dependency since my installation is migrated to postgresql.

Not sure if I forgot some other small change but that's what I remember. There are still a few dependencies left that didn't require any useflags and weren't from the dev-python category. If you use the default file database you probably want to put the sqlite dependency back.

The tar.gz that gets downloaded seems to just be based on the name of the ebuild file. In my case I modified the homeassistant-min package since that was what I was using before but a cleaner version of this would probably get rid of the -min/-full packages and just do one which relies on a virtual environment. Preferably the ebuild should set that up properly as well, this is very much just a quick hack.

Also worth nothing is that I have not yet tried doing a --depclean to remove all dependencies that were previously installed and then restarted homeassistant again. I will do that sometime next week probably but I'm pretty sure that doing so risks breaking something I missed.

Edit: I took a quick closer look at the zwave-js-server-python, which I also remember updating to version 0.48.1 but I see now it was completely uneccessary either way because this package is from this overlay and its outdated. My homeassistant installation already installed version 0.51.2 of this package in it's virtual environment, so it's better to just remove this use flag and dependency even if you use zwave.

The other part you need is zwave-js-server which tatsh is maintaining in his overlay, or an installation of zwavejs2mqtt (which I think has changed name to "Z-Wave JS UI" now).

Sure, here's a link: https://pastebin.com/hAhSrUcP It probably has a bunch of problems for machines other than mine. I don't really know how to make ebuilds, but small hacks and generating new digest files I can handle. The things I remember changing are: 1. Removing almost all dependencies but I left everything regarding systemd and zwave-js since I use those two and I know the zwave-js service is running outside of homeassistant (this package which is from the tatsh overlay is probably not pulled in by the part I left in there though.. not entirely sure). 2. Removing most things regarding testing since I don't really care about that. The "test" use-flag still exists though which I'm not sure why. I don't have it selected. 3. Found something about python 3.9 which I removed since it seemed inconsistent and I'm on 3.11 now (and so is official homeassistant). 4. Removed sqlite dependency since my installation is migrated to postgresql. Not sure if I forgot some other small change but that's what I remember. There are still a few dependencies left that didn't require any useflags and weren't from the dev-python category. If you use the default file database you probably want to put the sqlite dependency back. The tar.gz that gets downloaded seems to just be based on the name of the ebuild file. In my case I modified the homeassistant-min package since that was what I was using before but a cleaner version of this would probably get rid of the -min/-full packages and just do one which relies on a virtual environment. Preferably the ebuild should set that up properly as well, this is very much just a quick hack. Also worth nothing is that I have not yet tried doing a --depclean to remove all dependencies that were previously installed and then restarted homeassistant again. I will do that sometime next week probably but I'm pretty sure that doing so risks breaking something I missed. Edit: I took a quick closer look at the zwave-js-server-python, which I also remember updating to version 0.48.1 but I see now it was completely uneccessary either way because this package is from this overlay and its outdated. My homeassistant installation already installed version 0.51.2 of this package in it's virtual environment, so it's better to just remove this use flag and dependency even if you use zwave. The other part you need is zwave-js-server which tatsh is maintaining in his overlay, or an installation of zwavejs2mqtt (which I think has changed name to "Z-Wave JS UI" now).

fariouche commented

2023-09-25 14:02:17 +02:00

thanks. It is indeed a lot smaller. I will test it when i have a moment

Ghost commented

2023-09-28 13:14:14 +02:00

An update: I installed 2023.9.3 and tried to depclean which teached me that there's a lot of python packages as well that need to exist in the virtual environment or be installed to the system-wide python packages directory for homeassistant to even be able to start and automatically install the rest of the requirements for integrations in use. I think most (maybe all?) of them are documented here: https://github.com/home-assistant/core/blob/dev/requirements.txt .

But I basically looked at the logs with journalctl and installed everything it was complaining about one by one to fix my installation again. Unless this project becomes more up-to-date again I'll probably keep installing the latest version manually with my own ebuild every time and I'll try to improve it a bit every time but as of right now I don't really have a much better ebuild to share.

An update: I installed 2023.9.3 and tried to depclean which teached me that there's a lot of python packages as well that need to exist in the virtual environment or be installed to the system-wide python packages directory for homeassistant to even be able to start and automatically install the rest of the requirements for integrations in use. I think most (maybe all?) of them are documented here: https://github.com/home-assistant/core/blob/dev/requirements.txt . But I basically looked at the logs with journalctl and installed everything it was complaining about one by one to fix my installation again. Unless this project becomes more up-to-date again I'll probably keep installing the latest version manually with my own ebuild every time and I'll try to improve it a bit every time but as of right now I don't really have a much better ebuild to share.

fariouche commented

2023-09-30 15:12:32 +02:00

Hello,

Yes, installing manually homeassistant we need to first install the requirements.txt (but the versions inside are not enough, we need to check the constraints (at the top of the file) to be 100% we do not end up with an incompatible version :( )

Maybe a solution would be to have a "ultra-min" version, with just the bare minimum for homeasisstant to start. It will be far easier to maintain in case of breakage. This will require a small number of python package from this repo (if they exist)

Hello, Yes, installing manually homeassistant we need to first install the requirements.txt (but the versions inside are not enough, we need to check the constraints (at the top of the file) to be 100% we do not end up with an incompatible version :( ) Maybe a solution would be to have a "ultra-min" version, with just the bare minimum for homeasisstant to start. It will be far easier to maintain in case of breakage. This will require a small number of python package from this repo (if they exist)

fariouche commented

2023-09-30 15:15:15 +02:00

I believe that @onkelbeh do have a script that just install home assistant core with bare minimum requirements, starts it, and and check the downloader packages to compare them against the available ebuilds...

onkelbeh commented

2023-10-07 20:28:23 +02:00

Hey Guys,

was very busy otherwise. Sorry for that.
Will clean up soon and a do a build with 2023.10.x
Have not read all the tickets, will read these during the compiler is running.

Hey Guys, was very busy otherwise. Sorry for that. Will clean up soon and a do a build with 2023.10.x Have not read all the tickets, will read these during the compiler is running.

👍 1

jump commented

2023-10-10 10:49:49 +02:00

I'm very glad to hear from you onkelbeh. Please leave a annotation the next time you're busy otherwise. Just a few words in the README.md that the project is temporary suspended because of other more important things in your life. It's much easier for us to understand what's going on. There are people who care about you and this project, including me.

👍 2

Ghost commented

2023-10-10 14:39:49 +02:00

Also glad to hear you are back onkelbeh, your work on this is much appreciated. I upgraded to 2023.10.1 today with yet another modified ebuild of yours and it worked great.

This time I looked at the requirements.txt file, package-constraints.txt file and re-added all dependencies I previously removed which are available at the correct versions either from the official gentoo repository or this repository and the result was this:
https://pastebin.com/QCmAZDyh

After merging this I ran a --depclean followed by
sudo -u homeassistant pip3.11 install -r requirements.txt --target=/etc/homeassistant/deps/lib/python3.11/site-packages -U

To make sure all required dependencies missing from the ebuild would got installed and then started homeassistant with no issues or error messages. The virtual environment was not emptied before this though so if there were packages I had installed previously which would somehow not get picked up by this procedure I wouldn't have detected it.

Also glad to hear you are back onkelbeh, your work on this is much appreciated. I upgraded to 2023.10.1 today with yet another modified ebuild of yours and it worked great. This time I looked at the requirements.txt file, package-constraints.txt file and re-added all dependencies I previously removed which are available at the correct versions either from the official gentoo repository or this repository and the result was this: https://pastebin.com/QCmAZDyh After merging this I ran a --depclean followed by `sudo -u homeassistant pip3.11 install -r requirements.txt --target=/etc/homeassistant/deps/lib/python3.11/site-packages -U` To make sure all required dependencies missing from the ebuild would got installed and then started homeassistant with no issues or error messages. The virtual environment was not emptied before this though so if there were packages I had installed previously which would somehow not get picked up by this procedure I wouldn't have detected it.

👍 1

fariouche commented

2023-10-21 16:14:23 +02:00

I see that the 2023.10 branch is almost ready to be merged!!

onkelbeh commented

2023-10-21 16:59:04 +02:00

Yep.
But still a lot to do :-)

Yep. But still a lot to do :-)

onkelbeh commented

2023-10-22 19:24:38 +02:00

Just added the missing annotation (or was it Chat-GPT?) to the main page, as well as 2 fresh releases of Home Assistant and ESPHome.

Still, there are a couple of packages without Mister Gorny's pypi-Eclass implemented.
If anybody needs one of these, just let me know. Or file a PR.

Happy automating! (<- hehe, would never have thought of something like this)

Just added the missing annotation (or was it Chat-GPT?) to the main page, as well as 2 fresh releases of Home Assistant and ESPHome. Still, there are a couple of packages without Mister Gorny's pypi-Eclass implemented. If anybody needs one of these, just let me know. Or file a PR. Happy automating! (<- hehe, would never have thought of something like this)

❤️ 1

jump commented

2023-10-28 17:54:53 +02:00

Thank you all together.

jump closed this issue

2023-10-28 17:54:53 +02:00

Sign in to join this conversation.