new ebuild generation script + homeassistant-full-2024.11.3 #4126

oo-stef · 2025-01-14T23:51:11+01:00

oo-stef commented

2025-01-14 23:51:11 +01:00

First-time contributor

Hi,

I re-wrote the ebuild generation script in Python, with some added features (see below).
I would like to go a bit further in a future version (mainly reducing the installed size homeassistant, removing the integrations and translations that were not configured).

I used this script to generate the ebuild for home assistant-full-2024.11.3 which is included.
I also included the dependencies not yet in the tree that were required for my setup.

genebuild.py is based on genebuild.sh
usage: genebuild.py [-h] [version]

integrations used by "core integrations" (as defined in bootstrap.py) cannot be disabled with use-flags any more
requirements of core are read via the requirements.txt file; the requirements of each integration is read from manifest.json of that integration
package_constraints.txt is used to create (soft) blockers
REQUIRED_USE is generated based on the manifest files (some integrations depend on other integrations)

Signed-off-by: Stef Simoens stef.github@bisse.be

Hi, I re-wrote the ebuild generation script in Python, with some added features (see below). I would like to go a bit further in a future version (mainly reducing the installed size homeassistant, removing the integrations and translations that were not configured). I used this script to generate the ebuild for home assistant-full-2024.11.3 which is included. I also included the dependencies not yet in the tree that were required for my setup. genebuild.py is based on genebuild.sh usage: genebuild.py [-h] [version] - integrations used by "core integrations" (as defined in bootstrap.py) cannot be disabled with use-flags any more - requirements of core are read via the requirements.txt file; the requirements of each integration is read from manifest.json of that integration - package_constraints.txt is used to create (soft) blockers - REQUIRED_USE is generated based on the manifest files (some integrations depend on other integrations) Signed-off-by: Stef Simoens <stef.github@bisse.be>

oo-stef added 1 commit 2025-01-14 23:51:11 +01:00

new ebuild generation script + homeassistant-full-2024.11.3 b26562c03d

genebuild.py is based on genebuild.sh
usage: genebuild.py [-h] [version]
- integrations used by "core integrations" (as defined in bootstrap.py) cannot be disabled with use-flags any more
- requirements of core are read via the requirements.txt file; the requirements of each integration is read from manifest.json of that integration
- package_constraints.txt is used to create (soft) blockers
- REQUIRED_USE is generated based on the manifest files (some integrations depend on other integrations)

Also contains the generated ebuild for homeassistant-full-2024.11.3 + the dependencies not yet in the tree for my configuration

Signed-off-by: Stef Simoens <stef.github@bisse.be>

oo-stef referenced this pull request

2025-01-14 23:53:57 +01:00

automatic ebuild generator #4125

fariouche commented

2025-01-15 08:36:23 +01:00

First-time contributor

thanks, looks promising.

What about integrations that are handled by default_config in the configuration.yaml?
do you handle the required python versions automatically too?

Reducing the size of homeassistant was also something i wanted to test. have you already tested? if yes, how much is the size reduction?

thanks, looks promising. What about integrations that are handled by default_config in the configuration.yaml? do you handle the required python versions automatically too? Reducing the size of homeassistant was also something i wanted to test. have you already tested? if yes, how much is the size reduction?

oo-stef commented

2025-01-15 09:17:24 +01:00

First-time contributor

Hello,

there are integrations that are loaded “by default” in bootstrap.py - these (and their requirements) are always pulled in. Examples are “frontend” but also “zone”. See line 204 in the script. And see bootstrap.py.

The integrations in “default_config” are optional but marked with + in the ebuild, so included by default, but they can be disabled by USE=“-dhcp” for example. (cf line 319 in the script).

The generated ebuild can be diff’ed against a previous ebuild. There should be less dependencies, and some dependencies are no longer pinned (e.g. the dependencies present in requirements.txt but in no other integration).

I’m using the 2024.11.3 (ebuild included in this PR as well), and I now have less dependencies and a happier emerge as less packages are pinned.

Regarding the size reduction:

only the “homeassistant” folder takes +-500MiB on my computer
I see that there are +- 100 MiB translations. My idea is to use the L10N use-expand to only include the languages that are interesting for me (so: you will be able to use the languages with this use-expand).
my other idea is to remove the disabled (not included in USE, not marked as “internal”) integrations. Depending on the integrations each one uses … I’m expecting a huge reduction in install size (and also a minimal smoother run, as integrations I do not want to use but are auto-discovered would not be installed nor auto-detected any more).
I already tested the size reduction. There are some edge cases with virtual integrations, and the files in “generated” need to be re-generated. I’m mostly thinking on ways to run this script after the removal WITHOUT adding extra BDEPEND (it should be possible with minor tweaks).

Ideally I would like this PR to be merged before creating the improved script PR.

To answer your last question: this script does NOT list nor create missing dependency ebuilds. I was merging the package and manually adding the not found packages (some versions were already done by @onkelbeh (thanks!) - the missing packages for my config are included in this PR as well).

Hello, there are integrations that are loaded “by default” in bootstrap.py - these (and their requirements) are always pulled in. Examples are “frontend” but also “zone”. See line 204 in the script. And see bootstrap.py. The integrations in “default_config” are optional but marked with + in the ebuild, so included by default, but they can be disabled by USE=“-dhcp” for example. (cf line 319 in the script). The generated ebuild can be diff’ed against a previous ebuild. There should be less dependencies, and some dependencies are no longer pinned (e.g. the dependencies present in requirements.txt but in no other integration). I’m using the 2024.11.3 (ebuild included in this PR as well), and I now have less dependencies and a happier emerge as less packages are pinned. Regarding the size reduction: - only the “homeassistant” folder takes +-500MiB on my computer - I see that there are +- 100 MiB translations. My idea is to use the L10N use-expand to only include the languages that are interesting for me (so: you will be able to use the languages with this use-expand). - my other idea is to remove the disabled (not included in USE, not marked as “internal”) integrations. Depending on the integrations each one uses … I’m expecting a huge reduction in install size (and also a minimal smoother run, as integrations I do not want to use but are auto-discovered would not be installed nor auto-detected any more). - I already tested the size reduction. There are some edge cases with virtual integrations, and the files in “generated” need to be re-generated. I’m mostly thinking on ways to run this script after the removal WITHOUT adding extra BDEPEND (it should be possible with minor tweaks). Ideally I would like this PR to be merged before creating the improved script PR. To answer your last question: this script does NOT list nor create missing dependency ebuilds. I was merging the package and manually adding the not found packages (some versions were already done by @onkelbeh (thanks!) - the missing packages for my config are included in this PR as well).

fariouche commented

2025-01-15 12:38:43 +01:00

First-time contributor

thanks for the clarifications.
in my case i'm running ha in a readonly filesystem that 8 want to keep small if possible.
in the past, not so long ago, i was only using the pyc files and deleted all the py files. it used to work but something broke after a release, most likely they changed the way they import modules. i was saving 50% of the size.

i also tried in the past to remove integrations that i don't need, but failed (don't remember the details but i think patching ha was necessary)
as long as you don't rely on dynamically added integrations it should be fine(like hacs)

about hacs, i've created a script that intgrates external integrations. I've ebuilds for adaptive lightings and mi-scale. it's very simple to add them but a bit specific to my system (symlinks to ha home folder to the folders were the integrations are installed)

thanks for the clarifications. in my case i'm running ha in a readonly filesystem that 8 want to keep small if possible. in the past, not so long ago, i was only using the pyc files and deleted all the py files. it used to work but something broke after a release, most likely they changed the way they import modules. i was saving 50% of the size. i also tried in the past to remove integrations that i don't need, but failed (don't remember the details but i think patching ha was necessary) as long as you don't rely on dynamically added integrations it should be fine(like hacs) about hacs, i've created a script that intgrates external integrations. I've ebuilds for adaptive lightings and mi-scale. it's very simple to add them but a bit specific to my system (symlinks to ha home folder to the folders were the integrations are installed)

bass_dr referenced this pull request

2025-03-02 21:11:19 +01:00

automatic ebuild generator #4125

This pull request has changes conflicting with the target branch.