New integrations and dependencies #17

New Issue

zentoo · 2020-08-26T14:17:45+02:00

zentoo commented

2020-08-26 14:17:45 +02:00

On my system I use several integrations that are not in home-assistant ebuild and USE flags.
In order that you can integrate them I clean the home assistant deps directory and take a look in log and in the new deps directory generated.

From logs:

2020-08-26 11:26:01 ERROR (MainThread) [homeassistant.setup] Setup failed for nodered: Unable to import component: No module named 'integrationhelper'
2020-08-26 11:26:05 ERROR (MainThread) [homeassistant.setup] Setup failed for breaking_changes: Unable to import component: No module named 'integrationhelper'
2020-08-26 11:26:07 ERROR (MainThread) [homeassistant.setup] Setup failed for ipp: Unable to import component: No module named 'pyipp'
2020-08-26 11:26:07 ERROR (MainThread) [homeassistant.setup] Setup failed for brother: Unable to import component: No module named 'brother'
2020-08-26 11:26:15 ERROR (MainThread) [homeassistant.setup] Setup failed for hacs: Unable to import component: No module named 'backoff'
2020-08-26 11:26:15 ERROR (MainThread) [homeassistant.setup] Setup failed for upnp: Unable to import component: No module named 'async_upnp_client'
2020-08-26 11:26:20 ERROR (MainThread) [homeassistant.setup] Setup failed for freebox: Unable to import component: No module named 'aiofreepybox'
2020-08-26 12:49:06 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration xiaomi_aqara: No module named 'xiaomi_gateway'

From new deps directory:

Cryptodome
PyQRCode-1.2.1.dist-info
PyXiaomiGateway-0.13.2.dist-info
__pycache__
aiofreepybox
aiofreepybox-0.0.8.dist-info
aiogithubapi
aiogithubapi-1.1.2.dist-info
async_upnp_client
async_upnp_client-0.14.13.dist-info
backoff
backoff-1.10.0.dist-info
brother
brother-0.1.14.dist-info
construct
construct-2.9.45.dist-info
deepmerge
deepmerge-0.1.0.dist-info
didl_lite
hacs_frontend
hacs_frontend-202008231558.dist-info
integrationhelper
integrationhelper-0.2.2.dist-info
ppadb
ptr.py
pure_python_adb-0.2.2.dev0.dist-info
pycryptodomex-3.9.8.dist-info
pyhaversion
pyhaversion-3.3.0.dist-info
pyipp
pyipp-0.10.1.dist-info
pyotp
pyotp-2.3.0.dist-info
pyqrcode
pytest_runner-5.2.dist-info
python_didl_lite-1.2.4.dist-info
queueman
queueman-0.5.dist-info
semantic_version
semantic_version-2.8.5.dist-info
tests
tools
xiaomi_gateway

On my system I use several integrations that are not in home-assistant ebuild and USE flags. In order that you can integrate them I clean the home assistant deps directory and take a look in log and in the new deps directory generated. From logs: ``` 2020-08-26 11:26:01 ERROR (MainThread) [homeassistant.setup] Setup failed for nodered: Unable to import component: No module named 'integrationhelper' 2020-08-26 11:26:05 ERROR (MainThread) [homeassistant.setup] Setup failed for breaking_changes: Unable to import component: No module named 'integrationhelper' 2020-08-26 11:26:07 ERROR (MainThread) [homeassistant.setup] Setup failed for ipp: Unable to import component: No module named 'pyipp' 2020-08-26 11:26:07 ERROR (MainThread) [homeassistant.setup] Setup failed for brother: Unable to import component: No module named 'brother' 2020-08-26 11:26:15 ERROR (MainThread) [homeassistant.setup] Setup failed for hacs: Unable to import component: No module named 'backoff' 2020-08-26 11:26:15 ERROR (MainThread) [homeassistant.setup] Setup failed for upnp: Unable to import component: No module named 'async_upnp_client' 2020-08-26 11:26:20 ERROR (MainThread) [homeassistant.setup] Setup failed for freebox: Unable to import component: No module named 'aiofreepybox' 2020-08-26 12:49:06 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration xiaomi_aqara: No module named 'xiaomi_gateway' ``` From new deps directory: ``` Cryptodome PyQRCode-1.2.1.dist-info PyXiaomiGateway-0.13.2.dist-info __pycache__ aiofreepybox aiofreepybox-0.0.8.dist-info aiogithubapi aiogithubapi-1.1.2.dist-info async_upnp_client async_upnp_client-0.14.13.dist-info backoff backoff-1.10.0.dist-info brother brother-0.1.14.dist-info construct construct-2.9.45.dist-info deepmerge deepmerge-0.1.0.dist-info didl_lite hacs_frontend hacs_frontend-202008231558.dist-info integrationhelper integrationhelper-0.2.2.dist-info ppadb ptr.py pure_python_adb-0.2.2.dev0.dist-info pycryptodomex-3.9.8.dist-info pyhaversion pyhaversion-3.3.0.dist-info pyipp pyipp-0.10.1.dist-info pyotp pyotp-2.3.0.dist-info pyqrcode pytest_runner-5.2.dist-info python_didl_lite-1.2.4.dist-info queueman queueman-0.5.dist-info semantic_version semantic_version-2.8.5.dist-info tests tools xiaomi_gateway ```

zentoo commented

2020-08-26 16:18:38 +02:00

xiaomi_gateway:

needed by home assistant xiaomi_aqara component
more recent version of :dev-python/PyXiaomiGateway

brother:
pyipp:

seems to replace dev-python/brother that is not a dependencie of homeassistant
both ebuild doesn't compile actually
could be a "brother" USE flag
for: brother printers integration
need: dev-python/pysnmp dev-python/pyipp (not sure but printer related)

aiofreepybox:

could be a "freebox" USE flag
for: modem/router from french ISP "Free"
need: dev-python/tqdm dev-python/urllib3 dev-python/aiohttp

async_upnp_client:

could be a "upnp" USE flag
for: UPnP integration (universal plug and play)
need: dev-python/tqdm dev-python/pygments dev-python/future dev-python/dateparser dev-python/voluptuous dev-python/aiohttp dev-python/async_timeout dev-python/defusedxml dev-python/urllib3

hacs_frontend:

needed by home assistant hacs component
could be a "hacs" USE flag that add hacs and hacs_frontend component
no specific deps

integrationhelper:

needed by nodered and breaking_changes integration
could be a nodered and breaking_changes USE flags
nodered USE flag is related to nodered (nodejs based application) that need an ebuild

ppadb:

older version of dev-python/pure-python-adb

pyhaversion:

seems a more recent release of dev-python/pyhaversion
it seems to be regenerated at each homeassistant startup

construct:

an old version of dev-python/construct
here because of dev-python/construct::gentoo more recent
I masked dev-python/construct::gentoo to get rid of this one

backoff:

seems to be the same version than dev-python/backoff (not installed)
can be a needed dependencie of homeassistant

didl_lite:

more recent version of dev-python/python-didl-lite (not installed)
can be a needed dependencie of homeassistant

Cryptodome:

more recent version than dev-python/pycryptodomex (not installed)
can be a needed dependencie of homeassistant

semantic_version:

the same than dev-python/semantic_version (not installed)
deps installed version is the same than gentoo repository one
can be a needed dependencie of homeassistant

pyqrcode:

strange because it's the same version than dev-python/pyqrcode (installed)

pyotp:

an old version of dev-python/pyotp
here because of dev-python/pyotp::gentoo more recent
I masked dev-python/pyotp::gentoo to get rid of this one

queueman:

https://pypi.org/project/queueman/
can be a needed dependencie of homeassistant

aiogithubapi:

https://pypi.org/project/aiogithubapi/
can be a needed dependencie of homeassistant

deepmerge:

https://pypi.org/project/deepmerge/
can be a needed dependencie of homeassistant

pytest_runner-5.2.dist-info & tests:

seems related to https://pypi.org/project/pytest-runner/
it's not a simple package but tests using pytest-runner

tools:

contain batterystats_codegen.py

I think these informations can be really usefull to fix actual homeassistant dependencies.

xiaomi_gateway: - needed by home assistant xiaomi_aqara component - more recent version of :dev-python/PyXiaomiGateway brother: pyipp: - seems to replace dev-python/brother that is not a dependencie of homeassistant - both ebuild doesn't compile actually - could be a "brother" USE flag - for: brother printers integration - need: dev-python/pysnmp dev-python/pyipp (not sure but printer related) aiofreepybox: - could be a "freebox" USE flag - for: modem/router from french ISP "Free" - need: dev-python/tqdm dev-python/urllib3 dev-python/aiohttp async_upnp_client: - could be a "upnp" USE flag - for: UPnP integration (universal plug and play) - need: dev-python/tqdm dev-python/pygments dev-python/future dev-python/dateparser dev-python/voluptuous dev-python/aiohttp dev-python/async_timeout dev-python/defusedxml dev-python/urllib3 hacs_frontend: - needed by home assistant hacs component - could be a "hacs" USE flag that add hacs and hacs_frontend component - no specific deps integrationhelper: - needed by nodered and breaking_changes integration - could be a nodered and breaking_changes USE flags - nodered USE flag is related to nodered (nodejs based application) that need an ebuild ppadb: - older version of dev-python/pure-python-adb pyhaversion: - seems a more recent release of dev-python/pyhaversion - it seems to be regenerated at each homeassistant startup construct: - an old version of dev-python/construct - here because of dev-python/construct::gentoo more recent - I masked dev-python/construct::gentoo to get rid of this one backoff: - seems to be the same version than dev-python/backoff (not installed) - can be a needed dependencie of homeassistant didl_lite: - more recent version of dev-python/python-didl-lite (not installed) - can be a needed dependencie of homeassistant Cryptodome: - more recent version than dev-python/pycryptodomex (not installed) - can be a needed dependencie of homeassistant semantic_version: - the same than dev-python/semantic_version (not installed) - deps installed version is the same than gentoo repository one - can be a needed dependencie of homeassistant pyqrcode: - strange because it's the same version than dev-python/pyqrcode (installed) pyotp: - an old version of dev-python/pyotp - here because of dev-python/pyotp::gentoo more recent - I masked dev-python/pyotp::gentoo to get rid of this one queueman: - https://pypi.org/project/queueman/ - can be a needed dependencie of homeassistant aiogithubapi: - https://pypi.org/project/aiogithubapi/ - can be a needed dependencie of homeassistant deepmerge: - https://pypi.org/project/deepmerge/ - can be a needed dependencie of homeassistant pytest_runner-5.2.dist-info & tests: - seems related to https://pypi.org/project/pytest-runner/ - it's not a simple package but tests using pytest-runner tools: - contain batterystats_codegen.py I think these informations can be really usefull to fix actual homeassistant dependencies.

zentoo commented

2020-08-26 16:48:36 +02:00

Actually I've fixed several ones by:

adding in package.mask:

 # homeassistant 0.114.3
dev-python/responses::gentoo
dev-python/construct::gentoo
dev-python/pyotp::gentoo

emerge:
dev-python/semantic_version
dev-python/pycryptodomex
dev-python/backoff

Actually I've fixed several ones by: - adding in package.mask: ``` # homeassistant 0.114.3 dev-python/responses::gentoo dev-python/construct::gentoo dev-python/pyotp::gentoo ``` - emerge: dev-python/semantic_version dev-python/pycryptodomex dev-python/backoff

onkelbeh added the

Bug

Enhancement

labels 2020-08-26 19:01:42 +02:00

onkelbeh self-assigned this 2020-08-26 19:01:50 +02:00

onkelbeh commented

2020-08-26 19:01:58 +02:00

Whoa, this list is too big to solve,
perhaps you can split this in smaller units next time,
let's split this up a bit:

For the compile errors I created seperate tickets:

pyipp: #18
brother: #19

for the cleanup jobs:

backoff: #21
pycryptodomex: #22

For the useflags on your wishlist: #20
Although, there are way too many components/integrations to adopt all in use flags.
Some time ago I had a dev ebuild with all components, the last version of it still resides in app-misc/homeassistant, I just renamed it to homeassistant-9999.off, I can fill it with the actual component list.

Unsure what you mean with:

pyqrcode (can't see your installed version)

[I] dev-python/pyqrcode
    Verfügbare Versionen:   1.2.1-r1^t{xpak:2} {png test PYTHON_TARGETS="python3_6 python3_7 python3_8 python3_9"}
    Installierte Versionen: 1.2.1-r1^t{xpak:2}(13:22:47 15.08.2020)(-png -test PYTHON_TARGETS="python3_7 -python3_6 -python3_8 -python3_9")
    Startseite:             https://github.com/mnooner256/pyqrcode https://pypi.org/project/PyQRCode/
    Beschreibung:           A pure Python QR code generator with SVG, EPS, PNG and terminal output

pyotp (2.3.0 exists as ebuild)

 [I] dev-python/pyotp
     Verfügbare Versionen:   ~2.2.6[1] ~2.2.7[1] 2.3.0[1] ~2.4.0 {test PYTHON_TARGETS="python3_6 python3_7 python3_8"}
     Installierte Versionen: 2.3.0[1](10:12:02 10.05.2020)(-test PYTHON_TARGETS="python3_7 -python3_6 -python3_8")
     Startseite:             https://github.com/pyauth/pyotp https://pypi.org/project/pyotp/
     Beschreibung:           PyOTP is a Python library for generating and verifying one-time passwords.

The only explanation I have is that the versions you installed differ.

Cannot find any references from HA for:

queueman
deepmerge

Perhaps you can do a grep -ir for them in either:

/usr/lib/python3.7/site-packages/homeassistant/components
/usr/lib/python3.7/site-packages/homeassistant
and, if nothing helps in
/usr/lib/python3.7/site-packages

Perhaps we can find which package pulls them.

I would avoid to use package.mask in this case, I would be nicer if we find whatever pulls the wrong versions in.

For the other components ebuilds are in the repo:

* dev-python/PyXiaomiGateway [1]
     Verfügbare Versionen:   ~0.11.2 ~0.12.0 ~0.12.2 ~0.12.3 ~0.12.4 {test PYTHON_TARGETS="python3_6 python3_7 python3_8"}
     Startseite:             https://github.com/Danielhiversen/PyXiaomiGateway/ https://pypi.org/project/PyXiaomiGateway/
     Beschreibung:           A library to communicate with the Xiaomi Gateway

except:

hacs (never heard)
nodered, my skills with node.js are insufficient to nail an ebuild for it :-)

Whoa, this list is too big to solve, perhaps you can split this in smaller units next time, let's split this up a bit: For the compile errors I created seperate tickets: * pyipp: https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/18 * brother: https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/19 for the cleanup jobs: * backoff: https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/21 * pycryptodomex: https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/22 For the useflags on your wishlist: https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/20 Although, there are way too many components/integrations to adopt all in use flags. Some time ago I had a dev ebuild with all components, the last version of it still resides in app-misc/homeassistant, I just renamed it to homeassistant-9999.off, I can fill it with the actual component list. Unsure what you mean with: - pyqrcode (can't see your installed version) ``` [I] dev-python/pyqrcode Verfügbare Versionen: 1.2.1-r1^t{xpak:2} {png test PYTHON_TARGETS="python3_6 python3_7 python3_8 python3_9"} Installierte Versionen: 1.2.1-r1^t{xpak:2}(13:22:47 15.08.2020)(-png -test PYTHON_TARGETS="python3_7 -python3_6 -python3_8 -python3_9") Startseite: https://github.com/mnooner256/pyqrcode https://pypi.org/project/PyQRCode/ Beschreibung: A pure Python QR code generator with SVG, EPS, PNG and terminal output ``` - pyotp (2.3.0 exists as ebuild) ``` [I] dev-python/pyotp Verfügbare Versionen: ~2.2.6[1] ~2.2.7[1] 2.3.0[1] ~2.4.0 {test PYTHON_TARGETS="python3_6 python3_7 python3_8"} Installierte Versionen: 2.3.0[1](10:12:02 10.05.2020)(-test PYTHON_TARGETS="python3_7 -python3_6 -python3_8") Startseite: https://github.com/pyauth/pyotp https://pypi.org/project/pyotp/ Beschreibung: PyOTP is a Python library for generating and verifying one-time passwords. ``` The only explanation I have is that the versions you installed differ. Cannot find any references from HA for: 1. queueman 1. deepmerge Perhaps you can do a grep -ir for them in either: - /usr/lib/python3.7/site-packages/homeassistant/components - /usr/lib/python3.7/site-packages/homeassistant and, if nothing helps in - /usr/lib/python3.7/site-packages Perhaps we can find which package pulls them. I would avoid to use package.mask in this case, I would be nicer if we find whatever pulls the wrong versions in. For the other components ebuilds are in the repo: ``` * dev-python/PyXiaomiGateway [1] Verfügbare Versionen: ~0.11.2 ~0.12.0 ~0.12.2 ~0.12.3 ~0.12.4 {test PYTHON_TARGETS="python3_6 python3_7 python3_8"} Startseite: https://github.com/Danielhiversen/PyXiaomiGateway/ https://pypi.org/project/PyXiaomiGateway/ Beschreibung: A library to communicate with the Xiaomi Gateway ``` except: * hacs (never heard) * nodered, my skills with node.js are insufficient to nail an ebuild for it :-)

onkelbeh commented

2020-08-26 20:19:12 +02:00

Hmmm.
An ebuild for node-red would be a big thing, way too diffcult for me.
There must be a reason why it isn't done yet.

Hmmm. An ebuild for node-red would be a big thing, way too diffcult for me. There must be a reason why it isn't done yet.

zentoo commented

2020-08-27 03:15:44 +02:00

Really sorry for the big list but I've started by one and finally decide to track all dependencies.

I've to mask some gentoo ebuilds because my system is in ~amd64 and so recent version ebuilds from gentoo repo override versions from your repo. But It's ok for me since I manage gentoo servers on daily dutty.

I open new tickets for each package now ;)

Else some misc points:

node-red
I understand for node-red ebuild.
I manage a classical install out of system actually.
hacs
It's one of the most famous integration on homeassistant.
It permit to add others integrations and frontend ressources as themes and additional cards for lovelace.
Your repository and hacs permits me to have a really clean installation globally.
-> https://hacs.xyz/
deepmerge
It's a dependencies of dev-python/pyipp
queueman
I finally find it by greping in whole homeassistant configuration directory:
It's a dependencies of hacs in homeassistant custom_components.
pyqrcode
I really don't understand this one:
I got dev-python/pyqrcode-1.2.1-r1 installed and PyQRCode-1.2.1 in deps directory.
Is it a problem of dupe/other name ?

Really sorry for the big list but I've started by one and finally decide to track all dependencies. I've to mask some gentoo ebuilds because my system is in ~amd64 and so recent version ebuilds from gentoo repo override versions from your repo. But It's ok for me since I manage gentoo servers on daily dutty. I open new tickets for each package now ;) Else some misc points: * node-red I understand for node-red ebuild. I manage a classical install out of system actually. * hacs It's one of the most famous integration on homeassistant. It permit to add others integrations and frontend ressources as themes and additional cards for lovelace. Your repository and hacs permits me to have a really clean installation globally. -> https://hacs.xyz/ * deepmerge It's a dependencies of dev-python/pyipp * queueman I finally find it by greping in whole homeassistant configuration directory: It's a dependencies of hacs in homeassistant custom_components. * pyqrcode I really don't understand this one: I got dev-python/pyqrcode-1.2.1-r1 installed and PyQRCode-1.2.1 in deps directory. Is it a problem of dupe/other name ?

zentoo commented

2020-08-27 03:17:20 +02:00

Note: pyipp and brother compiles fine but I needed to mask last version that was too much recent for homeassistant.

onkelbeh commented

2020-08-30 12:15:47 +02:00

Hi, Comment ça va?

deepmerge went already in with d381207f75
queueman added 157df878b6

pyqrcode: already had an issue with that: my 'old' package was called PyQRCode (as the Pypi name is), but the same module existed in the main repo with a different name, so I removed mine in bd54952aae, please re-check,
the installed library should be correctly named PyQRCode-1.2.1, perhaps a version issue?
The only references I find are:

components/homekit/manifest.json:    "PyQRCode==1.2.1"
auth/mfa_modules/totp.py:REQUIREMENTS = ["pyotp==2.3.0", "PyQRCode==1.2.1"]

so 1.2.1 should be OK.

node-red is on my list for the upcoming winter :-)
And I will take a look at hacs in the near future.

Hi, Comment ça va? deepmerge went already in with https://git.edevau.net/onkelbeh/HomeAssistantRepository/commit/d381207f75e3ac78ac1e72d768860a74df83134d queueman added https://git.edevau.net/onkelbeh/HomeAssistantRepository/commit/157df878b62f43468723975449b5835f0807482c pyqrcode: already had an issue with that: my 'old' package was called PyQRCode (as the Pypi name is), but the same module existed in the main repo with a different name, so I removed mine in https://git.edevau.net/onkelbeh/HomeAssistantRepository/commit/bd54952aae0ca2d0cb2879a56cbbb6631f6d4020, please re-check, the installed library should be correctly named `PyQRCode-1.2.1`, perhaps a version issue? The only references I find are: ``` components/homekit/manifest.json: "PyQRCode==1.2.1" auth/mfa_modules/totp.py:REQUIREMENTS = ["pyotp==2.3.0", "PyQRCode==1.2.1"] ``` so 1.2.1 should be OK. node-red is on my list for the upcoming winter :-) And I will take a look at hacs in the near future.

zentoo commented

2020-08-30 14:44:50 +02:00

Salut Ca va bien ! :)

You have done a really good work:

pyqrcode: OK
deepmerge: OK
queueman: OK

Salut Ca va bien ! :) You have done a really good work: * pyqrcode: OK * deepmerge: OK * queueman: OK

zentoo commented

2020-08-30 14:53:40 +02:00

Great news for node-red ! By the way, there is also zigbeet2mqtt that is npm based.
It's possible that zigbee2mqtt is more easy than node-red to manage.

My notes for node-red:

INSTALL

su - nodered
npm install node-red

UPDATE

su - nodered
cd .node-red
npm install node-red

My notes for zigbee2mqtt:

INSTALL:
https://www.zigbee2mqtt.io/information/virtual_environment.html
UPDATE:
update.sh modified to use openrc command sintead of systemd ones.

For init.d/conf.d files
If you need I can give you my init.d/conf.d files for both of them.
In fact they are really similar since I create a sort of universal init.d script with default values than can be overriden by parameters in conf.d file (command/options/user/group/pid/log/dependencies).

Great news for node-red ! By the way, there is also zigbeet2mqtt that is npm based. It's possible that zigbee2mqtt is more easy than node-red to manage. **My notes for node-red:** * INSTALL ``` su - nodered npm install node-red ``` * UPDATE ``` su - nodered cd .node-red npm install node-red ``` **My notes for zigbee2mqtt:** * INSTALL: https://www.zigbee2mqtt.io/information/virtual_environment.html * UPDATE: update.sh modified to use openrc command sintead of systemd ones. **For init.d/conf.d files** If you need I can give you my init.d/conf.d files for both of them. In fact they are really similar since I create a sort of universal init.d script with default values than can be overriden by parameters in conf.d file (command/options/user/group/pid/log/dependencies).

onkelbeh commented

2020-08-30 16:59:26 +02:00

Yep, please post them: init.d/conf.d

onkelbeh closed this issue

2021-04-10 22:05:06 +02:00

Sign in to join this conversation.