Init Script zwavejs2mqtt #269

New Issue

jump · 2022-03-22T12:14:03+01:00

jump commented

2022-03-22 12:14:03 +01:00

Do you have a more convenient init-script ? The tricky part was to stop the WS Server on port 3000. Changed the owner of /opt/zwavejs2mqtt after building with chown -R homeassistant:homeassistant.

#!/sbin/openrc-run
#Copyright 1999-2016 Gentoo Foundation
#Distributed under the terms of the GNU General Public License v2

name="Z-Wave JS Server"

depend() {
need net
}

start() {
ebegin "Starting ${RC_SVCNAME}"
cd /opt/${RC_SVCNAME}
start-stop-daemon --start --background --pidfile /var/run/${RC_SVCNAME}.pid --make-pidfile --user=homeassistant:homeassistant --exec /usr/bin/yarn -- start
sleep 3
eend $?
}

stop() {
ebegin "Stopping ${RC_SVCNAME}"
start-stop-daemon --stop --signal USR1 --pidfile /var/run/${RC_SVCNAME}.pid
sleep 1
ps aux | grep -v colour | grep -i server/bin/www.js | awk {'print $2'} | xargs kill -9 > /dev/null 2>&1
eend
}

Do you have a more convenient init-script ? The tricky part was to stop the WS Server on port 3000. Changed the owner of /opt/zwavejs2mqtt after building with chown -R homeassistant:homeassistant. #!/sbin/openrc-run #Copyright 1999-2016 Gentoo Foundation #Distributed under the terms of the GNU General Public License v2 name="Z-Wave JS Server" depend() { need net } start() { ebegin "Starting ${RC_SVCNAME}" cd /opt/${RC_SVCNAME} start-stop-daemon --start --background --pidfile /var/run/${RC_SVCNAME}.pid --make-pidfile --user=homeassistant:homeassistant --exec /usr/bin/yarn -- start sleep 3 eend $? } stop() { ebegin "Stopping ${RC_SVCNAME}" start-stop-daemon --stop --signal USR1 --pidfile /var/run/${RC_SVCNAME}.pid sleep 1 ps aux | grep -v colour | grep -i server/bin/www.js | awk {'print $2'} | xargs kill -9 > /dev/null 2>&1 eend }

onkelbeh added the

Enhancement

label 2022-03-24 06:47:10 +01:00

onkelbeh self-assigned this 2022-03-24 06:47:14 +01:00

onkelbeh commented

2022-03-24 07:27:15 +01:00

:-)

to be honest, I don't have a script.

16436 ?        Ss     0:31 SCREEN
16437 pts/2    Ss     0:00 -/bin/bash
17760 ?        I      0:58 [kworker/1:0-events_freezable]
22277 pts/2    Sl+    0:00 node /usr/bin/yarn start
22283 pts/2    Sl+    0:01 /usr/bin/node /opt/zwavejs2mqtt/.yarn/releases/yarn-3.1.1.cjs start
22297 pts/2    Sl+   43:41 /usr/bin/node --preserve-symlinks server/bin/www.js

It runs in a disconnected screen session since a couple of years.
Making an Ebuild for it is on the schedule since .... ever.

:-) to be honest, I don't have a script. ``` 16436 ? Ss 0:31 SCREEN 16437 pts/2 Ss 0:00 -/bin/bash 17760 ? I 0:58 [kworker/1:0-events_freezable] 22277 pts/2 Sl+ 0:00 node /usr/bin/yarn start 22283 pts/2 Sl+ 0:01 /usr/bin/node /opt/zwavejs2mqtt/.yarn/releases/yarn-3.1.1.cjs start 22297 pts/2 Sl+ 43:41 /usr/bin/node --preserve-symlinks server/bin/www.js ``` It runs in a disconnected screen session since a couple of years. Making an Ebuild for it is on the schedule since .... ever.

jump commented

2022-03-24 09:58:01 +01:00

Poster

A better way to stop the WS Server

netstat -nlp | grep node | grep 3000 | awk {'print $7//[!0-9]/'} | xargs kill -9 > /dev/null 2>&1

A better way to stop the WS Server netstat -nlp | grep node | grep 3000 | awk {'print $7//[!0-9]/'} | xargs kill -9 > /dev/null 2>&1

Tatsh commented

2022-03-25 05:49:26 +01:00

I'm doing a similar thing in systemd for Z-Wave JS. It runs as my user but ideally would run as its own dedicated user. Plus read from /etc/zwave.js as I needed this to set my S0 and S2 keys. For some reason that never worked from within HA or core.config_entries file.

I can submit a PR to add this sort of thing.

Also, zwave-js server can be an ebuild. It only has a few dependencies and can be done in a flat manner (compared to the average NPM project having thousands of dependencies and not being able to be flattened).

zwave-js.service:

[Unit]
Description=Z-Wave JS Server
After=network-online.target
Before=homeassistant.service
Wants=network-online.target

[Service]
ExecStart=/usr/bin/node /home/tatsh/.node/bin/zwave-server --host=localhost --port=3000 --config=/etc/zwave.js /dev/serial/by-id/usb-0658_0200-if00
PermissionsStartOnly=True
Restart=on-failure

CapabilityBoundingSet=~CAP_BLOCK_SUSPEND
CapabilityBoundingSet=~CAP_CHOWN
CapabilityBoundingSet=~CAP_FOWNER
CapabilityBoundingSet=~CAP_FSETID
CapabilityBoundingSet=~CAP_IPC_LOCK
CapabilityBoundingSet=~CAP_KILL
CapabilityBoundingSet=~CAP_LEASE
CapabilityBoundingSet=~CAP_LINUX_IMMUTABLE
CapabilityBoundingSet=~CAP_MKNOD
CapabilityBoundingSet=~CAP_NET_ADMIN
CapabilityBoundingSet=~CAP_SETFCAP
CapabilityBoundingSet=~CAP_SETGID
CapabilityBoundingSet=~CAP_SETPCAP
CapabilityBoundingSet=~CAP_SETUID
CapabilityBoundingSet=~CAP_SYS_ADMIN
CapabilityBoundingSet=~CAP_SYS_BOOT
CapabilityBoundingSet=~CAP_SYS_CHROOT
CapabilityBoundingSet=~CAP_SYS_PACCT
CapabilityBoundingSet=~CAP_SYS_PTRACE
CapabilityBoundingSet=~CAP_SYS_RAWIO
CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG
DeviceAllow=char-ttyACM
DevicePolicy=closed
LockPersonality=yes
NoNewPrivileges=yes
PrivateTmp=yes
ProcSubset=pid
ProtectClock=yes
ProtectControlGroups=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=noaccess
ProtectSystem=strict
RemoveIPC=yes
RestrictAddressFamilies=AF_INET
RestrictNetworkInterfaces=lo
RestrictRealtime=yes
RestrictSUIDSGID=yes
SocketBindDeny=any
SystemCallArchitectures=native
SystemCallFilter=~@clock
SystemCallFilter=~@cpu-emulation
SystemCallFilter=~@debug
SystemCallFilter=~@module
SystemCallFilter=~@mount
SystemCallFilter=~@obsolete
SystemCallFilter=~@privileged
SystemCallFilter=~@raw-io
SystemCallFilter=~@reboot
SystemCallFilter=~@resources
SystemCallFilter=~@swap

Group=tatsh
User=tatsh

[Install]
WantedBy=multi-user.target

I'm doing a similar thing in systemd for Z-Wave JS. It runs as my user but ideally would run as its own dedicated user. Plus read from /etc/zwave.js as I needed this to set my S0 and S2 keys. For some reason that never worked from within HA or `core.config_entries` file. I can submit a PR to add this sort of thing. Also, zwave-js server can be an ebuild. It only has a few dependencies and can be done in a flat manner (compared to the average NPM project having thousands of dependencies and not being able to be flattened). `zwave-js.service`: ```ini [Unit] Description=Z-Wave JS Server After=network-online.target Before=homeassistant.service Wants=network-online.target [Service] ExecStart=/usr/bin/node /home/tatsh/.node/bin/zwave-server --host=localhost --port=3000 --config=/etc/zwave.js /dev/serial/by-id/usb-0658_0200-if00 PermissionsStartOnly=True Restart=on-failure CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CapabilityBoundingSet=~CAP_CHOWN CapabilityBoundingSet=~CAP_FOWNER CapabilityBoundingSet=~CAP_FSETID CapabilityBoundingSet=~CAP_IPC_LOCK CapabilityBoundingSet=~CAP_KILL CapabilityBoundingSet=~CAP_LEASE CapabilityBoundingSet=~CAP_LINUX_IMMUTABLE CapabilityBoundingSet=~CAP_MKNOD CapabilityBoundingSet=~CAP_NET_ADMIN CapabilityBoundingSet=~CAP_SETFCAP CapabilityBoundingSet=~CAP_SETGID CapabilityBoundingSet=~CAP_SETPCAP CapabilityBoundingSet=~CAP_SETUID CapabilityBoundingSet=~CAP_SYS_ADMIN CapabilityBoundingSet=~CAP_SYS_BOOT CapabilityBoundingSet=~CAP_SYS_CHROOT CapabilityBoundingSet=~CAP_SYS_PACCT CapabilityBoundingSet=~CAP_SYS_PTRACE CapabilityBoundingSet=~CAP_SYS_RAWIO CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG DeviceAllow=char-ttyACM DevicePolicy=closed LockPersonality=yes NoNewPrivileges=yes PrivateTmp=yes ProcSubset=pid ProtectClock=yes ProtectControlGroups=yes ProtectHostname=yes ProtectKernelLogs=yes ProtectKernelModules=yes ProtectKernelTunables=yes ProtectProc=noaccess ProtectSystem=strict RemoveIPC=yes RestrictAddressFamilies=AF_INET RestrictNetworkInterfaces=lo RestrictRealtime=yes RestrictSUIDSGID=yes SocketBindDeny=any SystemCallArchitectures=native SystemCallFilter=~@clock SystemCallFilter=~@cpu-emulation SystemCallFilter=~@debug SystemCallFilter=~@module SystemCallFilter=~@mount SystemCallFilter=~@obsolete SystemCallFilter=~@privileged SystemCallFilter=~@raw-io SystemCallFilter=~@reboot SystemCallFilter=~@resources SystemCallFilter=~@swap Group=tatsh User=tatsh [Install] WantedBy=multi-user.target ```

onkelbeh commented

2022-03-25 07:04:50 +01:00

What about install / update? OK, to wrench these into an Ebuild, I never did this with node stuff. There aren't any re-usable node-js 'templates' in main repo either. Wish I'd know more about building with yarn, currently I am stuck (again) at the forbidden network access in Gentoo's build sandbox.

I wonder if a pre-compiled binary would work across architectures. Anton made one for the frontend: https://github.com/onkelbeh/HomeAssistantRepository/actions/workflows/make-ha-frontend-sdist.yml

Currently I update it with:

root@ms3:cd /opt/zwavejs2mqtt
root@ms3:/opt/zwavejs2mqtt # git remote -v
origin	https://github.com/zwave-js/zwavejs2mqtt (fetch)
origin	https://github.com/zwave-js/zwavejs2mqtt (push)
root@ms3:/opt/zwavejs2mqtt # git pull origin master
root@ms3:/opt/zwavejs2mqtt # yarn install
root@ms3:/opt/zwavejs2mqtt # yarn run build

then, start with

root@ms3:/opt/zwavejs2mqtt # yarn start

as long I keep it running in foreground (in it's screen session), a simple ^C is catching the process, so I had no problem stopping it, yet. And we should find a clean way to end the process :-)

The update process won't fit into an Ebuild anyway, so a clean install is neccesary.
Where (and how) should the config be located? Think about /etc/{PN}. There binaries to /opt/{PN}?

What about install / update? OK, to wrench these into an Ebuild, I never did this with node stuff. There aren't any re-usable node-js 'templates' in main repo either. Wish I'd know more about building with yarn, currently I am stuck (again) at the forbidden network access in Gentoo's build sandbox. I wonder if a pre-compiled binary would work across architectures. Anton made one for the frontend: https://github.com/onkelbeh/HomeAssistantRepository/actions/workflows/make-ha-frontend-sdist.yml Currently I update it with: ```bash root@ms3:cd /opt/zwavejs2mqtt root@ms3:/opt/zwavejs2mqtt # git remote -v origin https://github.com/zwave-js/zwavejs2mqtt (fetch) origin https://github.com/zwave-js/zwavejs2mqtt (push) root@ms3:/opt/zwavejs2mqtt # git pull origin master root@ms3:/opt/zwavejs2mqtt # yarn install root@ms3:/opt/zwavejs2mqtt # yarn run build ``` then, start with ```bash root@ms3:/opt/zwavejs2mqtt # yarn start ``` as long I keep it running in foreground (in it's screen session), a simple ^C is catching the process, so I had no problem stopping it, yet. And we should find a clean way to end the process :-) The update process won't fit into an Ebuild anyway, so a clean install is neccesary. Where (and how) should the config be located? Think about /etc/${PN}. There binaries to /opt/${PN}?

jump commented

2022-03-25 09:21:47 +01:00

Poster

Both instances are desired (config 8091 and server 3000). Problem is you can start 3000 server from within 8091. So a second init script is useless.

No home entries needed after building so far
chown -R zwavejs2mqtt:zwavejs2mqtt /opt/zwavejs2mqtt
/opt/zwavejs2mqtt/store/settings.json
gpasswd --add zwavejs2mqtt dialout (ttyACL0)
env var HOST=x.x.x.x (Binding of 8091 instance)

#!/sbin/openrc-run
# Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

name="Z-Wave JS Server"

depend() {
	need net
}

checkconfig() {
        if [ ! -e /opt/${RC_SVCNAME} ] ; then
                eerror "No /opt/${RC_SVCNAME} directory exists!"
        fi
}

start() {
	checkconfig || return 1
        ebegin "Starting ${RC_SVCNAME}"
        cd /opt/${RC_SVCNAME}
        start-stop-daemon --start --background --pidfile /var/run/${RC_SVCNAME}.pid --make-pidfile --user=zwavejs2mqtt:zwavejs2mqtt --exec /usr/bin/yarn -- start
        sleep 3
        eend $?
}

stop() {
        ebegin "Stopping ${RC_SVCNAME}"
        start-stop-daemon --stop --signal USR1 --pidfile /var/run/${RC_SVCNAME}.pid
        sleep 1
        netstat -nlp | grep node | grep 3000 | awk {'print $7//[!0-9]/'} | xargs kill -9 > /dev/null 2>&1
        eend
}

Both instances are desired (config 8091 and server 3000). Problem is you can start 3000 server from within 8091. So a second init script is useless. No home entries needed after building so far chown -R zwavejs2mqtt:zwavejs2mqtt /opt/zwavejs2mqtt /opt/zwavejs2mqtt/store/settings.json gpasswd --add zwavejs2mqtt dialout (ttyACL0) env var HOST=x.x.x.x (Binding of 8091 instance) ``` #!/sbin/openrc-run # Copyright 1999-2016 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 name="Z-Wave JS Server" depend() { need net } checkconfig() { if [ ! -e /opt/${RC_SVCNAME} ] ; then eerror "No /opt/${RC_SVCNAME} directory exists!" fi } start() { checkconfig || return 1 ebegin "Starting ${RC_SVCNAME}" cd /opt/${RC_SVCNAME} start-stop-daemon --start --background --pidfile /var/run/${RC_SVCNAME}.pid --make-pidfile --user=zwavejs2mqtt:zwavejs2mqtt --exec /usr/bin/yarn -- start sleep 3 eend $? } stop() { ebegin "Stopping ${RC_SVCNAME}" start-stop-daemon --stop --signal USR1 --pidfile /var/run/${RC_SVCNAME}.pid sleep 1 netstat -nlp | grep node | grep 3000 | awk {'print $7//[!0-9]/'} | xargs kill -9 > /dev/null 2>&1 eend } ```

Tatsh commented

2022-03-25 10:48:19 +01:00

What about install / update? OK, to wrench these into an Ebuild, I never did this with node stuff. There aren't any re-usable node-js 'templates' in main repo either.

You may not have to really build anything. Node stuff is supposed to come with the 'dist' version of the code unless it requires compiling (like code in C++, etc). This is the version of the code that is ready to run. Since we are talking about scripts it's the same as installing a random Perl/Python script in /usr/bin. So generally your ebuild just has to install the dist pre-built files.

I am happy to make these ebuilds, but eventually there should be a script to generate them by using npm or going through the npmjs.com APIs.

> What about install / update? OK, to wrench these into an Ebuild, I never did this with node stuff. There aren't any re-usable node-js 'templates' in main repo either. You may not have to really build anything. Node stuff is supposed to come with the 'dist' version of the code unless it requires compiling (like code in C++, etc). This is the version of the code that is ready to run. Since we are talking about scripts it's the same as installing a random Perl/Python script in `/usr/bin`. So generally your ebuild just has to install the dist pre-built files. I am happy to make these ebuilds, but eventually there should be a script to generate them by using `npm` or going through the `npmjs.com` APIs.

👍 1

onkelbeh commented

2022-03-25 15:17:40 +01:00

You are very welcome :-)

Had to fight a lot with the node stuff from https://github.com/home-assistant/frontend related to Bug #259, where the frontend guys discontinued the SDIST from pypi due to upload restrictions.

I first tried to 'compile' or whatever their install script does in the Ebuild, installation took ~25 minutes on a bigger host. This did not work on smaller appliances like Rapberry or Odroids, and the source on Github did not contain the build artifacts and was also lacking of some newer translations.

Then we made https://github.com/onkelbeh/HomeAssistantRepository/actions/workflows/make-ha-frontend-sdist.yml, which runs about half an hour on Github (with 3 XEON Silver CPU's), which works until now.

The next release of the frontend will use the new Asset.

You are very welcome :-) Had to fight a lot with the node stuff from https://github.com/home-assistant/frontend related to Bug https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/259, where the frontend guys discontinued the SDIST from pypi due to upload restrictions. I first tried to 'compile' or whatever their install script does in the [Ebuild](https://git.edevau.net/onkelbeh/HomeAssistantRepository/commit/ce2d37ebe89f0d5e92430da963d0f79193481946), installation took ~25 minutes on a bigger host. This did not work on smaller appliances like Rapberry or Odroids, and the source on Github did not contain the build artifacts and was also lacking of some newer translations. Then we made https://github.com/onkelbeh/HomeAssistantRepository/actions/workflows/make-ha-frontend-sdist.yml, which runs about half an hour on Github (with 3 XEON Silver CPU's), which works until now. The next release of the frontend will use the [new Asset](https://github.com/home-assistant/frontend/pull/11566).

jump closed this issue

2022-03-31 22:21:09 +02:00

Tatsh commented

2022-04-03 03:53:50 +02:00

As an update on doing zwave-js-server as an ebuild, it's too difficult right now without some script to handle it. The zwave-js dependency is only set as a development one, so I mistakenly thought it would be simple to package just a few Node deps. Simple Node deps do work great as ebuilds if the files get put in /usr/lib/node (one of the only default module paths in /usr).

On a semi-related note, go-module.eclass deprecated EGO_SUM because it makes really large manifests for some packages. Instead Gentoo will now do tarballs of dependencies, which means you need somewhere to host said tarballs unfortunately. On this front, Node.js packages can be treated the same way, and every package will have to have its own root like /usr/lib/node/<package name> and all the deps go in /usr/lib/node/<package name>/node_modules as captured from npm or similar.

The approach used by Python and Perl in Gentoo is not scalable for Node.js. Multiple issues:

Too many ebuilds to maintain
Most major packages cannot use a flat dependency tree (slotting would be a nightmare)
Versioning of often used dependencies is sporadic between major packages even for simple ones like is-string

As an update on doing zwave-js-server as an ebuild, it's too difficult right now without some script to handle it. The `zwave-js` dependency is only set as a development one, so I mistakenly thought it would be simple to package just a few Node deps. Simple Node deps do work great as ebuilds if the files get put in `/usr/lib/node` (one of the only default module paths in `/usr`). On a semi-related note, `go-module.eclass` deprecated `EGO_SUM` because it makes really large manifests for some packages. Instead Gentoo will now do tarballs of dependencies, which means you need somewhere to host said tarballs unfortunately. On this front, Node.js packages can be treated the same way, and every package will have to have its own root like `/usr/lib/node/<package name>` and all the deps go in `/usr/lib/node/<package name>/node_modules` as captured from npm or similar. The approach used by Python and Perl in Gentoo is not scalable for Node.js. Multiple issues: * Too many ebuilds to maintain * Most major packages cannot use a flat dependency tree (slotting would be a nightmare) * Versioning of often used dependencies is sporadic between major packages even for simple ones like `is-string`

onkelbeh commented

2022-04-03 10:36:38 +02:00

Think we should keep this case open for the Ebuild.

Another approach could be to install the whole stuff, and pack it to a 'binary' release (like we do with the frontend). Cannot tell if a pre-installed binary will be independend (platform, locale, etc.).

Hosting won't be a problem, and I think it could be possible to setup an action at Github to do the work.

Take a look at https://github.com/onkelbeh/HomeAssistantRepository/blob/master/.github/workflows/make-ha-frontend-sdist.yml, this one builds the frontend. It puts it to my Nextcloud, currently I have the binary frontend published on hasf.edevau.net (Home Assistant Supplementary Files = hasf), mostly to keep track of the count of 'real' installations.

Think we should keep this case open for the Ebuild. Another approach could be to install the whole stuff, and pack it to a 'binary' release (like we do with the frontend). Cannot tell if a pre-installed binary will be independend (platform, locale, etc.). Hosting won't be a problem, and I think it could be possible to setup an action at Github to do the work. Take a look at https://github.com/onkelbeh/HomeAssistantRepository/blob/master/.github/workflows/make-ha-frontend-sdist.yml, this one builds the frontend. It puts it to my Nextcloud, currently I have the binary frontend published on hasf.edevau.net (Home Assistant Supplementary Files = hasf), mostly to keep track of the count of 'real' installations.

onkelbeh reopened this issue

2022-04-03 10:36:38 +02:00

Tatsh commented

2022-04-04 01:27:53 +02:00

Here is the ebuild, still a WIP, especially since it does not have a OpenRC init script.

https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.15.0.ebuild

Note that keys are specified in a separate file and these are imported in the configuration file (since the configuration file is normal Node code).

More files can probably be cleaned out in src_prepare().

Here is the ebuild, still a WIP, especially since it does not have a OpenRC init script. https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.15.0.ebuild Note that keys are specified in a separate file and these are imported in the configuration file (since the configuration file is normal Node code). More files can probably be cleaned out in `src_prepare()`.

Tatsh commented

2022-04-04 02:41:07 +02:00

Main issue with above is that NodeJS compiled stuff is both architecture (of course) and ABI version specific. Something compiled for NodeJS 16 will not work on 14. Maybe the reverse is true but I doubt it. One way to resolve this may be to run npm rebuild inside the package directory. This worked on one of my machines to fix ABI issues.

Main issue with above is that NodeJS compiled stuff is both architecture (of course) and ABI version specific. Something compiled for NodeJS 16 will not work on 14. Maybe the reverse is true but I doubt it. One way to resolve this may be to run `npm rebuild` inside the package directory. This worked on one of my machines to fix ABI issues.

Ghost commented

2022-04-07 20:48:01 +02:00

@Tatsh
If you put 'RDEPEND="net-libs/nodejs:="' in the ebuild then portage would rebuild the package every time nodejs is updated.

@Tatsh If you put 'RDEPEND="net-libs/nodejs:="' in the ebuild then portage would rebuild the package every time nodejs is updated.

Tatsh commented

2022-04-07 20:56:01 +02:00

First I have to figure out how to make npm rebuild work within the sandbox. It makes some network requests.

First I have to figure out how to make `npm rebuild` work within the sandbox. It makes some network requests.

Ghost commented

2022-04-07 21:04:19 +02:00

I'm still using homeassistant-bin. Thats the ebuild that makes a python virtualenv.
So i did make a package.env with 'app-misc/homeassistant-bin no-network-sandbox'.
Of course that's a last resort.

I'm still using homeassistant-bin. Thats the ebuild that makes a python virtualenv. So i did make a package.env with 'app-misc/homeassistant-bin no-network-sandbox'. Of course that's a last resort.

onkelbeh commented

2022-04-09 13:54:45 +02:00

sandbox network issues have been the main reason for #269 (comment).

I believe that some sort of node stuff (cache etc.) goes into the building users home dir. After a clean build I have ~250 MB in ~/.npm

sandbox network issues have been the main reason for https://git.edevau.net/onkelbeh/HomeAssistantRepository/issues/269#issuecomment-1594. I believe that some sort of node stuff (cache etc.) goes into the building users home dir. After a clean build I have ~250 MB in `~/.npm`

onkelbeh commented

2022-04-09 13:57:27 +02:00

I have contacted a nodejs expert (IT teacher), unfortunaltely I am out on a trip with limited time & access, will return wednesday.

onkelbeh added the

help wanted

label 2022-04-09 14:25:38 +02:00

Tatsh commented

2022-04-11 07:41:28 +02:00

I have figured out how to package this more properly with Yarn's offline feature. BDEPEND gets yarn added. No big deal to me.

Ebuilds:

Node-gyp is required to build a native dependency that zwave-js-server needs. We should never depend on prebuilt binaries. Any package that does not have a binary dependency does not need Node-gyp.

How to reproduce this for any NPM package (run yarn cache clean first):

Get the dependency URLs by running yarn --verbose add NAME_OF_PACKAGE 2>&1 | egrep 'Performing "GET" request to "https://registry.yarnpkg.*tgz' | sed -e 's/"\.\?//g' > urls.log
I am prefixing these distfiles with node- to avoid generic name collissions like so:
```
sed -re 's/^"//g' -e 's/"\.$//g' -e 's|/-/(.*)|/-/\1 -> node-\1|g' < urls.log
```
Note that this does not fix the @-prefixed packages. For those you should edit the target name and add @SCOPE- after node-. So for example: node-config-9.0.0.tgz becomes node-@zwave-js-config-9.0.0.tgz.
All of these lines become SRC_URI.
In the ebuild, set up src_unpack to not unpack .tgz files.
Set up a packages directory and symlink all the node-* packages removing the node- prefix. See src_prepare.
Set up Yarn to use a prefix (does not matter what it is but it must be accessible and writeable).
Set up Yarn to use an offline mirror. This is the packages directory you made in step 4.
Copy the project's generated package.json to the FILESDIR as well as yarn.lock. You must place these in the work directory prior to running any Yarn command.
You must set the npm_config_nodedir environment variable for Node-gyp to not download a tarball of headers. The correct value for Gentoo is /usr/include/node.
Use the yarn install command, but pass --offline --build-from-source at minimum.
In src_install, if the package has items to place in PATH, symlink them into /usr/bin (or appropriate location) manually. See src_install.

If during src_compile you see an error regarding a missing package attempting to be downloaded by Yarn, add this URI to SRC_URI in the same format as the others.

I have figured out how to package this more properly with Yarn's offline feature. BDEPEND gets yarn added. No big deal to me. Ebuilds: * [node-gyp](https://github.com/Tatsh/tatsh-overlay/blob/master/dev-util/node-gyp/node-gyp-9.0.0.ebuild) * [zwave-js-server](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild) Node-gyp is required to build a native dependency that zwave-js-server needs. We should never depend on prebuilt binaries. Any package that does not have a binary dependency does not need Node-gyp. How to reproduce this for any NPM package (run `yarn cache clean` first): 1. Get the dependency URLs by running `yarn --verbose add NAME_OF_PACKAGE 2>&1 | egrep 'Performing "GET" request to "https://registry.yarnpkg.*tgz' | sed -e 's/"\.\?//g' > urls.log` 2. I am prefixing these distfiles with `node-` to avoid generic name collissions like so: ```shell sed -re 's/^"//g' -e 's/"\.$//g' -e 's|/-/(.*)|/-/\1 -> node-\1|g' < urls.log ``` Note that this does not fix the `@`-prefixed packages. For those you should edit the target name and add `@SCOPE-` after `node-`. So for example: `node-config-9.0.0.tgz` becomes `node-@zwave-js-config-9.0.0.tgz`. All of these lines become `SRC_URI`. 3. In the ebuild, [set up `src_unpack` to not unpack `.tgz` files](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild#L157=). 4. Set up a packages directory and symlink all the `node-*` packages removing the `node-` prefix. See [src_prepare](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild#L157=). 5. [Set up Yarn to use a prefix](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild#L181=) (does not matter what it is but it must be accessible and writeable). 6. [Set up Yarn to use an offline mirror](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild#L182=). This is the packages directory you made in step 4. 5. Copy the project's generated `package.json` to the `FILESDIR` as well as `yarn.lock`. You must place these in the work directory prior to running any Yarn command. 6. [You must set the `npm_config_nodedir` environment variable](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild#L193=) for Node-gyp to not download a tarball of headers. The correct value for Gentoo is `/usr/include/node`. 7. Use the `yarn install` command, but pass `--offline --build-from-source` at minimum. 8. In `src_install`, if the package has items to place in PATH, symlink them into `/usr/bin` (or appropriate location) manually. See [src_install](https://github.com/Tatsh/tatsh-overlay/blob/master/app-misc/zwave-js-server/zwave-js-server-1.16.0.ebuild#L204=). If during `src_compile` you see an error regarding a missing package attempting to be downloaded by Yarn, add this URI to `SRC_URI` in the same format as the others.

🎉 1

onkelbeh commented

2022-04-17 09:55:21 +02:00

This is cool, have not tried it yet, currently too busy otherwise, and I would have to add sysvinit scripts, I do not use systemd anywhere.

Two questions:

would you mind if I add it to this repo?
when are you going to use zigbee2mqtt :-)

This is cool, have not tried it yet, currently too busy otherwise, and I would have to add sysvinit scripts, I do not use systemd anywhere. Two questions: 1) would you mind if I add it to this repo? 2) when are you going to use [zigbee2mqtt](https://github.com/onkelbeh/HomeAssistantRepository/issues/17) :-)

onkelbeh commented

2022-11-01 15:53:47 +01:00

@Tatsh did it already:
https://github.com/Tatsh/tatsh-overlay/tree/master/app-misc/zwave-js-server

@Tatsh did it already: https://github.com/Tatsh/tatsh-overlay/tree/master/app-misc/zwave-js-server

onkelbeh closed this issue

2022-11-01 15:53:47 +01:00

Tatsh commented

2022-11-15 14:38:08 +01:00

This is cool, have not tried it yet, currently too busy otherwise, and I would have to add sysvinit scripts, I do not use systemd anywhere.

Two questions:

would you mind if I add it to this repo?

when are you going to use zigbee2mqtt :-)

on 2), why should i use that? I don't own any Zigbee devices.

> This is cool, have not tried it yet, currently too busy otherwise, and I would have to add sysvinit scripts, I do not use systemd anywhere. > > Two questions: > 1) would you mind if I add it to this repo? > 2) when are you going to use [zigbee2mqtt](https://github.com/onkelbeh/HomeAssistantRepository/issues/17) :-) > on 2), why should i use that? I don't own any Zigbee devices.

Sign in to join this conversation.