assist-pipeline piper wyoming whisper #2705

New Issue

RayOfLight · 2024-01-15T22:12:30+01:00

RayOfLight commented

2024-01-15 22:12:30 +01:00

I've installed whisper-cpp, wyoming, but I'm unable to integrate them to hass.

It seems that on hass, you have to install both piper and whisper ADDONS, and piper and whisper INTEGRATIONS.

As our install is "core", I understand we install whisper-cpp system-wide. so far so good.

As I'm not using "addons", my hass install does not detect whisper, this is something I'm doing wrong.

I open this to ask you if anybody has tried to make a local assist pipeline and how did you manage to do that, thank you.

(objective: replace google assistant, interact with hass toggles)

I've installed whisper-cpp, wyoming, but I'm unable to integrate them to hass. It seems that on hass, you have to install both piper and whisper ADDONS, and piper and whisper INTEGRATIONS. As our install is "core", I understand we install whisper-cpp system-wide. so far so good. As I'm not using "addons", my hass install does not detect whisper, this is something I'm doing wrong. I open this to ask you if anybody has tried to make a local assist pipeline and how did you manage to do that, thank you. (objective: replace google assistant, interact with hass toggles)

onkelbeh added the

Enhancement

help wanted

labels 2024-04-08 19:25:25 +02:00

zentoo commented

2024-04-30 14:12:59 +02:00

I've struggled to make it working a few month ago but I've kept my launch scripts that can help you:

Wyoming piper:

Piper installation in venv: python3 -m pip install wyoming-piper

wyoming_piper.sh

#!/bin/bash

VOICE=fr_FR-upmc-medium
DATA="/opt/piper/data"
PIPER="/opt/piper/piper/piper"
HOST=0.0.0.0
PORT=10200

python -m wyoming_piper --data-dir $DATA --piper $PIPER --voice $VOICE --uri tcp://$HOST:$PORT

with data files to download manually or via piper: (choose data size set and langage)

[       4096]  /opt/piper/data
├── [   76733615]  fr_FR-upmc-medium.onnx
└── [       4996]  fr_FR-upmc-medium.onnx.json

Wyoming whisper:

Whisper installation in venv: python3 -m pip install wyoming-faster-whisper

wyoming_whisper.sh

#!/bin/bash
# DATA: curl -L -s https://github.com/rhasspy/models/releases/download/v1.0/asr_faster-whisper-base.tar.gz | tar -zxvf - -C $DATA/

LANG=en
MODEL=base-int8
HOST=0.0.0.0
PORT=10300
DATA="/opt/whisper/data"

python3 -m wyoming_faster_whisper --uri tcp://$HOST:$PORT --model $MODEL --beam-size 1 --language $LANG --data-dir "$DATA" --download-dir "$DATA"

with data files to download manually or via whisper: (choose data size set and int8/int16)

[       4096]  /opt/whisper/data/
├── [       4096]  base
│   ├── [       2020]  config.json
│   ├── [  290423781]  model.bin
│   └── [     459861]  vocabulary.txt
├── [       4096]  base-int8
│   ├── [       2020]  config.json
│   ├── [   79120341]  model.bin
│   └── [     459861]  vocabulary.txt

I remember to have tried docker installation too with these docker images:

rhasspy/wyoming-piper
rhasspy/wyoming-whisper

I've tried to make my own ebuilds but failed because of whisper dependencies that need ctranslate2 (a C++ and Python library for efficient inference with Transformer models). It requires an ebuild that compile C++ and install python wrapper.

Anyway I let it mostly because of really bad efficiency and speed for my main french langage that require a medium model at minimum to be useable and more processing than CPU (so a GPU).
English is working more easily with lighter models but with an expected latency on CPU.

I hope this post from my memory can help you. Good luck !

I've struggled to make it working a few month ago but I've kept my launch scripts that can help you: **Wyoming piper:** Piper installation in venv: `python3 -m pip install wyoming-piper` wyoming_piper.sh ``` #!/bin/bash VOICE=fr_FR-upmc-medium DATA="/opt/piper/data" PIPER="/opt/piper/piper/piper" HOST=0.0.0.0 PORT=10200 python -m wyoming_piper --data-dir $DATA --piper $PIPER --voice $VOICE --uri tcp://$HOST:$PORT ``` with data files to download manually or via piper: (choose data size set and langage) ``` [ 4096] /opt/piper/data ├── [ 76733615] fr_FR-upmc-medium.onnx └── [ 4996] fr_FR-upmc-medium.onnx.json ``` **Wyoming whisper:** Whisper installation in venv: `python3 -m pip install wyoming-faster-whisper` wyoming_whisper.sh ``` #!/bin/bash # DATA: curl -L -s https://github.com/rhasspy/models/releases/download/v1.0/asr_faster-whisper-base.tar.gz | tar -zxvf - -C $DATA/ LANG=en MODEL=base-int8 HOST=0.0.0.0 PORT=10300 DATA="/opt/whisper/data" python3 -m wyoming_faster_whisper --uri tcp://$HOST:$PORT --model $MODEL --beam-size 1 --language $LANG --data-dir "$DATA" --download-dir "$DATA" ``` with data files to download manually or via whisper: (choose data size set and int8/int16) ``` [ 4096] /opt/whisper/data/ ├── [ 4096] base │ ├── [ 2020] config.json │ ├── [ 290423781] model.bin │ └── [ 459861] vocabulary.txt ├── [ 4096] base-int8 │ ├── [ 2020] config.json │ ├── [ 79120341] model.bin │ └── [ 459861] vocabulary.txt ``` I remember to have tried docker installation too with these docker images: - rhasspy/wyoming-piper - rhasspy/wyoming-whisper I've tried to make my own ebuilds but failed because of whisper dependencies that need ctranslate2 (a C++ and Python library for efficient inference with Transformer models). It requires an ebuild that compile C++ and install python wrapper. Anyway I let it mostly because of really bad efficiency and speed for my main french langage that require a medium model at minimum to be useable and more processing than CPU (so a GPU). English is working more easily with lighter models but with an expected latency on CPU. I hope this post from my memory can help you. Good luck !

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: onkelbeh/HomeAssistantRepository#2705