I've been playing with homeassistant voice integration and it's pretty darn good

There are some gpt and llama experiments you can run there, but it's finally getting to the point where a local assistant you host yourself might be viable soon

Getting audio into it and having a little speaker so it can talk back is both the easiest and the hardest part. I need to play around more with some esp32s and microphone modules