The LLM is also allowed to respond to arbitrary events of is own, such as time stamps and motion detectors, and it just comes up with what it thinks you want. It would probably work at least as well as these other systems, until it decided that one of your light names was too naughty.

Actually, I created a toy system of a virtual robot that could consume events like FORWARD, LEFT, LOOK UP, SAY [x], etc. ChatGPT was able to control it really well.