Does anyone use terraform for onpremise clusters? If so, what is your setup, what hypervisor do you use it with? Are you happy with it? Or maybe you would rather replace it with a set of ansible roles?
I set up Terraform with libvirt for my local VM host; I think it's much better suited for managing infrastructure components than Ansible is.
Outside personal stuff, I've done a few environments where the (mostly unchanging) infrastructure is set up with Terraform and then configuration and operations (like upgrades) are orchestrated with Ansible, and it works well.
Now, I bet someone might be tempted to claim you should never even need to upgrade VM instances and immutable infrastructure solves everything, but sometimes it's just ridiculously simpler to do in-place upgrades; orchestrating image building, testing and deployment is not easier than running an Ansible playbook to do in-place upgrades unless you already have infrastructure that does it for you.
If the software you're installing is properly written and provided via OS package managenment, often you just don't gain enough benefit from immutable systems considering the overhead.
We have an on-site OpenStack cluster and use Terraform in an ad hoc way for managing (some) infrastructure. It's by far the easiest way to do so, opposed to OpenStack's API and SDK (the latter of which is so poorly documented it beggar's belief!). Ansible is usually used in tandem with Terraform, to decouple the infrastructure and configuration management.
We use it for provisioning vsphere VMs with all of the provisioning done through cloud-init. Can't say I like cloud-init but our onprem stuff is pretty simple and almost all of the VMs can be rebuilt if a change is needed so it works well enough but I'd probably use ansible if I were starting new.
