I’ll have to look into this a little to see if it might be useful for some other things we have, if we’re even allowed to used it. We’re running everything through AAP which is useful for audits. We do have a home-grown tool for installing and maintaining state of a lot of the things we need to setup. We install that during setup with Ansible and wait for it to finish its initial setup to make sure there are no errors before we turn it over to the customer.

The other issue we have is our change management process. We need to provide a single update with the status of everything when it’s done. So our process can only go as fast as the slowest server. We could change this, but it would be a political nightmare I don’t want to deal with.