Good question! So, there's actually clear separation of concerns between Ansible and Cinc that makes a lot of sense for this LXC-based approach.
First, these aren't immutable containers – they're LXC containers treated as persistent, stateful systems (more VM-like than Docker-like).
Here's how the tools are used:
Ansible handles the "outside" (Proxmox host level):
- Container provisioning via Proxmox API (`base/roles/container/`)
- LXC lifecycle management (create, start, stop, destroy)
- Base infrastructure setup (SSH keys, networking, storage mounts)
- Host-to-container bootstrapping
Cinc (Chef) handles the "inside" (within each container):
- Application-specific configuration (`config/recipes/`, `libs//recipes/`)
- Service management and desired state enforcement
- Runtime configuration updates
- Cross-container state coordination (like the Git service managing repositories)
*Why this makes sense for LXC:*
Unlike Docker's "build once, deploy everywhere" philosophy, LXC containers in this system are *long-lived infrastructure pieces* that need ongoing configuration management. Each container might run for months/years and needs to adapt to changing requirements, handle updates, manage state, etc.
The recursive self-containment aspect is particularly clever – the control plane (running in its own LXC container) uses the exact same base configuration and tooling as the containers it manages, ensuring consistency and enabling the whole system to bootstrap itself.
So while you're right that immutable containers don't typically need provisioning stages, this isn't really following the immutable container pattern – it's more like "Infrastructure as Code" for persistent container-based services, which absolutely benefits from both provisioning and configuration management layers.