That would result in a much larger payload, and a payload size that is variable depending on how long it has been, and how much audio was record-able between requests. But if instead we observe a small payload of constant size given the same request, regardless of how long between requests, then we can reasonably be assured the device is not doing this.
Any smart tv could be doing the same thing. Why do you think that Amazon in particular is likely to be conducting a massive, secret and illegal surveillance operation?