> Now that storage is infinite and cheap, it's become trendy for engineers to never actually remove anything, but to just set a "deleted" flag in a database somewhere. This makes a lot of tricky things easier, but there are pitfalls.
Also deleting stuff for real for real immediately is really hard.
You need to delete it from the database, the one that's replicating to, caches, online backups AND offline backups. And that's after you've confirmed and reconfirmed from the user that they actually want to delete the data, not just store it in the recycle bin or something idiotic like that.
The easiest way is just to flag as deleted and prune on some kind of schedule. Backups will be overwritten at some point and the data will go away eventually.
Or, though a more technically complex, encrypt every "deletable" unit of data with its own key, have a different backup policy for encrypted data and for keys to it - one which, upon user request, would allow you to quickly purge every trace of relevant keys.
This doesn't work well in practice at any kind of scale -- these types of databases have existed for many years. The key management overhead is considerable, causing integer factor loss of performance for database-y systems. Additionally, it very significantly increases the storage footprint since most techniques for minimizing storage utilization no longer work given this requirement.
It isn't just "technically complex", the limiting factor is your system may be 10x slower and use 10x more storage. The economics of operating these systems is so poor that they are only used in extremely niche environments where the requirements justify the extreme cost and performance limitations.
Also deleting stuff for real for real immediately is really hard.
You need to delete it from the database, the one that's replicating to, caches, online backups AND offline backups. And that's after you've confirmed and reconfirmed from the user that they actually want to delete the data, not just store it in the recycle bin or something idiotic like that.
The easiest way is just to flag as deleted and prune on some kind of schedule. Backups will be overwritten at some point and the data will go away eventually.