Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

My experience has been a mixture of "when all you have is a hammer ..." and Pointy Haired Bosses LOVE kafka, and tend to default to it because it's what all their Pointy Haired Boss friends are using

In a more generous take, using some buffered ingest does help with not having to choose between a c500.128xl ingest machine and dropping messages, but I would never advocate for standing up kafka just for log buffering



at that point you are likely slowing down your applications - I think a basic OpenTelemetry collector mostly solves this, and if you go beyond the available buffer there, then dropping it is the appropriate choice for application logs.


Dropping may be an unacceptable choice for some applications, though. For example dropping request logs is really bad, because now you have no idea who is interacting with your service. If a security breach happens and your answer is "like, bro, idk what happened man, we load shedded the logs away" that's not a great look...




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: