Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

No, I think it's strictly about where aggregations and other computations occur. You hint at that when you mention CSV files. Borgmon keeps X hours of data in memory and looks up anything older from TSDB. I never understood how to aggregate data from the same service, after the fact, in a reliable fashion. Or if it was possible at all. I believe it was only working when looking at data in memory. That's why people had rules to aggregate data at multiple levels as it was being ingested, then persisting that into TSDB.

I don't remember the latter being much more sophisticated than a bunch of files in GFS with the timeseries and some metadata, either. Monarch, I think, is based on Bigtable. It has a richer API. It might even use BT coprocessors to perform computations closer to the where the raw data is stored, rather than in the Monarch frontends. (I haven't watched the public talk, so I'm just going from vague recollection.)



Borgmon/TSDB was also a bigtable, an enormous one known as 'dumptruck'.


I am pretty sure that was developed for Monarch. Perhaps later it was adopted by BM to consolidate storage, but still through a dumb API? I know that originally it had a bunch of files in GFS, with a layer of indirection, because there was a tool to fix them.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: