Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Prometheus is as convenient as /streamz in the sense that it has service discovery, so you don't have to do any work to have your endpoints discovered and scraped.

I haven't found anything as good as Monarch and the internal dashboard, though. Monarch always made sense to me, by not doing anything clever. For example, if you want to aggregate across different streams, you have to "align" them first, and the alignment is the operation that defines how to make up missing points. Other systems don't have that alignment step, and it causes me to struggle with everything else I want to do.

As far as I can tell, Prometheus and InfluxDB have implicit alignment, but are never clear on what the rules are. At my last job, I collected some of the exact same metrics as I did at Google, and while I knew exactly how to get the charts I wanted with Monarch, I could never figure out how to get them with InfluxDB (I later found out that it simply didn't support my use case). (The exact use case was collecting counters from network devices opportunistically, and then generating a bandwidth chart for a section of the network. At Google it was easy; I could do the differencing to turn the packet counters into "bytes sent over the last X seconds", then align the streams so that data was available at each time, then aggregate over topology. With InfluxDB... the query language lets you express that, but it doesn't yield correct results. I complained about it on HN and the author of did a lot of handwaving about how what I want to do is wrong, or something, and so I just wrote my own thing instead.)

The other thing I miss from Monarch is the default visualization for histograms. I have to manually reproduce that in Grafana with a series of manual queries like histogram_quantile(0.99, ...), histogram_quantile(0.90, ...), histogram_quantile(0.85, ...), ... where that was just the default visualization. Again, the open source world has some defaults, but I can't make heads or tails of what they are (try a default histogram visualization in Grafana)... Monarch just did the right thing by default.

I miss it.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: