Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The Evolution of Data at Reddit (redditblog.com)
60 points by yarapavan on March 7, 2018 | hide | past | favorite | 10 comments


"Third party vendor". Is this Qubole? If so, would be nice to give him an Internet high-five by naming them :)


https://www.flydata.com/blog/hive-and-redshift-a-brief-compa...

Used Redshift for our large data needs. No reason to look to anything else, given our time-shifted data could easily be fit into Redshift. For larger data, we would have to have some other solution.


This is extremely interesting and I'm eager to see the follow-up.

I'm currently planning something very similar at my company and any reference to how other smaller-scale companies handle this when scaling is invaluable. Are there any other blogs from other companies which have talked about similar things?


Not a direct answer to your question, but I maintain an HN-like aggregator of engineering blogs here: https://engineered.at

These types of posts pop up there pretty frequently


These need to be sorted by publishing time, not by your database entry time. You can tell because you've recently added Percona, and it's filling up the top with backdated stuff. I'd also have preferred just a minimalist interface. We don't need anymore voting/commenting stuff at this point.


Interesting feedback. Not sure how I could make the interface more minimal, it's pretty stark as it is.

The backdated stuff is weird I'll admit, but they also posted a bunch of things in the last couple days so could be screwing with the ranking algo. I'll check it out.


Just google for <Company> engineering blog. I know for sure Facebook and Google have good ones.


> The tool we used did not support using Hive as a backend, and given the speed of even relatively basic Hive queries, it wouldn’t make much sense to use that for reporting metrics.

Did Reddit try setting up Presto? Analytics, not just visualization, with Hive is really hard.


I've had a great experience with Hive!

One thing that has made me sad is that there is that Rails / Ruby has little to no support. Many of the repos are over four to five years old.


I think nowadays they use Microsoft PowerBI for dashboards/reporting




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: