This is a very rewarding thing to do. I recommend it to everyone. You don't necessarily even need to use AI.
I filter HN with http://hacker-newspaper.gilesb.com/, which pulls RSS, filters it, and reformats it on an hourly cron job. I mainly did it for the typography -- I disagree with just about every visual design decision on Hacker News -- but added very primitive filtering after the fact. I throw out any story from TechCrunch, Zed Shaw, Steve Yegge, and Jeff Atwood, because I just got tired of them, and any story with "YC" in it, too, because I got tired of seeing job ads for Y Combinator startups. (In fact it was the job ad for a Curebits marketing manager right after their scandal that did it.)
When Apple launched the iPad, I went in and added a simple regex to filter out any story about it. Hacker News is a great source for skimming but occasionally gets fixated on topics. I get like a hundred uniques a day so it's not exactly a huge hit, but I've thought about making a commercial version with customization. It got featured on Mashable and somebody created an iPad app which looked very, VERY similar, which I'm going to take as validating my design. But whether or not I ever startupify it, anyone who wants a customized version can just fork the project, deploy their own version in like ten or twenty minutes, and tweak regexes to their heart's content. It's on GitHub (https://github.com/gilesbowkett/hacker_newspaper) and only requires the most basic proficiency with cron, ruby, and python.
I also want to add comment-scanning. Right now I don't use comment links at all. The code extracts them but then simply throws them away. I don't want to add comment links back in unless I can also set it up to alert me if the comments thread contains comments from raganwald, patio11, jashkenas, amyhoy, etc -- basically automated comment elitism. I'm not trying to be a dick with that, I'm just a busy dude.
Anyway, when I set out to do this, I planned on doing a bunch of Bayesian whatnot, but I found that I got most of the way there just tweaking regexes occasionally. Likewise there's a lot of rough edges I could clean up, e.g., text encoding is a bit of a mess, and summarization in the style of http://tldr.it/ would make it way more useful.
But I recommend it because making deliberate decisions about what info you want to get from HN makes it a lot less like watching TV and a lot more like doing actual research into topics which interest you. It's surprising how much more enjoyable HN becomes when viewed through a customized filter.
A word of advice about using the comment links in the hn's rss feed, from personal experience: try not to scrape them in an interval. I mistakably ran a piece of code that went through 7~10 different posts in a short period of time (30 secs) and my server was banned immediately.
I filter HN with http://hacker-newspaper.gilesb.com/, which pulls RSS, filters it, and reformats it on an hourly cron job. I mainly did it for the typography -- I disagree with just about every visual design decision on Hacker News -- but added very primitive filtering after the fact. I throw out any story from TechCrunch, Zed Shaw, Steve Yegge, and Jeff Atwood, because I just got tired of them, and any story with "YC" in it, too, because I got tired of seeing job ads for Y Combinator startups. (In fact it was the job ad for a Curebits marketing manager right after their scandal that did it.)
When Apple launched the iPad, I went in and added a simple regex to filter out any story about it. Hacker News is a great source for skimming but occasionally gets fixated on topics. I get like a hundred uniques a day so it's not exactly a huge hit, but I've thought about making a commercial version with customization. It got featured on Mashable and somebody created an iPad app which looked very, VERY similar, which I'm going to take as validating my design. But whether or not I ever startupify it, anyone who wants a customized version can just fork the project, deploy their own version in like ten or twenty minutes, and tweak regexes to their heart's content. It's on GitHub (https://github.com/gilesbowkett/hacker_newspaper) and only requires the most basic proficiency with cron, ruby, and python.
I also want to add comment-scanning. Right now I don't use comment links at all. The code extracts them but then simply throws them away. I don't want to add comment links back in unless I can also set it up to alert me if the comments thread contains comments from raganwald, patio11, jashkenas, amyhoy, etc -- basically automated comment elitism. I'm not trying to be a dick with that, I'm just a busy dude.
Anyway, when I set out to do this, I planned on doing a bunch of Bayesian whatnot, but I found that I got most of the way there just tweaking regexes occasionally. Likewise there's a lot of rough edges I could clean up, e.g., text encoding is a bit of a mess, and summarization in the style of http://tldr.it/ would make it way more useful.
But I recommend it because making deliberate decisions about what info you want to get from HN makes it a lot less like watching TV and a lot more like doing actual research into topics which interest you. It's surprising how much more enjoyable HN becomes when viewed through a customized filter.