Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Memory leaks detection paper co-authored by Netflix CEO Reed Hastings in 1992 [pdf] (stanford.edu)
203 points by mfiguiere on Dec 4, 2022 | hide | past | favorite | 95 comments


Hastings co-founded Pure Software in 1991. Their products were for software troubleshooting.

He co-founded Netflix in 1997.

https://en.wikipedia.org/wiki/Reed_Hastings


Pure was an amazing product. For C++ development, we relied on it for my team at SAIC, and when I moved to Angel Studios (we developed games and content for Nintendo, Sega, Disney) I immediately asked the owners to purchase a company license.

I saw Reed Hastings being interviewed on YouTube last night - fascinating how he has so many personal friendships with CEOs of competing companies.


Purify could find memory errors in just a few minutes that would take engineers days or weeks to track down. The dynamic instrumentation capability on existing binaries was huge. The first time I saw it in use was at Sybase. Running a demo version for an hour found enough problems that we instantly sprang for the license.


I used it at Sybase when I interned there in 2009. I've been fascinated by good developer tools ever since. It was such a joy to run, understand the reports and make fixes, and repeat.


Didn't IBM Rational buy Purify and make it 10x more expensive?


isn't that IBMs MO? Pretty sure rational was a startup at one point, too.


Purify was awesome back then, before Linux and the open source tools.

(I think Purify might've been the first phoning-home tool I saw, finding a way to email Pure if you exceeded your simultaneous licenses, but I might've misunderstood at the time. Around that time, it was normal for big-ticket Unix workstation software to have network license managers, which were tied with stale NFS mounts at our site as the most common cause of a fleet of expensive engineers on expensive workstations with expensive software all suddenly unable to work. :) )

A related awesome C development tool we used for discovering memory errors before then was the Saber-C debugger. https://www.softwarepreservation.org/projects/interactive_c/...


It would talk to your local license server, but it was definitely NOT communicating back to corporate.


What timeframe are you talking about?

I could be mistaken, since I no longer have the files from close to 30 years ago.

But I did think I saw a case in which it would try to email corporate about using too many seats.

It was memorable because that's the first time I saw that, and it seemed clever but invasive.


Late 90s/early 2000s. I definitely spent some time with the licensing code and I know it definitely did nothing like that then.


My experience must've been a little before that (I left that job in the mid-90s).

I could be mistaken. Or maybe the licensing code changed (due the Web explosion, or to evolution of a great product, or some other reason).


>Angel Studios

Was this the company that made Midtown Madness 2?


I think so, after I left.


I worked at Rational Software when we acquired Pure software. We sold tons of Purify+. The legend of course was the Hastings took the $ he made from selling Purify to Rational to start Netflix. So as a Rational employee, I had a contribution to Netflix's existence... Or so I like to tell myself ;).


Pure products were amazing. I always found the way Quantify represented profiling to be incredibly helpful back in the ancient days. I know there’s more advanced visualizations but I always found the logarithmically scaled directed edges in quantify to speak to me better than flame charts and other related techniques, I think because it represented the full execution graph rather than individual transactional paths. Anyway huge respect for Reed as an engineer and have always been impressed with both Netflix and the quality of the performance, debugging, and other efforts that have come out from them. He clearly values excellent engineering.


Do you have a picture of these "logarithmically scaled directed edges"?

I can imagine what you mean but a visual aid would be super useful.


I kagi’ed around and I can’t, sorry. The software is 30 some odd years old and was acquired by IBM and sublimated as all ibm acquisitions do. But my memory of it could well be approximated by using dot to create nodes for execution points with edges to calls that are rendered with a weight proportional to the log of the execution time. The nice thing is it allowed for recursive calls to be rendered as cycles, and there are a lot of non recursive cycles I found. It also did a reasonably good job of clustering so highly popular calls fanned into a sub graph but with each graph calling in clustered independently.


Thanks a lot!

I appreciated the description anyway!


No wonder Netflix has such a huge emphasis on engineering talent.


But Netflix was originally DVD rental. At that time it wasn't a tech company was it?


I follow Marc Randolph on Tiktok and he's spoken before that they knew things would eventually go online: https://www.tiktok.com/@marc_randolph/video/7079079298182909...


Is there anything to support that history? Documents from that 1997? Considering the company was founded by just trying to come up with random ideas I doubt they had too much foresight.


They were a tech company of their time: they had an excellent web site, search (and recommendation IIRC) etc, especially compared to most of their contemporaries.

Nowadays I wouldn't consider anyone in the DVD-renting business to be a tech company. I would still consider Netflix a tech business under their current business model.

Note that Netflix is relatively small and not really in control of their fate; they were presumably included in the "FAANG" acronym to make it fun. Now that two of those companies have changed their name it doesn't even fit (A, A, A, M and...N? Not the other M?)


Netflix was included in FAANG because it was a high growth tech stock, growing faster than all the others when it was included. It was the fastest growing stock on the market multiple times in the 2010s and was the fastest growing stock for the entire decade.


"Nowadays I wouldn't consider anyone in the DVD-renting business to be a tech company."

Netflix still rents DVDs.[1]

I actually much prefer their DVD service, because their selection of movies on DVD is infinitely superior to their streaming service, whose selection is absolutely atrocious.

Netflix's streaming service might have world-class delivery, but what they deliver is crap.

[1] - https://dvd.netflix.com/


> (A, A, A, M and...N? Not the other M?)

Well now we can say "MAAAAN it went downhill"


Recommendation accuracy was much more important for DVD rental, because of the wider selection of content and the slower feedback loop. So a good recommendation algorithm was a core competency for the business.


And when seeing his internal recommendation algorithm team hitting the wall in recommendation accuracy, Hastings boldly created a 1M$ "Netflix Prize" competition to create a better recommendation algorithm [0]. This boosted research efforts in the field and earned him and Netflix respect in ML academic circles.

[0] https://en.wikipedia.org/wiki/Netflix_Prize


Netflix was originally intended as a streaming company. The DVD rentals were a strategy to solve some non-technical problems with the business while the infrastructure for streaming caught up.


That's fascinating and an incredible example of long term planning. Do you have a source for this?

edit: A very cursory check of the Wikipedia page and https://www.vox.com/2017/9/13/16288364/streampunks-book-exce... suggest that it probably isn't true that Netflix was "originally intended as a streaming company" considering at it's founding online streaming was still far from feasible, but I'm open to evidence to the contrary.


My source is that I worked at Netflix in the early days, it was always part of the strategy. We were discussing streaming internally almost a decade before they actually delivered it.

While streaming technology was early and teetering on the edge of viability, the real challenge was maneuvering the studios to license content that they didn't want to license because they wanted to retain absolute control. They were very averse to streaming for multiple reasons. The gambit of using DVDs in a series of Hobson's choices for the studios almost worked but ultimately failed at the late stages for an important reason that was difficult to bridge.


From Netflix's IPO prospectus, dated May 22, 2002:

> We currently provide titles to our subscribers on DVD only. However, we continue to monitor additional delivery technologies and, when appropriate, believe that we are well-positioned to offer digital distribution and additional delivery options to our subscribers.

And further down, a bullet point in their growth strategy section:

> Implementing Digital Delivery. We continuously monitor the development of additional digital distribution technologies. Historically, new technologies, including the VCR and more recently the DVD player, have led to the creation of additional distribution channels for filmed entertainment. We intend to utilize our strong relationships with the studios to obtain rights to acquire and deliver filmed entertainment through emerging digital distribution platforms as they become economically, commercially and technologically viable for those subscribers who prefer digital distribution.

And later, regarding competition:

> We believe that our strategy of developing a large and growing subscriber base and our ability to personalize our library to each subscriber by leveraging our extensive database of user preferences positions us favorably to provide digital distribution of filmed entertainment as that market develops.

https://www.sec.gov/Archives/edgar/data/1065280/000101287002...

Streaming on the internet was already exploding by 1997; relatively speaking, of course. I distinctly remember first watching South Park--myself and several other students, working in the computer lab technical assistance office for work-study, huddled around a Mac. This likely would have been circa 1998-1999, if not late 1997 when it debuted. (In fact, I don't think I ever watched South Park on cable television until many years later, while staying at hotels.) Around that time I also remember a major, well-funded streaming website that was supposed to revolutionize Indie film distribution. Anybody working in computing or the film industry knew where distribution was heading--the only question was when all the pieces would come together to make it viable, broadband adoption being the most important. (Not sure if a majority of American households were yet "online", but if not that was already a rapidly approaching inevitability.) There were many ventures which dove in too early.

While I wasn't paying attention to (or even knew much about) Netflix before it's IPO, certainly by then it was well recognized--and recognized as brilliant--that Netflix's strategy was to build relationships with studios and rights holders on the one hand, and customers on the other, precisely to position themselves to dominate digital distribution when the time came. Online DVD rentals was the ramp, permitting them to build momentum so they'd have a head start ahead of anybody else in the digital distribution space. And it worked spectacularly; it took nearly a decade, if not more, for anybody to even come close to Netflix's lead. (Wikipedia says Netflix began streaming in 2007, same year as Hulu, but Netflix was still dominating until only a few years ago.)


Netflix was founded in 1997. You're talking about 2002.


are you a young un? dont you guys remember real audio? i remember streaming the NBA playoffs over dialup all the way in Australia. Video streaming was already a thing, it just didn't have a killer library.


Do you consider Amazon a tech company prior to AWS product launch?


Amazon made web-based book selling mainstream. Emphasis on web. Of course it was a tech company.


And the web based dvd rental then ? I mean, they were doing exactly the same as Amazon, why distinguish?


Banks have online customer sites. Are they tech companies?


If they did it when Amazon started their web store they would be.

How do you think Amazon infra gave birth to AWS? Does that sound to you like something a retail company would do?


Banks are tech companies, the business leadership just don't know it.


The business leadership of some banks do in fact know it.


It was an online website right around the dot com bubble.

It definitely was a tech company by that time’s standards.

Nowadays every company has an online store, but that was not the case when Amazon was selling books online that you usually had to go to a store for.


> It was an online website right around the dot com bubble.

Much sooner than that.

Amazon started in 1995. Not a lot of ecommerce websites existed then. One the earlier ones is Powell's Books which started selling in 1993 via telnet, in 1994 via web.


Tech company did and still does to an extent refer to "online" first, vs brick and mortar. In the context of the dot com boom, they were definitely tech


> At that time it wasn't a tech company was it?

At that time you would have to physically shop at Blockbuster to pick out a movie to watch.

With Netflix you picked what you wanted to watch online, maintained a queue and they would automatically send you DVDs through the mail.

It was absolutely a tech company since its inception.


it absolutely was a tech company. the 'Cinematch' recommender , replaced eventually, was one of the differences brought-up in the netflix/blockbuster lawsuits.


Sending dvds all over the USA with such quick turnaround was more impressive to me than streaming video.


I have a copy of the Pure Software Engineering Handbook which was created for internal use only. It's amazingly comprehensive and well-organized. I've never seen any kind of onboarding that comes close.


Having met him before, his clarity of thought just comes across in this paper as well. Some people are just wired with that talent that makes me so jealous. Reed is one of those people with high EQ and IQ (with the ability of handle his inner Elon diplomatically)


Curious what "inner Elon" implies


The little voice that says something is only worth doing if it boosts your ego somehow.


[flagged]


You can't even mention Musk's name anymore without certain sections of our society spewing absolute venom. Really something to behold.


You're citing an example of high EQ (unless you think he really means it, which is doubtful). It won't hurt him in any way to say that and he gets credit with Elon and his fans.


Of course he means it. And praising Elon, who is controversial, could absolutely hurt him.

Also, EQ isn't the same as machiavellian scheming to further your self interest.


Reed Hastings knows too much about creativity to believe that. I don't condone this kind of fake praise, but it is easy, costless and will make you a "good guy" to a little demographic that can be troublesome if aimed at you. Even if the public all sees through Elon and associating with him becomes anathema, it won't bite him at all.

I'm reminded of some commercials that kept playing on YouTube a few years ago where John Mayer was calling Kanye the most creative genius he knew. I'm sure he didn't mean it, Kanye is not someone you want to endorse now, and no one cares that Mayer said that.


You don't like Elon and expect people you hold in esteem to agree even when they state the opposite. I know this is mind-blowing but there exist smart, talented people who disagree with you. And not just about this.

Also, the claim that praising Elon is a safer play than condemning him is not obviously true. There are plenty of Elon haters out there who could be "troublsome". Like you and the other guy who lost his shit in this thread.


I don't really dislike Elon. I think he doesn't do things from a bad place but possibly because he's not neurotypical he's been sucked into spending his energy pleasing his craziest fans.

We'll have to disagree on Hastings quotes, to me they seem clearly a way to speak nicely about Musk with a wink to those who don't idolize him.


"Brave" and "creative" do not always have positive connotations. They may be be used to diplomatically describe a reckless confabulator


There's no reasonable way to interpret his comments that way. You're projecting your opinion onto someone who doesn't share it.


> You're projecting your opinion onto someone who doesn't share it.

Two things:

1. interfacing with politicians (I'm including both executives and senators in this group from personal experience) has taught me that many are exceptionally diligent with the words they use. "Brave" and "creative" are neutral adjectives politically even if they have positive connotations interpersonally. If a politician doesn't say something exclusively, denotationally positive, it's almost always a deliberate hedge. I'm not saying that's the case with Reed, but I'd bet a dollar that it is, because most people who's impressions he'd likely care about (execs, congress, etc) use and read the same language.

2. Unless you spoke with him and heard otherwise, aren't you likewise projecting?


No, I'm not projecting, I'm taking what he said at face value...

He didn't just say "brave" and "creative". He said more than that, including some semi-negative comments about Elon's general behavior, but the thrust of his comments is very positive.

It's absolutely hilarious to see people who hate Elon react this way. You can't compute someone you respect disageeeing with you.


> It's absolutely hilarious to see people who hate Elon react this way. You can't compute someone you respect disageeeing with you.

The fact that you jumped to this conclusion when I said nothing to indicate how I feel about Elon in my only comment on this entire thread says a lot about where you're coming from.

Anyhow, the fact that his comments were:

"[I am] 100% convinced that he is trying to help the world in all of his endeavors"

"He's trying to help the world on that front because he believes in free speech and its power for democracy,"

and

"how he goes about it is not how I would do it."

should give you a pretty clear read about where he lands right now:

1. Reed at least publicly believes Elon is coming from a good place

2. Reed disagrees with the methods Elon's using.

3. "Brave" and "most creative" contextualized by the statements above are given a neutral tone and meaning.

Tldr: Reed basically said "he's coming from a good place but his approaches are bad"


I read you correctly. Congratulations on looking up the full comments (or at least more of them) but you're still reading it wrong because you can't handle the fact that someone smart disagrees with you.


k


Read the rest of the interview.

  Hastings added that he is “100% convinced” that Musk is “trying to help the world
  in all his endeavors,” and added that he was “excited” about Musk’s Twitter
  acquisition. Musk “just spent all this money for democracy and society to have a
  more open platform — and I am sympathetic to that,” Hastings said.


I may be in an overly cynical mood, but the word "trying" in that phrase changes the color quite a bit, and frankly, the word didn't need to be there. Carefully parsed, the addition is not a ringing endorsement as it appears on the surface.


Okay I'll bite: How does glorifying Nazis count as trying to help the world?


What's worse? Hating people you have never met or idolizing people you have never met? Maybe spend your energy elsewhere instead of getting so worked up that you have to start calling people names.


All that said, I’m sure he’s a great engineer.


[flagged]


What's worse is whatever you're doing here. It's a really ugly look, some may say.


Vilifying Nazi worship is an ugly look? Okay then.


Would you please stop? No matter how right you are, fulmination is not what this site is for (and destroys what it is for).

https://news.ycombinator.com/newsguidelines.html

Edit: We've had to ask you this quite a few times already:

https://news.ycombinator.com/item?id=32289007 (July 2022)

https://news.ycombinator.com/item?id=32146400 (July 2022)

https://news.ycombinator.com/item?id=22660362 (March 2020)

https://news.ycombinator.com/item?id=19438509 (March 2019)

https://news.ycombinator.com/item?id=19070048 (Feb 2019)


Consider that your perspective on the man is hopelessly warped by the media frenzy surrounding him. Sadly, you are a victim of it just as much as he is.


[flagged]


Who's envious?


The one with "inferior" in their handle.


And what am I supposed to be envious of?


Back in the day Reed told me that he added the leak detection to Purify v1 towards the end of the dev cycle and was very surprised that it wound up being one of their biggest selling points


Purify was a fantastic tool. I was so surprised to find out that he went on to found Netflix.


I remember using that and libefence in the 90s to find memory leaks in my MUD. Purify was better. Easier to use and found more stuff. It is 2022, leaking memory still a problem.


Recently I was thinking about how the Netflix software strategy of hiring only experienced engineers [0] (and paying them really well), is pretty beneficial for almost any tech team/company.

Things like the CS experience that Reed Hastings has, seems like the key driver here.

[0] https://www.teamblind.com/post/Why-Netflix-only-hires-senior...


I hadn't realized Pure and Netflix had the same founder; Purify was an amazing tool, which we had a license for HP-UX at uni. It was the kind of software where you knew you'd ask for it to be bought by any future employer.


what's the difference between this and valgrind?


A major difference is that purify is a binary transformation performed before or after linking. Valgrind/memcheck is a virtual machine that jit-compiles executable code and adds its instrumentation in the process. So, with purify, your linked executable includes the checks. With valgrind, the unmodified executable can run with or without checks.


And for a more practical comparison: https://valgrind.org/gallery/survey_03/q4.html

The takeout is: valgrind is free and doesn't require recompilation, purify has a more complete UI, they both have similar performance impact and bug-finding abilities.


Also, Purify runs on Windows.


I used Purify back in the day, and while it worked well, getting it working was a royal PITA. IIRC it was necessary to recompile EVERYTHING with Purify.

Anyway, I would think the gold standard these days is clang's Address Sanitizer (https://github.com/google/sanitizers/wiki/AddressSanitizer). clang/ASAN also runs on Mac in addition to Linux.


Having never used Purify back in the day, I really wonder, do you think address sanitizer is an adequate or better replacement for Purify?


It's been a while, but IIRC ASAN catches most/all of the memory-related errors that Purify did.

Purify did have a debugger-like GUI that would break on errors, which could be handy.

valgrind can also do most of the same checks, but is much slower.

On my team we use both


[flagged]


That's not how it usually happens in my experience. In this specific case, I worked at Pure Atria back in the day and it was my impression that Reed had some hacker chops.


Have no doubt in this case - a sibling comment explains that Netflix was founded 5 years after this paper.

https://news.ycombinator.com/item?id=33859953


> EDIT: Wow. OK, I guess I'm the only one in the world who's ever seen that happen.

It happens regularly in larger companies. Most patents that I have also have others on them, because secondary people had to approve my applications. That's just how these things work. The only exceptions to this are those patent applications that I approved for others.

Remember, work on a patent may be more complex than the work on a corresponding invention. Also, the patent must be useful to the company in order for them to cover the bill. Taking a single employee's word for that is not enough.


The other people are on there because by law they have to be. Anyone involved in the invention in any way has to be on the patent or it can be found invalid.


That's the legal reason but not the business reason, which is the actual driver and what causes the legality to occur.


It’s worth questioning but he wasn’t a big deal in the early 90s and if Wikipedia is correct he finished a Stanford CS MS in B 1988. This seems very much in character.


It can invalidate a patent if an inventor is missing or one is named that did not actually contribute to the invention.

https://ipwatchdog.com/2020/10/21/getting-patent-devastating...

This is different from scientific papers, where it is poor taste to add people that did not actually contribute anything, but has no legal implications.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: