A brilliant anecdote, yet I am curious how you can still believe in the dreaded "lisp curse," after experiencing so much success with it.
Is it an attribute of Lisp that there exists a mythical lone-wolf that seeks to produce software so pure and elegant that no mere mortal can possibly understand it? Is it an attribute of Lisp that this lone-wolf, like a by-gone action hero, prefers to work alone?
Lisp, like any sufficiently useful language, simply gives you enough rope. It's good, sturdy, and well-tested rope. You could probably climb a mountain with it. Or hang yourself. Your pick. The rope has very little to do with it.
I experienced personal success with it. It's not that Lisp causes people to not be able to work together. The causality runs in the other direction. Lisp tends to attract people who are antisocial by nature, because if they use Lisp they can be vastly more productive as individuals than if they don't. But then you get a community full of antisocial people, and they tend to drive away the team players. So the team players go off and use C/Perl/Ruby/Python/whatever because those communities are full of other team players. As a result, the whole becomes greater than the sum of the parts, and the team players win, even though as individuals they may be generally less productive/smart/whatever.
On those rare occasions when Lisp programmers manage to all row together in the same direction they can produce some truly spectacular results (Viaweb, ITA, RAX). But those seem to be the exceptions.
Lisp has this convenient nature that it is better than every language, yet under its own weight it has nothing to show for it.
Maybe I can start a similar myth with Logo. It requires no proof. Simply one or two anecdotes, great hubris, and the ability to say with a straight face that the reason why everyone who uses it isn't productive is because we're geniuses. Anyone else in?
> yet under its own weight it has nothing to show for it.
That's factually incorrect. There's plenty to show for it, you just have to scale to the size of humanity's computing concerns from four or five decades ago.
Moreover, in no field of creative human endeavor have popularity and worthiness been correlated. You think all those billboard top 10 acts came from Juliard?
It's not. Although I'll be the first to admit that it comes off as trollish. It's just I see these pronouncements about a language, which frankly, I don't think can be backed up. I like Lisp as a language. I really do. But to me that's not sufficient to annoint it a super language.
The evidence of its superiority is lacking. The one thing that languages generate is work product, programs/libraries/etc...
In contrast, look at something like VB. An oft-scorned language, but a language that you can walk into any Fortune 500 company and can probably find 10M LOC across the enterprise used for business critical purposes. And further, the interesting thing about VB is that you probably couldn't have replaced the language with any other language and had the same productivity.
Now of course this last assertion is hard to prove. But at least we have existence. At least we can say, "if there was a better solution than VB, no one used it". And it was rediscovered by every enterprise in the world.
Again, I'm not saying Lisp is a bad language. But to the best of my ability it seems to be about on par with other modern languages (and that in itself might be compliment enough for some, given its vintage). I see a couple of truly noteworthy Lisp projects, a lot of middling ones, and that's it. About what I'd expect for a language with a user base of its size.
Over the past 40 years there hasn't been any mind-blowing Lisp projects. And there's nothing wrong with that, I wouldn't necessarily expect it, except that its supposed to be a super language. Not only a super language, but a language practiced by the brightest amongst us. And by accounts of some evangelists, a toolchain that is magnitudes better than that available for other languages.
Yet as a result I typically get 3 projects noted to me when we talk about notable Lisp projects. Emacs, ViaWeb, and ITA. Three great projects. But if someone said that they were written in C and not Lisp, everyone would believe it. And when people go back and talk about important/influential software in history, Emacs is likely the only one to be mentioned. And of course there's been tons of other very important SW that has been written in languages other than Lisp, from Tex, to Mosaic, to Lotus 123, to gcc, to iOS, to Google, and so on.
The worse is better, Lisp super-loner, rationalization seems like a pity party. And what makes it sad is that it's simply not necessary. There's no shame in liking a language for characteristics you find appealing. But don't try to sell it as some super language. Because when you do people will say that the proof of the pudding is in the tasting. And the tasting are the programs. If you want to show that Lisp can do what others only dream of, show us the programs that make us drop our jaws. Show us the programs that make us rush to relearn Common Lisp. RoR didn't catch on because it had a sexy name. It caught on because the pudding was pretty tasty -- although a bit cloying for my taste.
Moreover, in no field of creative human endeavor have popularity and worthiness been correlated. You think all those billboard top 10 acts came from Juliard?
Are you arguing that for music that college serves little purpose? That we're entering some type of music education bubble. :-)
"I'll be the first to admit that it comes off as trollish. It's just I
see these pronouncements about a language that make claims, which
frankly I don't think can be backed up. I like Lisp as a language. I
really do. But to me that's not sufficient to annoint it a super
language."
The post we are commenting on was by a person who was not making a
claim, but relaying an experience. I will paraphrase three arguments:
the parent post blog link was saying "look, I'm not an amazing
programmer, but in my experience, armed with Common Lisp, I can
approximate what I see ur-programmers do with other languages at
Google." He in turn was replying to a person saying "I don't think
Lisp makes you more productive." The "Lisp is not really a super
language, it has nothing to show for itself, and I could make similar
claims about logo." thing got interjected by you, I think.
"Moreover, in no field of creative human endeavor have popularity and
worthiness been correlated. You think all those billboard top 10 acts
came from Juliard?"
"Are you arguing that for music that college serves little purpose?
That we're entering some type of music education bubble. :-)"
No, I think that's what you're arguing. I'm saying Lisp is worthy,
and should not be judged by its popularity, because the worthiness is
what reallly matters. You're arguing it has "nothing to show for
itself". I'm sure with a Sun Java marketing budget and the right
people we could make Lisp as popular as we wanted. We could have
whitepapers and case studies to "show" as much as you like. I'm only
concerned with popularity when people dismiss my arguments about
worthiness the minute they assess the popularity. I have the same
problem when I try to tell them about Sun Ra or the Residents.
So all that being said, to address your actual point (however
disjoint from the OP it might be), nobody's claiming it's a "super
language", just often better for the people who commit to using it
despite it's unpopularity. I can back up my claims of betterness with
arguments about worthiness, but not about popularity, but then I don't
think the later kind of argument matters in the first place.
The post we are commenting on was by a person who was not making a claim, but relaying an experience.
He was not only relaying an experience, as he cites the Lisp Curse. See:
"I do believe in the Lisp curse that the power of the language is in some respects self-undermining because it empowers the individual and so tends to attract people who don't work well in teams."
And I'm sure you've read the Lisp Curse.
His essay argues that he's really not a great programmer, but Lisp gets him to principal engineer! He further implies that Lisp really is something beyond other languages with his statement about the planner code (of code, he never considers that porting often fails even within the same language family for a variety of reasons).
I'm sure with a Sun Java marketing budget and the right pepole we could make Lisp as popular as we wanted.
How do you explain C, C++, Python, Ruby, Perl, Javascript, and PHP? None had very extensive marketing budgets.
So all that being said, to address your actual point (however disjoint from the OP it might be), nobody's claiming it's a "super language", just often better for the people who commit to using it despite it's unpopularity. I can back up my claims of betterness with arguments about worthiness, but not about popularity, but then I don't think the later kind of argument matters in the first place.
I don't know what your metric of worthiness is. Maybe its the existence of some esoteric feature. You say I talk about popularity, but I only do so to ask for an example of the "worthiness" you speak of. As I said before the proof of the pudding is in the tasting. If you tell me you have some great pudding, but it tastes like cow dung, I won't be impressed regardless of how many fine ingredients you use and advanced cooking methods employed. It still tastes like cow dung. And I'm not saying that Lisp is cow dung, but I am saying that fancy ingredients and world class ovens don't impress me if you're making the same McDonald's sandwich as everyone else.
And when you start creating great pudding, you won't need the marketing budget.
"How do you explain C, C++, Python, Ruby, Perl, Javascript, and PHP? None had very extensive marketing budgets."
I think a lot of these have larger marketing budgets than you imagine. I can't fathom the amount of money Microsoft spends promoting its C++ language tools, for example.
Regardless of whether languages other than Java become popular or not because of marketing, I still maintain that popularity and worthiness are not correlated. The majority of people may not eat cow dung, but they certainly don't eat really great pudding. They eat the same so-so pudding everyone else does, and most of them don't realize or don't care that it could be better. You will have nothing to "show" for coming up with a better pudding recipe unless you spend money on advertising, manufacturing, distribution, etc..
As for an example of worthiness, Lisp's advantages have been detailed one metric kerjillion times elsewhere. Macros, conditions, and the MOP are the usual suspects in the case of Common Lisp.
I think his point was not "why is Lisp not popular", but if it is so powerful, why has it not become more mainstream based upon that? If a language is truly as powerful as its advocates assert, should it need a marketing budget? Ruby, via Ruby on Rails, is famous, and has been used on lots of highly visible (successful?) projects, with no traditional marketing budget.
I think he's asking, if Lisp is so awesome, why doesn't someone ever do something awesome with it? It seems like a fair question.
I am sure there are other success stories which are not heard. At Cleartrip we had a largish team of Common Lisp programers. We became from new entrant to a number 2 position with software written in Common Lisp. So many myths about lisp were proven plain wrong.
* you need to be brilliant to write lisp - none of us were "super" lisp programers with years of experience. We had common sense and nice simple code.
* hard to learn lisp - not any more than, say, ruby. Out of the 10 odd lispers we had only 2 had CL exposure. The rest learned on the go. Whichever language / platform you use you have to train your team if they are fresh.
* Lisp is slow - haha.
* Lisp software is hard to run - haha again. we had months of uptime and we could patch our servers on the go.
* Lispers are pricks - We were all normal people who worked together well. Lispers, Ruby guys, JS guys - all.
But still everything was not rosy.
* When we started, documentation was a huge problem. We would never have got approved except for Edi, who's efforts were an inspiration and a basis of our stuff.
* Library support was sparse. Now it is a lot better. But the community is small. Not that they are all pricks, but they are stretched too thin. If I have seen anything, if they use lisp for work, most of them have to fight daily battles against the anti-lisp crowd which will not look rationally at any good work you have done.
* Library management - now much better because of quicklisp (thanks Zach)
* For most people lack of a usual ORM and a MVC framework was unthinkable. CLSQL has been a pain point.
Programming in the Lisp environment, with the REPL, is a pleasure. I have had to do php after than and O-M-G - its horrible.
All I will say to other fellow Lispers - The naysayers are always present. Trust your instinct. You will enjoy life better if you are able to love what you do.
(Cleartrip does not run on CL now. Java, I think. Because of reasons other than technical IMHO.)
Which is why I have high hopes for Racket and Clojure - less NIH syndrome, more libraries, friendlier communities, and a healthy respect for good ideas from a wide variety of programming languages, not just Lisp.
Have you ever found it difficult to understand someone else's lisp project, because they have customized the language so much? (or vice versa, has a fellow lisper found your code hard to understand)
I wonder if the flexibility and power of lisp might work against co-operation in this way too.
I've witnessed confusion caused by operator-overloading in C++ (I heard that that's common); lisp is more customizable.
> Have you ever found it difficult to understand someone else's lisp project, because they have customized the language so much?
Yes. But that is not a problem unique to Lisp. It's possible to write unreadable code in any language. And in general Lisp's introspection capabilities and macroexpand make it easier to reverse-engineer things that in other languages.
opps, I what you mean; I actually meant well-written lisp.
Have you found that hard to understand, when it was heavily customized? I agree that tracing through complex code is taxing in any language; and it probably becomes much more important what concepts the author has chosen (and things like coupling/cohesion), than the language.
i.e. That the problem wasn't that the code was badly written, but it was because they had "customized the language so much". Extensive customization i how you're supposed to use lisp; it's an important source of its power. (The C++ example I mentioned was well-written.)
I can imagine it going either way (e.g. that lisp's customizations often aid clarity, by reducing code size); so I'm interested to hear someone's experience of how it actually goes in practice.
I have seen that happen before, but only when the domain was inherently complex. An example I actually committed; for a variety of reasons I needed a continuation driven LL(1) parser for my PhD thesis, and I needed it to be in Java. I wrote this as a Scheme program that output Java code. I tried hard to make things understandable (since I had to debug all of it!) but the Java code was basically incomprehensible and the Scheme program that generated it required some relatively sophisticated understandings of parser theory to understand the notation I was using.
A good theoretical example is: imagine trying to understand what a CPS driven compiler is doing with no understanding of CPS. That would be pretty hard in any language, and it's actually better in Lisp because you can express things closer to the theory.
If there is one feature in lisp that might fit that category it would be macros. They are great for single programmer productivity while I have seen people who worked in large teams complain about lisp macros. However like every other feature macros could be minimized by the companies coding style handbook.
Is it an attribute of Lisp that there exists a mythical lone-wolf that seeks to produce software so pure and elegant that no mere mortal can possibly understand it? Is it an attribute of Lisp that this lone-wolf, like a by-gone action hero, prefers to work alone?
Lisp, like any sufficiently useful language, simply gives you enough rope. It's good, sturdy, and well-tested rope. You could probably climb a mountain with it. Or hang yourself. Your pick. The rope has very little to do with it.