LLVM IR = "Low Level Virtual Machine" (originally written to be a replacement for the existing code generator in the GCC stack) "Intermediate Representation" (virtual machine language, prior to being crunched into actual machine code).
I hope I'm not the only one here upvote your comment simply because you decompressed IR into Intermediate Representation.
I like to read about LLVM and some of the things that are a bit (ok, in a lot of cases, way) above what I do day-to-day, except that it's sometimes taken for granted that the readers know the acronym/reference. I read the whole article and I understood what it was referring to, but for the life of me I couldn't land on the two words because IR in my world is always infrared.
These are exactly the reasons why I am very skeptical about pNaCl. I've done some work with LLVM and I've been stuck on some of these points: the IR is target-dependent, the function call conventions are basically those of C, the optimization passes and the code generator are extremely slow when compared to an ad-hoc JIT (this would mean terrible startup time for web applications). And, furthermore, the size of the bitcode is still enormous, comparable to the generated machine code or even worse (think about the size of C++ binaries).
I still think that LLVM is an incredibly great project, it brought compiler infrastructure to 2010s. But it is designed to be a static compiler framework, it hardly fits other purposes (think Unladen Swallow, don't know about Rubinius).
LLVM is being used as a JIT in Open Shading Language, a DSL for advanced 3D renderers. The JIT + resulting code was so fast, the OSL team dropped their batch rendering API altogether. (The resulting shaders are 25% faster than the hand-coded C language shaders they replaced.)
Currently, the LLVM JIT-powered OSL is now responsible for 100% of the shading duties at Sony Pictures Imageworks. I'd say that's a resounding success for LLVM as a JIT.
I'm not sure how that refutes the point. OSL is a shading language, that's an environment dominated by runtime execution time, not startup latency. So it might be a "JIT" in that there's no stored binary, but it's performance criteria are much (much!) closer to those of gcc or clang than to, say, a JVM or script interpreter.
And, like the post said: LLVM IR is a compiler IR. It's a great fit here.
>> [LLVM] is designed to be a static compiler framework, it hardly fits other purposes [...]
I presented a ready example where LLVM was being used not as a "static compiler framework", and was being used for "other purposes". If that's not an outright refutation, it's at least a useful data point.
Right, and my point was that that ready example was as close to a "static compiler framework" as you can get without actually being one. It's an exception-that-proves-the-rule case.
OSL shaders are specialized dynamically at runtime and then JIT'd using LLVM. That's about as far from a static compiler framework like GCC or Clang as you can get, from my perspective.
In addition, fast JIT speed was a very high priority/need and the OSL team spent a few months carefully tuning the LLVM passes to give fast code at a low JIT cost. (All of this was done and discussed publicly; feel free to grep the mailing list.)
Would you mind explaining how OSL is as close to something like Clang or GCC as you can get? I just don't see it.
JIT compilers have to make a tradeoff between time spent on analysis and time spent on execution; this is what makes them hard. It makes them hard because it means they need different levels of optimization, depending on how often a piece of code is going to get executed. If the code is only going to get run once or twice, it usually doesn't make sense to translate it from an efficient interpreter encoding (which LLVM does not have); if it's going to run a few hundred times, it can get a little bit more analysis, while if it's going to be the core of a loop, it makes a lot of sense to run analysis that may take multiple milliseconds.
Using a static list of analyses that are always run before execution would be a static compiler approach, not a JIT approach. A JIT will generally profile the code, optimize it when it gets hot, and deoptimize it when assumptions made during optimization no longer hold (e.g. virtual calls in JVM being optimized to static calls because only one definition of a virtual method exists, but subsequently a new class is loaded that overrides that method). All this dynamic runtime modification of the code (not just initial compilation, but modification of existing, executing code) is where JIT technology is distinct from static compiler technology.
> Would you mind explaining how OSL is as close to something like Clang or GCC as you can get? I just don't see it.
It seems that Open Shading Language is not really Just-in-Time (JIT) compiled. It's compiled from LLVM IR to native machine code Ahead-of-Time (AOT) but at runtime and then executed. This is actually not very far from a static compiler, the only difference is that the target architecture is known at runtime and the LLVM IR is compiled to machine code using that information.
What is the difference of JIT and AOT here is that in a JIT situation, there is some form of an interpreter that is executing byte code of some kind (probably not LLVM IR). When this interpreter reaches a loop of some kind, it will attempt to compile it (from bytecode to LLVM IR and finally from LLVM IR to native code). The interpreter then calls the compiled code, which will run for as long as possible and finally return control to the interpreter which will continue interpreting until it finds another opporturnity for JIT'ing. A JIT compiler is typically employed when the original source language cannot be compiled statically to machine code, because of e.g. dynamic typing.
So from what I can tell, OSL is just a static ahead of time compiler, with the final stage of compilation taking place at runtime.
Please tell me if some of my background facts were incorrect (in particular about OSL).
I think this is correct. OSL is using LLVM in an AOT fashion, though a lot of runtime code generation and specialization is being done just prior to running the AOT LLVM JIT.
As opposed to something like, say, Google v8, which is using runtime feedback to make hot code paths fast (and to remove dynamism when it can be shown to be safe).
I guess I wasn't aware that people weren't including dynamic code generation and AOT compilation in the "JIT" category. To me JIT meant generating machine code "at runtime", and both OSL and v8 would be at opposite ends of the JIT "at runtime" code generation/compilation spectrum -- OSL on the AOT side and v8 on the keep-running-the-compiler side.
In src/liboslexec/llvm_instance.cpp, OSL uses a custom list of 13 LLVM optimizer passes (plus implicit passes pulled in as dependencies) -- less than what Clang uses at -O2 for C, but still a fair amount. Then it runs the full LLVM "-O2" backend. Even if the compilation happens "dynamically" from the perspective of the application, OSL is using many compiler features more commonly associated with "static" compilation.
JIT speed is relative. Shader programs are often executed many many many times, so OSL can likely afford to make different tradeoffs than many other JIT-using applications.
Shaders compile once and execute millions of times. Most code on the Web code compiles once and executes zero, one, or a small number of times. He's saying LLVM is less suited to the latter case because it compiles slowly (though the resulting code is fast).
In the case of pNaCl, startup time and code size issues could easily be addressed by providing the architecture specific versions server side. Either the developer provides those versions as well, or a third party web service could do the translation.
The LLVM IR format could just be the developer target so that things are actually portable, and we don't wind up in a situation where 90% of the NaCl plugins are x86-only.
> And, furthermore, the size of the bitcode is still enormous, comparable to the generated machine code or even worse (think about the size of C++ binaries)
Actually I believe the bitcode is much larger than C or C++ binaries. The issue is that LLVM bitcode is basically a statically-typed language, lower than C. So for example to convert a pointer from one type to another you need to do an explicit cast (using an LLVM 'bitcast' operation). However, when you compile all the way down to native code, you have no need for such niceties and you just copy the value. So the bitcode ends up significantly larger than the native code would be, in order to maintain static typing correctness. There are some other issues as well.
LLVM bitcode was not designed for size, it's - exactly as the article says - just a compiler IR. So it isn't designed for size, portability, JITing speed, or anything like that. Trying to morph it into those is problematic and worse, may lead to compromises in LLVM's core goals. Which would be a shame since LLVM is a damn good compiler IR!
On Oct 4, 2011, at 11:53 AM, Dan Gohman wrote:
> In this email, I argue that LLVM IR is a poor system for building a
> Platform, by which I mean any system where LLVM IR would be a
> format in which programs are stored or transmitted for subsequent
> use on multiple underlying architectures.
Hi Dan,
I agree with almost all of the points you make, but not your conclusion. Many of the issues that you point out as problems are actually "features" that a VM like Java doesn't provide. For example, Java doesn't have uninitialized variables on the stack, and LLVM does. LLVM is capable of expressing the implicit zero initialization of variables that is implicit in Java, it just leaves the choice to the frontend.
Many of the other issues that you raise are true, but irrelevant when compared to other VMs. For example, LLVM allows a frontend to produce code that is ABI compatible with native C ABIs. It does this by requiring the frontend to know a lot about the native C ABI. Java doesn't permit this at all, and so LLVM having "this feature" seems like a feature over-and-above what high-level VMs provide. Similiarly, the "conditionally" supported features like large and obscurely sized integers simply don't exist in these VMs.
The one key feature that LLVM doesn't have that Java does, and which cannot be added to LLVM "through a small matter of implementation" is verifiable safety. Java bytecode verification is not something that LLVM IR permits, which you can't really do in LLVM (without resorting to techniques like SFI).
With all that said, I do think that we have a real issue here. The real issue is that we have people struggling to do things that a "hard" and see LLVM as the problem. For example:
1. The native client folks trying to use LLVM IR as a portable representation that abstracts arbitrary C calling conventions. This doesn't work because the frontend has to know the C calling conventions of the target.
2. The OpenCL folks trying to turn LLVM into a portable abstraction language by introducing endianness abstractions. This is hard because C is inherently a non-portable language, and this is only scratching the surface of the issues. To really fix this, OpenCL would have to be subset substantially, like the EFI C dialect.
> LLVM isn't actually a virtual machine. It's widely acknoledged that the
> name "LLVM" is a historical artifact which doesn't reliably connote what
> LLVM actually grew to be. LLVM IR is a compiler IR.
It sounds like you're picking a very specific definition of what a VM is. LLVM certainly isn't a high level virtual machine like Java, but that's exactly the feature that makes it a practical target for C-family languages. It isn't LLVM's fault that people want LLVM to magically solve all of C's portability problems.
Why copy-paste? (Well, I think I know why...)
Should we do the same with all messages from ML now? Just use normal ML UI, like gmane, I provided link to earlier. If you need direct one to Chris' mail, it would be:
Actually I'm not sure why this particular response was pointed out, as it is not perfect rebuttal. It's almost always better to even only skim whole thread than be picky about which mail to read carefully, as you'll always miss something then.
I'd guess Chris Lattner's response was singled out because Chris is the lead developer for LLVM, so even if it isn't the best response, it's probably still the most important.
I was already getting a bad feeling about LLVM recently, and this thread kind of cements that.
Everybody seems to have a different (sometimes radically different) idea of what LLVM is for. That can't be good for making progress, and it's definitely not good for guys like me wondering if LLVM is a sane choice for a project.
Don't take it too far. For purposes it was designed to serve, LLVM is great, best in its class. If your project is something that could benefit from LLVM, then by all means use it. If it isn't, then don't. As simple as that.
Many corporations (most of all Apple) bet millions of $$$s in resources on projects that depend on LLVM, so don't worry too much.
And what purpose was it designed to serve? That's exactly the problem I'm alluding to -- the LLVM developers don't even agree amongst themselves, so how am I supposed to know?
LLVM is a square peg. If your project needs a square shaped peg then it will do amazing things for you. On the other hand if you want it to do circle shaped things you will feel the pain.
Thanks for the fortune-cookie wisdom. That tells no one anything. We don't have square pegs or round holes, we have complex engineering problems to solve, and LLVM's complex shape is obscured by a community that doesn't agree on its shape, so we have no way to tell if it fits our holes.
I don't see any disagreement along those lines in the thread under discussion here. The LLVM project is quite clear (it says so on its homepage!) that it is not making a typical virtual machine along the lines of the JVM. What are you talking about? This sounds like completely random and out-of-context FUD.
And yet it says it might be used to build such, and this thread contains disagreement amongst the developers of the original purposes and the meaning of "virtual machine".
I'm just a developer who has looked into LLVM in the recent past for possible use in cross-platform projects and come away more confused than enlightened, and this thread just magnified it. Interpreting that as "FUD" is wildly unhelpful.
You are really making this out to be more confusing than it is. If you're not doing that intentionally, I think you should take a step back and try to look at things again without the assumption "This is confusing and everybody disagrees with each other."
Yes, the LLVM site does say it can be used in the construction of a virtual machine. That's factual. It has been used to construct actual, shipping virtual machines (e.g. Apple's MacRuby project). But that is not its primary or only use, and I don't think anyone would make that assumption if not for the name.
As for that thread, it contains some discussion among the developers about what LLVM is good for, but actually, nobody's really disagreeing with anybody else about what LLVM is and what it's good for.
Dan Gohman starts it out claiming that LLVM IR is not a very good bytecode for a portable virtual machine, and that if you want to build a portable VM, you'll need a separate layer for your bytecode.
Chris Lattner's reply (which you seem to read as disagrement) is essentially, "Yes, you're right, but here are the upsides that we get in exchange for those weaknesses as a bytecode."
In a sense. I was after a bytecode that had a JIT for the more common platforms (e.g. x86, ARM, possibly MIPS), could be extended for new ones without massive effort (even if sub-optimally), and had reasonable facilities for hooking into native code, and I was trying to avoid the complex baggage, legacy or otherwise, of a JVM.
LLVM initially looked like a candidate, but I walked away with conflicting ideas about whether it was even intended to be suitable for such a case, much less whether it actually was.
Think of LLVM as a back end for compiling C with exception support and a few more bits and pieces (e.g. GC hooks). Treat LLVM bitcode as a platform-specific artifact; i.e. don't try to share the same bitcode across multiple CPU targets and OS platforms, no more than you'd try to share the same C code across the same without using any conditional compilation symbols (i.e. it would be close to impossible).
LLVM bitcode isn't a good intermediate format for platform independence, for transport or for interpretation. What it is good for is having separate compilation while delaying whole program optimization until "link time" (but is actually LLVM linking time, not actual object file linking). LLVM bitcode lets you have multiple bits and pieces of a program written in different languages yet still have whole program optimization (e.g. inlining) across language boundaries. Look at it like that.
If you're trying to create a JIT for multiple platforms, I'd start with a bytecode format that is either easy to interpret or easy to transform trivially into naive CPU instructions (the former is better for high-level optimizations because it will retain high-level semantics, the latter a better shortcut for speed by skipping interpretation); treat LLVM as a CPU target, but be aware you'll probably want to generate different LLVM instructions for minor differences between final target platforms that poke through the abstraction layer. You'll need to either invent that bytecode format yourself or reusing someone else's (depending on what language you're trying to execute, I expect); but I wouldn't look to LLVM to give you that bytecode.
Just as a naive consumer of its facilities, I find LLVM produces binaries from C and ObjC code which are smaller and faster than their GCC equivalents. I'm attempting to write my own front-end as well, which is easier than I had expected. I don't care what the black box does, c.f. "The user doesn't care"(TM).
No one is complaining about using LLVM they way it's supposed to be used. The problem is that LLVM has been promoted for use cases that it totally sucks for (JIT, portable bitcode).
