Oh, you're talking about a policy limitation, not a technological one. (And if you're talking about the DOE or NSF/Teragrid/XSEDE clusters, then you're probably right. Haven't touched those in years -- and even when I did, I wasn't doing anything crazy.)
To be frank, if I was running a computer that was designed for peak performance, I probably wouldn't use Java. There are some very significant performance issues with garbage collection that prevent you from making peak use of them machine.
Supercomputers aren't built so that people can squander the resource (desktop PCs, closest clusters, and phones fulfill that role).
It is a technical limitation. Oftentimes the platform is so specialized that only a tiny handful of compilers are ported to it. Say, just gcc, g++, and gfortran, and xlc, xlC, and xlf. And just one version at that. Java would require porting the JVM to the cut-down, weird Linux on the compute nodes. Some $$$ machines don't even support dynamic linking! The number of these machines is so small that extensive compiler and tool support just isn't happening unless you want to add millions to the cost.
The JVM probably calls fork() and system(), no? Not allowed. Dynamic thread creation? Not allowed. And 50% of your flops go away unless your program uses the BG/Q-specific "double hummer" floating point instructions. These are primitive machines, in terms of development environment and typically require significant rewriting to get even "standard" system software working.
