I think you got it backward. Java could be overkill for small projects (although I never felt that way, ever).
Large projects need a complete solution that Java has to offer (Java as a platform, language, and its ecosystems).
Over the last 20-30 years, there's nothing out there in the market that really increase programmer's productivity other than Java (and its GC + tools like Maven, FindBugs, Jenkins, IDEs) and Rails (not Ruby, but Rails, Ruby itself is "meh").
I have yet to feel that way with other tools or programming languages yet (one of the reasons why they all failed the "mainstream" tests: only a select few thought that their niche tools are the most productive tools ever, the rest still aren't convinced and have yet to see big projects build on top of the said tools).
> Large projects need a complete solution that Java has to offer
I'm not sure that as true as it is popularly perceived to be; I think it may be more true that there is such a history of use of Java and similar platforms (e.g., .NET) in the environments that produce large systems that there is a cultural bias to thinking it is necessary and that much of the current experience of large systems is centered on that kind of platform.
Arguably, building large systems with less dependence on the kind of pervasive system-wide compile-time interdependencies fostered by Java and similar platforms and instead relying on smaller components with minimal compile-time interdependencies integrated at runtime using well-documented communications protocols and interaction standards -- a real platform agnostic, potentially heterogenous, decoupled Service Oriented Architecture (without, particularly, the return to compile-time interdependency fostered by the approach to SOAP/WSDL common on Java and similar platforms) -- would be better for large systems, particularly when the large systems have components that need to adapt rapidly to changing needs.
The last time I used it in JavaEE6, it was surprisingly very easy and pleasant to work with.
Server-side
===========
1. Write the services
2. Prepare your DTO classes, annotate them
3. Run through JDK command line utility => output: XSD, WSDL
Client-side
============
1. Grab that WSDL+XSD
2. Run them through JDK client-side tool utility
3. Consume the generated code (if you use maven, you can also prepare a JAR and make em as dependencies)
You can automate all of these via maven plugins easily for both sides.
Yes, you'd have to recompile them but it gives you a bit of security/guarantees unlike REST. I thought my experience was not painful enough to warrant to ignore this solution depending on the use-case/situations (i.e.: you could always go overboard).
Compare that to some random Netflix.rb (ruby based) library that internally relies on XPath... feels a bit shaky for me.
But YMMV and definitely it's more of my preferences than other people's.
I do aware of SOA-based solution by breaking down a big system to multiple services with the hope of each system can be independent of others and hopefully can be re-used easily by other services as well. The assumptions here is that each service has its own team (of 3-4 people) that maintain such systems. Sure, you can definitely write these services using anything you want and you certainly invalidate my opinions on that. But there's a price to be paid going toward this model: traceability is harder, collaboration effort also increases, more failure points, etc (I'm sure you're aware of these).
Large projects need a complete solution that Java has to offer (Java as a platform, language, and its ecosystems).
Over the last 20-30 years, there's nothing out there in the market that really increase programmer's productivity other than Java (and its GC + tools like Maven, FindBugs, Jenkins, IDEs) and Rails (not Ruby, but Rails, Ruby itself is "meh").
I have yet to feel that way with other tools or programming languages yet (one of the reasons why they all failed the "mainstream" tests: only a select few thought that their niche tools are the most productive tools ever, the rest still aren't convinced and have yet to see big projects build on top of the said tools).