With a plethora of new languages coming and going, people frequently try to do the cross-language performance comparisons. In most cases, that’s a strange thing to do, because even the minute differences in implementations can drive the benchmark scores off the cliff. It’s difficult even to compare a single small change in the otherwise stable environment. Trying to do the same when language compilers, bytecode generators, optimization opportunities, API differences, and contract obligations may differ drastically, happens to be orders of magnitude more complicated.
In this post, we will explore one of Java vs Scala comparisons. This is the extended version of my answer in the relevant thread on StackOverlow, and here we would see how to approach these problems, and what conclusions one may draw from the experiments.
As a good tradition, we will take some diversions into benchmarking methodology, so that even though the post itself is targeted to platform people, the non-platform people can still learn a few tricks. As usual, if you still haven’t learned about JMH and/or haven’t looked through the JMH samples, then I suggest you do that first before reading the rest of this post for the best experience.
Thanks to Nitsan Wakart and Pavel Rappo for review!
The discussion in that particular StackOverflow thread dates back a few questions, so instead of digging there, we will just take the latest benchmark code, and wrap it up with JMH. JMH already has the bindings for Java and Scala, which somewhat alleviates the difference in testing methodology. You can find full benchmark code