benchmarking jruby invokedynamic with a production application
As you read JRuby blog posts and other resources around the web, you may find references to JRuby’s compile.invokedynamic
option, that as of 9.1.7.0 continues to ship turned off by default.
So, why is it off? I asked the JRuby developers and here’s what they had to say about it:
@KnuX @tom_enebo Primarily startup and warm-up time: both increase. It has improved somewhat in Java 8.
— Charles Nutter (@headius) January 26, 2017
@knux @headius It is only for warmup reasons. Eventual performance is generally better (or at least as good).
— tom_enebo (@tom_enebo) January 26, 2017
…and here’s what Joe Kutner’s “Deploying with JRuby 9k” has to say about it:
The
invokedynamic
instruction promised great performance improvements for JRuby, but it turned out to improve only a few specific cases. […] for now, JRuby keeps this feature off by default. You may find it worthwhile to test your application withinvokedynamic
turned on. It’s possible you’ll see some performance gains depending on the kind of logic you have in your app.
So in some benchmarks it seems rather relevant, but others suggest than on production applications it may not be the way to go. So which is it?
To test it out, I decided to benchmark it with one of Talkdesk’s internal JRuby applications. This application (dubbed ciw) reads tasks from rabbitmq message queue, and communicates with another internal microservice in order to update or create new records based on the received data.
After warming up the application, I loaded 6000 tasks on the rabbitmq message queue and measured how long (in seconds) the application took to go through all the tasks and empty the queue. The tests were performed on a Core i7-6500U (2.5 GHz) with 8GB of RAM running Ubuntu Linux 16.10, and the JVM used was the Oracle Java(TM) SE Runtime Environment (build 1.8.0_121-b13) / Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
. The application was configured to use 16 threads to process tasks.
I tested four different configurations: both JRuby 9.0.5.0 and 9.1.7.0 are tested on their default configurations (compile.invokedynamic
turned off) and with the option turned on.
The results are shown below:
Turning compile.invokedynamic
on yields a 26% speedup for JRuby 9.1.7.0, shaving more than a minute from the time needed to execute the workload!
Furthermore, we can also see how JRuby performance has slightly improved in a year, between JRuby 9.0.5.0 (released in January of 2016) and JRuby 9.1.7.0 (released in January of 2017).
So what’s the verdict? If you’re not impacted by having longer start-up and warm-up times for your application, you should really be adding -Xcompile
.invokedynamic=true
to your JRUBY_OPTS
or application startup scripts.
As for JRuby it may be time to enable this option by default again. Or at least rename it to something more awesome-sounding, like --production-booster
;)
Update, 11th May 2017: Recently, while I was benchmarking a web application using JRuby, I discovered a use case where code using compile.invokedynamic
ran noticeably slower than when that option was disabled.
Turns out it was a bug, which the JRuby developers fixed awesomely quickly, making compile.invokedynamic
again faster than the non-invokedynamic code path. So, if you find any such instances, do report!