One of the most popular rants against ruby are based on its not so good performance.
Actually has been made improvements on this topic, specially in the new Ruby implementation (Ruby 1.9) based on the YARV virtual machine.
JRuby (Ruby over JVM implementation) has been focused in its performance from its lastest releases. Here at moove-it we are exploring the posibility of use Rails over JRuby in some JEE application servers, so we need some facts about JRuby performance (and other topics like gems compatibility, etc)
The Ruby community has put a set of benchmarks at ruby1.9 trunk: http://svn.ruby-lang.org/repos/ruby/trunk/benchmark/
So, we test with these benchmarks and here are the results!.
The code that run the benchmarks is like that:
block_to_benchmark = lambda { load BENCHMARKS_DIR + '/' + filename}
Benchmark.measure &block_to_benchmark
(yes, we are using the benchmark module bundled with the ruby standard lib)
|
Benchmark |
Ruby1.8 |
Ruby1.9 |
JRuby1.3.0 |
Ruby1.8 / Ruby1.9 |
Ruby1.8 / JRuby1.3.0 |
| bm_app_fib.rb | 9.02 | 4.00 | 3.83 | 2.25 | 2.36 |
| bm_app_mandelbrot.rb | 3.36 | 0.81 | 1.49 | 4.14 | 2.26 |
| bm_app_pentomino.rb | 144.24 | 91.97 | 105.64 | 1.57 | 1.37 |
| bm_app_raise.rb | 6.94 | 6.98 | 1.63 | -1.01 | 4.27 |
| bm_app_strconcat.rb | 2.94 | 1.63 | 1.30 | 1.81 | 2.27 |
| bm_app_tak.rb | 12.27 | 5.66 | 4.06 | 2.17 | 3.02 |
| bm_app_tarai.rb | 9.81 | 4.83 | 3.27 | 2.03 | 3.00 |
| bm_app_uri.rb | 6.59 | 3.77 | 3.69 | 1.75 | 1.79 |
| bm_io_file_create.rb | 7.17 | 3.12 | 7.70 | 2.30 | -1.07 |
| bm_io_file_read.rb | 2.27 | 0.61 | 0.64 | 3.71 | 3.54 |
| bm_io_file_write.rb | 1.77 | 10.98 | 0.31 | -6.22 | 5.64 |
| bm_loop_for.rb | 2.97 | 7.81 | 7.50 | -2.63 | -2.53 |
| bm_loop_generator.rb | 149.64 | 3.00 | 10.95 | 49.88 | 13.66 |
| bm_loop_times.rb | 4.88 | 6.88 | 7.95 | -1.41 | -1.63 |
| bm_loop_whileloop.rb | 11.23 | 3.67 | 9.42 | 3.06 | 1.19 |
| bm_loop_whileloop2.rb | 2.33 | 0.72 | 1.89 | 3.24 | 1.23 |
| bm_so_array.rb | 8.41 | 7.33 | 16.17 | 1.15 | -1.92 |
| bm_so_binary_trees.rb | 4.50 | 2.14 | 2.84 | 2.10 | 1.58 |
| bm_so_concatenate.rb | 2.49 | 1.95 | 3.03 | 1.27 | -1.22 |
| bm_so_exception.rb | 7.62 | 10.42 | 2.63 | -1.37 | 2.90 |
| bm_so_fasta.rb | 13.59 | 11.77 | 16.03 | 1.16 | -1.18 |
| bm_so_lists.rb | 2.27 | 1.38 | 1.48 | 1.65 | 1.53 |
| bm_so_mandelbrot.rb | 44.55 | 32.49 | 49.28 | 1.37 | -1.11 |
| bm_so_matrix.rb | 2.69 | 2.13 | 1.84 | 1.26 | 1.46 |
| bm_so_meteor_contest.rb | 52.55 | 25.17 | 22.02 | 2.09 | 2.39 |
| bm_so_nbody.rb | 35.80 | 26.61 | 16.78 | 1.35 | 2.13 |
| bm_so_nested_loop.rb | 6.09 | 6.88 | 8.64 | -1.13 | -1.42 |
| bm_so_nsieve.rb | 26.89 | 13.11 | 24.69 | 2.05 | 1.09 |
| bm_so_nsieve_bits.rb | 62.50 | 46.05 | 42.55 | 1.36 | 1.47 |
| bm_so_object.rb | 11.56 | 11.52 | 3.11 | 1.00 | 3.72 |
| bm_so_partial_sums.rb | 80.13 | 228.91 | 31.22 | -2.86 | 2.57 |
| bm_so_pidigits.rb | 10.33 | 10.44 | 7.03 | -1.01 | 1.47 |
| bm_so_random.rb | 4.59 | 12.88 | 1.86 | -2.80 | 2.47 |
| bm_so_sieve.rb | 0.84 | 0.34 | 0.63 | 2.45 | 1.35 |
| bm_so_spectralnorm.rb | 41.86 | 92.88 | 20.03 | -2.22 | 2.09 |
| bm_vm1_block.rb | 26.22 | 13.44 | 27.44 | 1.95 | -1.05 |
| bm_vm1_const.rb | 19.02 | 6.27 | 17.64 | 3.03 | 1.08 |
| bm_vm1_ensure.rb | 20.06 | 5.11 | 16.19 | 3.93 | 1.24 |
| bm_vm1_ivar.rb | 17.95 | 9.50 | 18.78 | 1.89 | -1.05 |
| bm_vm1_ivar_set.rb | 19.22 | 9.83 | 21.67 | 1.96 | -1.13 |
| bm_vm1_length.rb | 22.95 | 7.44 | 18.16 | 3.09 | 1.26 |
| bm_vm1_neq.rb | 20.81 | 6.58 | 14.98 | 3.16 | 1.39 |
| bm_vm1_not.rb | 14.91 | 5.58 | 12.64 | 2.67 | 1.18 |
| bm_vm1_rescue.rb | 15.72 | 4.64 | 21.38 | 3.39 | -1.36 |
| bm_vm1_simplereturn.rb | 23.84 | 9.66 | 16.05 | 2.47 | 1.49 |
| bm_vm1_swap.rb | 50.25 | 5.73 | 24.38 | 8.76 | 2.06 |
| bm_vm2_array.rb | 10.72 | 19.27 | 5.42 | -1.80 | 1.98 |
| bm_vm2_case.rb | 5.06 | 1.66 | 4.00 | 3.06 | 1.27 |
| bm_vm2_eval.rb | 32.20 | 200.98 | 69.41 | -6.24 | -2.16 |
| bm_vm2_method.rb | 15.45 | 9.45 | 11.36 | 1.63 | 1.36 |
| bm_vm2_mutex.rb | 5.47 | 6.34 | 7.30 | -1.16 | -1.33 |
| bm_vm2_poly_method.rb | 20.61 | 12.16 | 21.05 | 1.70 | -1.02 |
| bm_vm2_poly_method_ov.rb | 5.00 | 1.66 | 4.53 | 3.02 | 1.10 |
| bm_vm2_proc.rb | 12.00 | 3.86 | 6.75 | 3.11 | 1.78 |
| bm_vm2_regexp.rb | 5.89 | 19.25 | 5.63 | -3.27 | 1.05 |
| bm_vm2_send.rb | 5.05 | 2.11 | 4.38 | 2.39 | 1.15 |
| bm_vm2_super.rb | 5.75 | 3.17 | 4.44 | 1.81 | 1.30 |
| bm_vm2_unif1.rb | 5.20 | 1.99 | 3.61 | 2.62 | 1.44 |
| bm_vm2_zsuper.rb | 6.87 | 3.48 | 5.49 | 1.97 | 1.25 |
| bm_vm3_thread_create_join.rb | 1.95 | 7.70 | 19.13 | -3.94 | -9.79 |
| bm_vm3_gc.rb | 292.30 | 266.14 | 0.36 | 1.10 | 814.20 |
.
A looser conclusion may be that Ruby 1.9 is 95% faster than Ruby1.8, and JRuby 1.3.0 is 10% faster than 1.8, in general the new implementations are faster than Ruby1.8, especially 1.9 (twice as faster).
The benchmarks were under WindowsXP SP3, 4GB RAM, and a Intel Core 2 duo 2.0GHz. Happy hacking!
![[Post to Twitter]](http://blog.moove-it.com/wp-content/plugins/tweet-this/icons/tt-twitter.png){kind=icon}
You may want to look at our tips on benchmarking JRuby:
http://kenai.com/projects/jruby/pages/Benchmarks
The basic problems with your benchmark run are that the standard RBS suite runs each benchmark only once, and you did not run it with the –server flag to JRuby that turns on the optimizing JVM.
Each of the benchmarks is very short; this means that they will only rarely start to optimize. This obviously doesn’t reflect reality, since most applications people will run on JRuby will run longer than 5-10 seconds. In general to see the benefits of running on JRuby a benchmark needs to run long or be run repeatedly in the same JVM *without* redefining methods (which causes optimization to start over). RBS does neither of these.
The “–server” flag turns on the optimizing JIT in the JVM, where usually the less-optimized “client” JIT is used. “client” starts up faster, which is why we have it set as the default in JRuby.
Switching to –server and making the benchmarks run a few times each will at least double JRuby’s performance on most of these benchmarks.
i agree
Each of the benchmarks is very short; this means that they will only rarely start to optimize. This obviously doesn’t reflect reality, since most applications people will run on JRuby will run longer than 5-10 seconds. In general to see the benefits of running on JRuby a benchmark needs to run long or be run repeatedly in the same JVM *without* redefining methods (which causes optimization to start over). RBS does neither of these.
well very nice post and the discussion seems to be very classic man keep up the good work and i wana say that post some more information about this topic actually i am a student and i love reading blogs because it provide great help and information man i am a student and recently completed my CISSP practice exams and i am really concerned about your post topic..!!!!!!