Bullet: a gem to help reduce N+1 queries

Posted by Lucía Escanellas on November 11, 2010

Bullet is a plugin written by Richard Huang, that helps reducing the number of queries an application makes. It was first posted on 2009, but it is still a pretty useful gem to monitor your application for performance improvements.
It has several ways of notifying problems: by Growl notifications, JavaScript alerts by default, and even using XMPP too. Additionally, it saves on its own bullet.log the exact line and stack trace of what caused the alert, and if you want to, it can also write to the application log.

The project is on GitHub: http://github.com/flyerhzm/bullet

You are probably familiar with the N+1 query and cache counter problems, so let’s look how Bullet will detect these by monitoring the database queries.

The N+1 query problem

Suppose we have our application with accounts having many rate plans:

class RatePlan < ActiveRecord::Base
  belongs_to :account
end

If we iterate over the rate plans, accessing rate plans attributes, like:

RatePlan.all.each {
  |post| puts "#{rate_plan.name}, by #{rate_plan.account.name}"
}

Will be using N+1 queries:

  • one query to get all the rate plans
  • then, for each rate plan, one query to get the corresponding account.

(This means that if we have 1000 posts, we’ll be doing 1000+1 queries)

By using the :include option, we can retrieve the rate plans and their corresponding accounts on the same query:

RatePlan.find(:all, :include => :account).each {
   |post| puts "#{rate_plan.name}, by #{rate_plan.account.name}"
}

By detecting if we forgot to use eager loading or a counter cache, Bullet can save time looking for problems on the fly.
For example, Bullet will show this message:

N+1 Query Detected Message

We have to be careful not to abuse the :include option. If the rate_plans table is big enough, it could take too much server memory. Also, we could end up with big, slow requests, or getting from the server a lot of data that won’t be used. Bullet will show a message if it detects an unused eager loading.

Cache counters

Cache counters is another improvement in a one-to-many relationship. Because the _has_many_ relationship defines an attribute that is a collection, if we use the size property on this attribute, it will trigger a select count(*) on the child table. This is generally acceptable, except when we are using frequently count and we end up going to the database unnecessarily.
In order to avoid that, we can use counter caching: Active Record will maintain for a parent table the number of child references. Bullet also shows a message if it finds a cache counter is needed.

Configuration

You have to first install the gem, by adding it to the Gemfile, or by installing manually:

sudo gem install bullet --pre

The next thing is to add some configuration to the development environment. Bullet won’t do anything unless you configure it explicitly, and also note that it’s not a good idea to configure it on the production environment, because on that case your users would also receive those N+1 query alerts.

Configuration allows Growl notifications, and sending messages by XMPP (Jabber), but since the default JavaScript alerts are good enough for me, I configured it like this:

config.after_initialize do
  Bullet.enable = true
  Bullet.alert = true
  Bullet.bullet_logger = true
  Bullet.console = true
  Bullet.growl = false
  Bullet.rails_logger = true
  Bullet.disable_browser_cache = true
end

Note the last option, because disabling browser cache can save you some trouble on some configurations. In any case, if Bullet is not working, try first disabling browser cache.

With this configuration, it will show a JavaScript alert, the same message on the console, and the detail and stack trace on the bullet.log.

Ruby Benchmarks 2

Posted by Gian Zas on June 18, 2009

One of the most popular rants against ruby are based on its not so good performance.

Actually has been made improvements on this topic, specially in the new Ruby implementation (Ruby 1.9) based on the YARV virtual machine.

JRuby (Ruby over JVM implementation) has been focused in its performance from its lastest releases. Here at moove-it we are exploring the posibility of use Rails over JRuby in some JEE application servers, so we need some facts about JRuby performance (and other topics like gems compatibility, etc)

The Ruby community has put a set of benchmarks at ruby1.9 trunk: http://svn.ruby-lang.org/repos/ruby/trunk/benchmark/

So, we test with these benchmarks and here are the results!.

The code that run the benchmarks is like that:

block_to_benchmark = lambda { load BENCHMARKS_DIR + '/' + filename}
Benchmark.measure &block_to_benchmark

(yes, we are using the benchmark module bundled with the ruby standard lib)

Benchmark

Ruby1.8

Ruby1.9

JRuby1.3.0

Ruby1.8 /

Ruby1.9

Ruby1.8 /

JRuby1.3.0

bm_app_fib.rb 9.02 4.00 3.83 2.25 2.36
bm_app_mandelbrot.rb 3.36 0.81 1.49 4.14 2.26
bm_app_pentomino.rb 144.24 91.97 105.64 1.57 1.37
bm_app_raise.rb 6.94 6.98 1.63 -1.01 4.27
bm_app_strconcat.rb 2.94 1.63 1.30 1.81 2.27
bm_app_tak.rb 12.27 5.66 4.06 2.17 3.02
bm_app_tarai.rb 9.81 4.83 3.27 2.03 3.00
bm_app_uri.rb 6.59 3.77 3.69 1.75 1.79
bm_io_file_create.rb 7.17 3.12 7.70 2.30 -1.07
bm_io_file_read.rb 2.27 0.61 0.64 3.71 3.54
bm_io_file_write.rb 1.77 10.98 0.31 -6.22 5.64
bm_loop_for.rb 2.97 7.81 7.50 -2.63 -2.53
bm_loop_generator.rb 149.64 3.00 10.95 49.88 13.66
bm_loop_times.rb 4.88 6.88 7.95 -1.41 -1.63
bm_loop_whileloop.rb 11.23 3.67 9.42 3.06 1.19
bm_loop_whileloop2.rb 2.33 0.72 1.89 3.24 1.23
bm_so_array.rb 8.41 7.33 16.17 1.15 -1.92
bm_so_binary_trees.rb 4.50 2.14 2.84 2.10 1.58
bm_so_concatenate.rb 2.49 1.95 3.03 1.27 -1.22
bm_so_exception.rb 7.62 10.42 2.63 -1.37 2.90
bm_so_fasta.rb 13.59 11.77 16.03 1.16 -1.18
bm_so_lists.rb 2.27 1.38 1.48 1.65 1.53
bm_so_mandelbrot.rb 44.55 32.49 49.28 1.37 -1.11
bm_so_matrix.rb 2.69 2.13 1.84 1.26 1.46
bm_so_meteor_contest.rb 52.55 25.17 22.02 2.09 2.39
bm_so_nbody.rb 35.80 26.61 16.78 1.35 2.13
bm_so_nested_loop.rb 6.09 6.88 8.64 -1.13 -1.42
bm_so_nsieve.rb 26.89 13.11 24.69 2.05 1.09
bm_so_nsieve_bits.rb 62.50 46.05 42.55 1.36 1.47
bm_so_object.rb 11.56 11.52 3.11 1.00 3.72
bm_so_partial_sums.rb 80.13 228.91 31.22 -2.86 2.57
bm_so_pidigits.rb 10.33 10.44 7.03 -1.01 1.47
bm_so_random.rb 4.59 12.88 1.86 -2.80 2.47
bm_so_sieve.rb 0.84 0.34 0.63 2.45 1.35
bm_so_spectralnorm.rb 41.86 92.88 20.03 -2.22 2.09
bm_vm1_block.rb 26.22 13.44 27.44 1.95 -1.05
bm_vm1_const.rb 19.02 6.27 17.64 3.03 1.08
bm_vm1_ensure.rb 20.06 5.11 16.19 3.93 1.24
bm_vm1_ivar.rb 17.95 9.50 18.78 1.89 -1.05
bm_vm1_ivar_set.rb 19.22 9.83 21.67 1.96 -1.13
bm_vm1_length.rb 22.95 7.44 18.16 3.09 1.26
bm_vm1_neq.rb 20.81 6.58 14.98 3.16 1.39
bm_vm1_not.rb 14.91 5.58 12.64 2.67 1.18
bm_vm1_rescue.rb 15.72 4.64 21.38 3.39 -1.36
bm_vm1_simplereturn.rb 23.84 9.66 16.05 2.47 1.49
bm_vm1_swap.rb 50.25 5.73 24.38 8.76 2.06
bm_vm2_array.rb 10.72 19.27 5.42 -1.80 1.98
bm_vm2_case.rb 5.06 1.66 4.00 3.06 1.27
bm_vm2_eval.rb 32.20 200.98 69.41 -6.24 -2.16
bm_vm2_method.rb 15.45 9.45 11.36 1.63 1.36
bm_vm2_mutex.rb 5.47 6.34 7.30 -1.16 -1.33
bm_vm2_poly_method.rb 20.61 12.16 21.05 1.70 -1.02
bm_vm2_poly_method_ov.rb 5.00 1.66 4.53 3.02 1.10
bm_vm2_proc.rb 12.00 3.86 6.75 3.11 1.78
bm_vm2_regexp.rb 5.89 19.25 5.63 -3.27 1.05
bm_vm2_send.rb 5.05 2.11 4.38 2.39 1.15
bm_vm2_super.rb 5.75 3.17 4.44 1.81 1.30
bm_vm2_unif1.rb 5.20 1.99 3.61 2.62 1.44
bm_vm2_zsuper.rb 6.87 3.48 5.49 1.97 1.25
bm_vm3_thread_create_join.rb 1.95 7.70 19.13 -3.94 -9.79
bm_vm3_gc.rb 292.30 266.14 0.36 1.10 814.20

.

A looser conclusion may be that Ruby 1.9 is 95% faster than Ruby1.8, and JRuby 1.3.0 is 10% faster than 1.8, in general the new implementations are faster than Ruby1.8,  especially 1.9 (twice as faster).

The benchmarks were under WindowsXP  SP3, 4GB RAM, and a Intel Core 2 duo 2.0GHz. Happy hacking!

Trick to improve performance in rails, less requests with static resources 2

Posted by Pablo Ifran on April 08, 2009

Reducing the number of request made to the server improves the performance of a web application in about 80%.

There are many techniques that allow us to reduce the amount of requests that are made on a page, among them are: the sprites, put the stylesheets on top of the page, javascripts compress, among others.

But what’s offered by Rails to improve the performance of our web application?

It offers a great plugin called bundle_fu (http://code.google.com/p/bundle-fu/)
It allows us with a single request obtain all the javascripts and with another request all the stylesheets  (it also offers the possibility of compress javascripts).
Using this plugin is really easy but it’s very powerfull

<% bundle do -%>
  <%= javascript_include_tag :default -%>
  <%= javascript_include_tag "javascript1" -%>
  <%= javascript_include_tag "javascript2" -%>
  <%= javascript_include_tag "javascript3" -%>
  <%= stylesheet_link_tag "style1" -%>
  <%= stylesheet_link_tag "style2" -%>
  <%= stylesheet_link_tag "style3" -%>
  ...
<% end %>

All these javascripts and stylesheets are converted in only two files when the request is processed.