Here’s a quick little monkeypatch of the require method which will print out a tree showing how much memory each required file used:
# mem.rb
$require_level = 0
module MemTrackingRequire
def require(name)
$require_level += 1
mem = lambda { GC.start; `ps -o rss= -p #$$`.to_i }
before = mem.call
required = super
ensure
after = mem.call
$require_level -= 1
if required
puts "#{" "*$require_level}#{name} #{after - before}"
puts "-------" if $require_level.zero?
end
end
end
Object.class_eval { include MemTrackingRequire }
Here are some examples of how to use it (assuming it’s saved as mem.rb)
$ ruby -rmem -e 'require "rubygems"' rubygems/rubygems_version 0 rubygems/defaults 0 thread.so 24 thread 28 rbconfig 36 ... rubygems/config_file 928 rubygems 0 rubygems/custom_require 0 rubygems 1788 # <-- this is the number you are looking for, it represents the total amount of memory used by requiring 'rubygems'
$ ruby -rubygems -rmem -e 'gem "activerecord", "=2.2.2"; require "activerecord"' ... active_record/i18n_interpolation_deprecation 0 active_record 5896 activerecord 5896
$ ruby -rmem -e 'require "net/http"' ... uri/mailto 0 uri 60 net/http 800

That isn’t accurate actually. The primary problem is that Ruby’s heap jumps to just short of 8MB (this can be tuned at compile time for MRI) once it hits its first ceiling on the heap. So this can that one library uses far more than it really does.
For example try to require something like haml then json and then swap them. The order will change that delta. You’d be better off doing something like counting total objects in ObjectSpace. Event that will have problems though because the GC won’t kick in till it needs to and many objects like symbols can be shared.
Comment by Brian Mitchell — April 9, 2009 @ 7:48 pm
hey brian thanks for the comment
So yes, I know (a little) about the magic 8 mb and I partly messed with it a little and partly just ignored it. Maybe you’re right, but consider the two following things.
1) I tried setting $a = “a”*20*1024*1024 at the beginning of mem.rb to allocate 20 megs so we get past this 8mb issue. The results are almost exactly the same.
2) The issue causing the order to make a difference could be due to both gems requiring the same files. Of course the first one to require them will be held responsible for their memory, but the latter will not as they have already been required by the first. That being said, the only two files I see both of those gems requiring take up less than 1mb combined.
So maybe you’re right but can you explain why allocating 20mb before hand doesn’t make a diff?
Counting objects doesn’t seem to be good enough because it doesn’t take into account the memory used by C extensions being loaded. Also it doesn’t really tell you how much memory each object uses.
Comment by coderrr — April 9, 2009 @ 8:10 pm
Yea after looking a little more you’re definitely right. There are even other magic limits above 8mb which screw up the measurements so allocating X at the beginning doesn’t help.
Comment by coderrr — April 10, 2009 @ 12:58 am