Comments on: Determining how much memory a require uses http://coderrr.wordpress.com/2009/04/08/determining-how-much-memory-a-require-uses/ pronounced "coder" not "code err" (extended r optional) Sun, 20 Dec 2009 10:03:37 +0000 http://wordpress.com/ hourly 1 By: coderrr http://coderrr.wordpress.com/2009/04/08/determining-how-much-memory-a-require-uses/#comment-1308 coderrr Fri, 10 Apr 2009 00:58:08 +0000 http://coderrr.wordpress.com/?p=640#comment-1308 Yea after looking a little more you're definitely right. There are even other magic limits above 8mb which screw up the measurements so allocating X at the beginning doesn't help. Yea after looking a little more you’re definitely right. There are even other magic limits above 8mb which screw up the measurements so allocating X at the beginning doesn’t help.

]]> By: coderrr http://coderrr.wordpress.com/2009/04/08/determining-how-much-memory-a-require-uses/#comment-1307 coderrr Thu, 09 Apr 2009 20:10:25 +0000 http://coderrr.wordpress.com/?p=640#comment-1307 hey brian thanks for the comment So yes, I know (a little) about the magic 8 mb and I partly messed with it a little and partly just ignored it. Maybe you're right, but consider the two following things. 1) I tried setting $a = "a"*20*1024*1024 at the beginning of mem.rb to allocate 20 megs so we get past this 8mb issue. The results are almost exactly the same. 2) The issue causing the order to make a difference could be due to both gems requiring the same files. Of course the first one to require them will be held responsible for their memory, but the latter will not as they have already been required by the first. That being said, the only two files I see both of those gems requiring take up less than 1mb combined. So maybe you're right but can you explain why allocating 20mb before hand doesn't make a diff? Counting objects doesn't seem to be good enough because it doesn't take into account the memory used by C extensions being loaded. Also it doesn't really tell you how much memory each object uses. hey brian thanks for the comment

So yes, I know (a little) about the magic 8 mb and I partly messed with it a little and partly just ignored it. Maybe you’re right, but consider the two following things.

1) I tried setting $a = “a”*20*1024*1024 at the beginning of mem.rb to allocate 20 megs so we get past this 8mb issue. The results are almost exactly the same.
2) The issue causing the order to make a difference could be due to both gems requiring the same files. Of course the first one to require them will be held responsible for their memory, but the latter will not as they have already been required by the first. That being said, the only two files I see both of those gems requiring take up less than 1mb combined.

So maybe you’re right but can you explain why allocating 20mb before hand doesn’t make a diff?

Counting objects doesn’t seem to be good enough because it doesn’t take into account the memory used by C extensions being loaded. Also it doesn’t really tell you how much memory each object uses.

]]> By: Brian Mitchell http://coderrr.wordpress.com/2009/04/08/determining-how-much-memory-a-require-uses/#comment-1306 Brian Mitchell Thu, 09 Apr 2009 19:48:10 +0000 http://coderrr.wordpress.com/?p=640#comment-1306 That isn't accurate actually. The primary problem is that Ruby's heap jumps to just short of 8MB (this can be tuned at compile time for MRI) once it hits its first ceiling on the heap. So this can that one library uses far more than it really does. For example try to require something like haml then json and then swap them. The order will change that delta. You'd be better off doing something like counting total objects in ObjectSpace. Event that will have problems though because the GC won't kick in till it needs to and many objects like symbols can be shared. That isn’t accurate actually. The primary problem is that Ruby’s heap jumps to just short of 8MB (this can be tuned at compile time for MRI) once it hits its first ceiling on the heap. So this can that one library uses far more than it really does.

For example try to require something like haml then json and then swap them. The order will change that delta. You’d be better off doing something like counting total objects in ObjectSpace. Event that will have problems though because the GC won’t kick in till it needs to and many objects like symbols can be shared.

]]>