coderrr

December 4, 2008

Contributing to open source can be hard and frustrating

Filed under: bug, c, patch, ruby — Tags: , , , — coderrr @ 7:47 pm

Has anyone ever told you that you should try contributing to some open source projects? Maybe they said it as advice for improving your resume or maybe they offered it as a solution to boredom. Either way, contributing to open source isn’t always as easy as it sounds.

Disclaimer: I’m in no way bitching or complaining about the following projects/people. These are just some past experiences I wanted to share.

Here’s a list of a few open source projects I’ve written (useful) patches for which haven’t been merged into the original project.

First up: Hpricot
As documented by this post, over a year ago I patched the Hpricot C and Java extensions to support arbitrarily large HTML elements. Instead of throwing an error after a certain buffer size had been hit, it would increase the memory allocation size. I did so because I was writing software for someone which did some website crawling and could potentially (and actually did) run into very large elements on the web (which crashed hpricot or threw an error). So it had a real world use case, judging by the responses I’ve gotten since I wrote it quite a few other people have had uses for it, and it just made sense. (Oh, it also didn’t affect performance)

But to this day, over a year later, I haven’t been able to get _why to merge it into hpricot core. I created a patch, made sure all the tests passed, sent an email to the mailing list, got a ticket filed on their trac. I’ve even rewritten the patch once to make it work with his latest code on github. I even forked his repo, did a pull request, and messaged him. He even responded saying he’d get to it soon. Over a month later, still nothing. Now I understand the reality is that he’s just really busy, but my reality is that it really sucks. Also, PLEASE, don’t take this as bitching, I’m just telling a story, I have total love for _why.
Update: _why just merged my patch, yes!! :P

Number two:
I was using the seattle.rb project ImageScience to thumbnail images for one of my client’s apps. It crashed on BMPs, and since BMPs were one of my client’s requirements, I made a (tiny) patch to their C extension to get it working. Again, all the tests passed, I created a ticket, harassed some people on IRC, etc. Over a year later, nothing. I think someone even closed the bug without giving reason. Again, judging from the number of downloads of the patched gem, people found real use in it (I’m trying to save myself from everyone thinking I write a bunch of useless patches).

Next up, Rails:
This was when Rails was just beginning their mission of “let’s be thread safe”. I really enjoy playing in the spaghetti of threaded programming so I decided to see if I could help and find some issues to patch. I found a race condition in their processing of partials and wrote a patch to fix it, along with a brute force test to produce the race condition. I posted it to the mailing list and a discussion ensued. The gist of it was that “we don’t like mutexes”, and my patch was never accepted, and never got any +1s. Well guess what they ended up doing eventually before rails was considered thread-safe… they added a mutex -_-. Needless to say, this turned me off to contributing to Rails. (In the latest version they actually did make some changes which render the mutex unnecessary but that is irrelevant for this story). Again please note, I’m not trying to bitch, I have nothing but love for the Rails guys, this is just my story.

Finally:
For one of my contracts I reworked the FireWatir (automated UI testing) code to be thread-safe so that you could control multiple browsers at the same time in different threads. Something which is extremely helpful when you are trying to simulate multiple users simultaneously. Loads of people have used my patch to aid in their FireWatir testing. It passed the entire test suite. But even after multiple emails to the maintainer of the project’s repository (who actually used my patch in his projects) for some reason it never got merged.

So what’s the conclusion from all these stories? Even after doing all the “right things” like making sure all the tests pass, ensuring performance isn’t affected, contacting the appropriate people through the appropriate channels I’ve had many patches go unloved. It’s kind of disheartening but I think what I’m learning is that you just have to be relentlessly persistent. Most people are just really busy and require a lot of external motivation if you want them to do things (which in the grand scheme of things are quite unimportant) for you. Of course at the same time you don’t wanna bug the shit out of someone by emailing them all the time, so…

I wonder if anyone has similar experiences or any advice on this sort of thing…

15 Comments »

  1. You are totally right… and unfortunately the above scenarios are not rare (had similar experience with the seattle.rb guys – created a gem to get ruby2ruby et al working on win32, and all I got were meh, buzz od dude kind of remarks ; )

    Fortunately github can help the above situation – just fork a project, do your changes and if the maintainer is sensible, he will pull it back. If he’s not, you still have your own branch with all the paches which you can pull the changes of the original project into. Not perfect, but better than nothing.

    re FireWatir patch: w00t! Where can I get it?

    Comment by Peter — December 4, 2008 @ 8:37 pm

  2. http://coderrr.wordpress.com/2007/10/15/patch-to-firewatir-and-jssh-to-support-testing-with-multiple-concurrent-firefox-browsers/

    there are links to the patched gem and jssh xpi which are attached to pasties…

    this is the bs you have to go through to install it because it never got merged D;

    Comment by coderrr — December 4, 2008 @ 8:54 pm

  3. Thanks, will check it out!

    Comment by Peter — December 4, 2008 @ 9:32 pm

  4. This is just the way it is. In the end, maintainers are people too, and they might be busy, have doubts, be uninterested, etc. What I’ve learned over the years is that, if your patch doesn’t get through the first time, keep pinging the maintainer from time to time, but without being overly annoying.

    And I second Peter’s opinion that Git helps. It helps a lot – it makes disagreeing with the maintainer and forking a project sooo much easier. Everybody should switch to git. :)

    Comment by Hongli Lai — December 4, 2008 @ 9:58 pm

  5. I completely understand what you’re talking about. I posted a very, very simple patch to Screw.Unit a few months ago that made it compatible with Internet Explorer (something that it already claims to be), and submitted the patch to the original github project. To this day I have yet to hear anything back about it. Really, more than anything, it’s just frustrating. There’s a lot of talk about open source projects, but rarely does that seem to pan out.

    Another good example of this seems to be Merb, where there was always this idea of people contributing because it was for hackers, and it was going to do better as a web application framework by addressing Rails shortcomings. At Merb camp however, after listening to talk about how important the community is, one of the core Merb developers basically came out and said that any patches that someone submits probably won’t be included (or at least that was the feeling that I got).

    Comment by Jeremy Gailor — December 4, 2008 @ 10:27 pm

  6. I must be lucky, because I’m 2 for 2 with projects I’ve submitted patches to accepting my code. One of them was just a small, one-liner bugfix, though.

    Comment by pete — December 5, 2008 @ 1:18 am

  7. I have had similar problems. I haven’t tried a ton but getting things merged in and recognized if your not part of the ‘known’ group is hard.

    Comment by Dan — December 5, 2008 @ 5:54 am

  8. i never tried to send a patch but what you all explains sounds consitent with the bazard vs. cathedral manifesto. creative chaos have advantages, but it’s still chaos :)

    Comment by stephan — December 5, 2008 @ 8:50 am

  9. As far as the hpricot and imagescience problems, I think it just goes to show how harmful it is to have projects that only have a single comitter.

    I’ve had similar frustrations with Rake, but never anything like that on a project driven by a team of hackers. (Rdoc, Rubygems, Rubinius, Emacs, etc.) If you’re relying on one guy, then the odds of a patch getting accepted have more to do with what that one guy is working on during the week in which you submit the patch than the actual content of the patch.

    Comment by Phil — December 6, 2008 @ 7:06 pm

  10. Yea good point Phil

    Comment by coderrr — December 6, 2008 @ 7:17 pm

  11. Looks like your patch got merged in the case of Hpricot at least:

    http://github.com/why/hpricot/commit/654432de691fc0f6e4ab0ea9458c3095defb5205

    Thanks for fixing it!

    Comment by Phil — December 8, 2008 @ 1:30 am

  12. Hello,

    I am also running into the problem with ImageScience and BMP files. I applied your patch to ImageScience 1.1.3, and the exception I was getting in ImageScience.wrap_and_yield no longer happens, however if I try to resize a BMP using ImageScience (make it smaller), the resulting BMP file that is created doesn’t render properly. The file has a size to it, but I’m guessing its contents are corrupted in some way. I don’t have any issues resizing other image types (jpegs, pngs, etc.) using ImageScience.

    Just curious to know if you were using ImageScience to resize BMPs, and if so, did it work for you just using your BMP Fix patch?

    BTW – I filed a bug with the image_science project on RubyForge (http://rubyforge.org/tracker/index.php?func=detail&aid=24109&group_id=1513&atid=5921), but so far no response to it.

    Thanks,
    Denis

    Comment by Denis Ahearn — February 27, 2009 @ 3:18 pm

  13. Hey Denis,

    Thanks for the comment. We did resize bmp’s and it worked “correctly”. I say “correctly” because we were only supporting firefox at the time and it worked fine with that. I just tested it now and the BMPs it created are not viewable in IE.

    Comment by coderrr — February 27, 2009 @ 8:33 pm

  14. Looks like your ImageScience fix was merged in in April 2008, unless I’m misunderstanding…

    http://rubyforge.org/tracker/?group_id=1513&atid=5921&func=detail&aid=11586

    Comment by Paul Canavese — June 18, 2009 @ 4:52 pm

  15. wow yea you’re right…. i swear that wasn’t there before :P i must be blind… will update the post, thanks for pointing that out…

    ah i see what happened, it was accepted through a different ticket, not the one i was looking at…

    Comment by coderrr — June 18, 2009 @ 4:59 pm


RSS feed for comments on this post. TrackBack URI

Leave a comment

Blog at WordPress.com.