Tim, the Enchanter

What manner of man are you that can summon up code without C# or Java?

Monday, June 23, 2008

Can you find the exclusive lock?

Here's the quiz of the day: Can you find the exclusive lock in this code?

(hint: it begins on line 2, and ends on line 36)

It seemed reasonable that ActiveRecord.allow_currency would make my application be able to run concurrent queries. Well, that's what I thought :) Until I had discovered my 6 threads (which properly formed 6 new connections to the database) were still executing queries serially, instead of in parallel! When investigating what the hold up was, I found my culprit: Ruby 1.8's green threads lose control when you run native C functions, well at least Mysql#query anyways.

So, note to self and world: any time you run a native function that takes a long time, look forward to a big, exclusive lock down.

All the more reason I'm excited for Ruby 1.9's native threads

So, web, do you know if there's a way to "poll" ruby's thread scheduler while in a native C function while waiting for some other task to finish?

Labels: nerd, ruby

posted by Tim Harper # 7:48 PM

Comments:

don't know how with 1.8.x. With 1.9 you could release a thread temporarily from having its global lock, I believe. Or see my post on using fibers and evented sql for 1.9. Barring that I suppose maybe fork off or something might help.
GL.
-R

# posted by

Roger Pack : June 25, 2008 10:47 AM

note also that because ruby 1.9 still uses a global thread lock it only allows one thread to run at a time, so...currently anything that doesn't use rb_thread_block regions will still block the other threads. So current mysql will still block [even if compiled on 1.9]. Native threads, but not totally native :)
I believe Python works the same way. It's like the two languages are coalescing!

# posted by

Roger Pack : July 7, 2008 9:26 PM

I discovered this after I posted this article :) So, so much for my excitement for native POSIX threads. Grrr. Ah well - the forking route and spawning little tiny query workers has proved quiet effective to get queries to execute in parallel.

# posted by

Tim Harper : July 7, 2008 9:36 PM

sounds like another good way. I guess fork is ruby's traditional way to try and be multi-threaded :)

# posted by

Roger Pack : July 15, 2008 2:04 PM

I like that you tagged this post "nerd." I wish there was some way to tag my life like that.

# posted by

mikemetcalf : July 17, 2008 2:11 PM

Tim, the Enchanter

Monday, June 23, 2008

Can you find the exclusive lock?

Archives