Here's the quiz of the day: Can you find the exclusive lock in this code?
(hint: it begins on line 2, and ends on line 36)
It seemed reasonable that ActiveRecord.allow_currency would make my application be able to run concurrent queries. Well, that's what I thought :) Until I had discovered my 6 threads (which properly formed 6 new connections to the database) were still executing queries serially, instead of in parallel! When investigating what the hold up was, I found my culprit: Ruby 1.8's green threads lose control when you run native C functions, well at least Mysql#query anyways.
So, note to self and world: any time you run a native function that takes a long time, look forward to a big, exclusive lock down.
All the more reason I'm excited for Ruby 1.9's native threads
So, web, do you know if there's a way to "poll" ruby's thread scheduler while in a native C function while waiting for some other task to finish?
5 comments:
don't know how with 1.8.x. With 1.9 you could release a thread temporarily from having its global lock, I believe. Or see my post on using fibers and evented sql for 1.9. Barring that I suppose maybe fork off or something might help.
GL.
-R
note also that because ruby 1.9 still uses a global thread lock it only allows one thread to run at a time, so...currently anything that doesn't use rb_thread_block regions will still block the other threads. So current mysql will still block [even if compiled on 1.9]. Native threads, but not totally native :)
I believe Python works the same way. It's like the two languages are coalescing!
I discovered this after I posted this article :) So, so much for my excitement for native POSIX threads. Grrr. Ah well - the forking route and spawning little tiny query workers has proved quiet effective to get queries to execute in parallel.
sounds like another good way. I guess fork is ruby's traditional way to try and be multi-threaded :)
I like that you tagged this post "nerd." I wish there was some way to tag my life like that.
Post a Comment