Saturday, January 30, 2010

Getting ruby to use readline instead of libedit

I prefer readline for a variety of reasons:
  • I'm used to it, it's the same command editor that powers bash
  • It's very powerful.
  • In Ruby, it doesn't block other threads from running while waiting for input.
When I installed ruby 1-9 using rvm, it got compiled against libedit. I didn't feel like reinstalling, so I followed these instructions over here.

I'd thought that `port install readline` would suffice, but it didn't. I had to install readline-6.0 from source. Ruby doesn't compile against readline-5.0, it just spits out a load of errors. The following is what I did, if you would like to follow along:

Install readline: (I use homebrew, see here for an alternative method)
brew install readline
Then, go into the src ext/readline folder where rvm built your ruby. Mine was ~/.rvm/src/ruby-1.9.1-p378/ext/readline.
cd ~/.rvm/src/ruby-1.9.1-p378/ext/readline # <- your path will vary
ruby extconf.rb --with-readline-dir=/usr/local/Cellar/readline/6.0
Make sure you see this line in the output:
checking for readline/readline.h... yes
Then, build it:
make
When it finishes, run otool -l readline.bundle. You should see this:
       name /usr/local/lib/libreadline.6.0.dylib (offset 24)
 time stamp 2 Wed Dec 31 17:00:02 1969
    current version 6.0.0
compatibility version 6.0.0
If you see libedit anywhere in that output, it didn't link against libreadline. Failure. Now, if it's all successful, install the new readline.bundle.
  make install
Good luck!

Update:

If another version of Ruby is used to run the extconf.rb then the target version of Ruby for which you are compiling read line, you will get another stream of build errors, perhaps something like this:

gcc -I. -I. -I/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/universal-darwin9.0 -I. -DHAVE_READLINE_READLINE_H -DHAVE_READLINE_HISTORY_H -DHAVE_RL_DEPREP_TERM_FUNCTION -DHAVE_RL_COMPLETION_APPEND_CHARACTER -DHAVE_RL_BASIC_WORD_BREAK_CHARACTERS -DHAVE_RL_COMPLETER_WORD_BREAK_CHARACTERS -DHAVE_RL_BASIC_QUOTE_CHARACTERS -DHAVE_RL_COMPLETER_QUOTE_CHARACTERS -DHAVE_RL_FILENAME_QUOTE_CHARACTERS -DHAVE_RL_ATTEMPTED_COMPLETION_OVER -DHAVE_RL_LIBRARY_VERSION -DHAVE_RL_EVENT_HOOK -DHAVE_RL_CLEANUP_AFTER_SIGNAL -DHAVE_REPLACE_HISTORY_ENTRY -DHAVE_REMOVE_HISTORY -I/usr/local/include   -fno-common -arch ppc -arch i386 -Os -pipe -fno-common  -c readline.c
readline.c: In function ‘readline_readline’:
readline.c:82: error: ‘rb_io_t’ undeclared (first use in this function)
readline.c:82: error: (Each undeclared identifier is reported only once
readline.c:82: error: for each function it appears in.)
readline.c:82: error: ‘ofp’ undeclared (first use in this function)
readline.c:82: error: ‘ifp’ undeclared (first use in this function)
readline.c: In function ‘filename_completion_proc_call’:
readline.c:703: error: ‘filename_completion_function’ undeclared (first use in this function)
readline.c:703: warning: assignment makes pointer from integer without a cast
readline.c: In function ‘username_completion_proc_call’:
readline.c:730: error: ‘username_completion_function’ undeclared (first use in this function)
readline.c:730: warning: assignment makes pointer from integer without a cast
readline.c: In function ‘readline_readline’:
readline.c:82: error: ‘rb_io_t’ undeclared (first use in this function)
readline.c:82: error: (Each undeclared identifier is reported only once
readline.c:82: error: for each function it appears in.)
readline.c:82: error: ‘ofp’ undeclared (first use in this function)
readline.c:82: error: ‘ifp’ undeclared (first use in this function)
readline.c: In function ‘filename_completion_proc_call’:
readline.c:703: error: ‘filename_completion_function’ undeclared (first use in this function)
readline.c:703: warning: assignment makes pointer from integer without a cast
readline.c: In function ‘username_completion_proc_call’:
readline.c:730: error: ‘username_completion_function’ undeclared (first use in this function)
readline.c:730: warning: assignment makes pointer from integer without a cast
lipo: can't open input file: /var/folders/jz/jzGJ6Q4BFumSODlDitruR++++TM/-Tmp-//ccvDT283.out (No such file or directory)
make: *** [readline.o] Error 1

To resolve, run extconf.rb with the same version of Ruby for which you are recompiling readline.


Update 2:

ruby 1.8.6 and ruby 1.8.7 appear to have their threads (and signals) blocked by readline now. This seems to be a recent development as I have used readline with ruby 1.8.6 without blocking threads in a past build.

Ruby 1.9.1 doesn't currently block threads with readline


Update 3:

I have updated this entire post, consolidating and simplifying the content. Then I renumbered the updates. My blog is not immutable.

6 comments:

Randall Sutton said...

I have swithed from macports to homebrew because I like how it works better. It has readline. I just did brew install readline and got 6.0. Nice post!

http://github.com/mxcl/homebrew

Tim Harper said...

Been using brew for some time now. Don't know why it didn't occur to me to use it to install readline :P

I had to run 'brew link readline' to get it to link it so /usr/local/include/readline existed, though that's not entirely necessary as I can signal build scripts to find it elsewhere.

csexton said...

Thanks for the great post--worked like a charm. I had compiled my own ruby to get readline working, but this makes that overkill.

Roger Pack said...

pure ruby readline is another option I've used successfully in the past.

Adam R. B. Jack said...

Does your "update3 : block threads/signals" mean that it is best NOT to use readline, or look for a ure ruby readline? I would think so. Is there a way to test if this problem is occurring in my environment?

Marcus said...

Thanks! That helped me out quite a bit.