JRuby on Rails on Gentoo

After adding JRuby 1.0.0 RC2 to the tree, I began poking around using it to run a little blog I’ve been writing in rails. Surprisingly, the process was quite easy. I just had to package the JDBC wrapper for ActiveRecord.

So here are the details, assuming you are using mysql:

# emerge >=jruby-1.0.1_rc2-r1 activerecord-jdbc jdbc-mysql

I added quite a few changes to -r1, including the ability to use gems and other ruby packages already installed on the system. To do this, there is now a symlink from /usr/share/jruby/lib/ruby/gems to /usr/lib/ruby/gems, and the same for site_ruby as well. If you had an earlier version of jruby, they used to be a separate directories, where you’d have to install gems again. If that’s the case, you’ll see a warnings, and eventually an error that you need to move /usr/share/jruby/lib/ruby/gems and /usr/share/jruby/lib/ruby/site_ruby out of the way before you can continue.

After things have successfully merged, we are ready to add the appropriate configuration. I’m going to assume here that you have already have a rails app to work on, and using mysql for a database. If not, there may be a trillion or so places on the interwebs that explain how to get going.

First, we add need to add a bit to the config/environment.rb:

def using_jruby?
  RUBY_PLATFORM =~ /java/
end

if using_jruby? 
  require 'rubygems'
  RAILS_CONNECTION_ADAPTERS = %w(jdbc)
end

This can be placed after the require File.join(File.dirname(__FILE__), ‘boot’) line. If there’s a better place using_jruby? should be defined, please let me know.

Now we wander over to config/database.yml. Here is what was originally there:

development:
  adapter: mysql
  socket: /var/run/mysqld/mysqld.sock
  database: yourapp_development
  username: root
  password:

Now we update it to use:

development:
  <% if using_jruby? %>
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://localhost/yourapp_development
  <% else  %>
  adapter: mysql
  socket: /var/run/mysqld/mysqld.sock
  <% end %>
  database: yourapp_development
  username: root
  password:

So, if we’re using jruby, we use the JDBC adapter, with the appropriate driver, and point it at the right URL. Otherwise, it’ll use what we had before.

You’ll likely also want to apply this to the test and/or production databases.

One last thing we need to do is make sure that the jruby gets jdbc-mysql on the classpath. The simplistic way to do this would be to just:

$ export CLASSPATH=$(java-config --classpath jdbc-mysql)

That works for testing, but I want something a little more permanent. So now, we use a relatively undocumented feature of java-config.

$ mkdir -p ~/.gentoo/java-config-2/launcher.d/
echo CLASSPATH=\$(java-config --classpath jdbc-mysql) \
> ~/.gentoo/java-config-2/launcher.d/jruby

/usr/bin/jruby, and most Java applications on Gentoo use a launcher utility we wrote, to help with building classpaths, invoking the right class, and so on. What this snippet did was create some custom configuration for the jruby launcher, which would get used whenever you invoke jruby. In this case, we add jdbc-mysql to the classpath.

Now we should be all ready to actually use it.

$ jruby script/server

You should now be able to hit your application at the usual location. Keep in mind that unless you invoke Note: you’ll always have to invoke it with jruby, or alternatively, you can update the #! line in script/server to be “#!/usr/bin/env jruby”.

One issue I have run into is that things freak out when running unit tests, because it “can’t find an appropriate driver”, which doesn’t make sense when it works fine for development. Here’s a snippet:

$ jruby /usr/bin/rake  test
-----SNIP-----
1) Error:
test_create(BlogPostsControllerTest):
RuntimeError: The driver encountered an error: java.sql.SQLException: No suitable driver
    /usr/share/jruby/lib/ruby/gems/1.8/gems/ActiveRecord-JDBC-0.3.1/lib/active_record/connection_adapters/jdbc_adapter.rb:208:in `initialize'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/ActiveRecord-JDBC-0.3.1/lib/active_record/connection_adapters/jdbc_adapter.rb:10:in `new'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/ActiveRecord-JDBC-0.3.1/lib/active_record/connection_adapters/jdbc_adapter.rb:10:in `jdbc_connection'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/connection_specification.rb:262:in `send'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/connection_specification.rb:262:in `connection='
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/query_cache.rb:54:in `connection='
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/connection_specification.rb:230:in `retrieve_connection'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in `connection'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/fixtures.rb:247:in `create_fixtures'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/fixtures.rb:593:in `load_fixtures'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/fixtures.rb:538:in `setup_with_fixtures'
    /usr/share/jruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/fixtures.rb:575:in `setup'
    ./test/functional/blog_posts_controller_test.rb:17:in `setup'
-----SNIP-----

Lastly, here are some resources that may be useful:

  • http://www.headius.com/jrubywiki/index.php/Main_Page
  • http://www.headius.com/jrubywiki/index.php/JRuby_on_Rails
  • http://www.headius.com/jrubywiki/index.php/Running_Rails_with_ActiveRecord-JDBC

Aiding and abetting the enemy, rails 1.2.2, and #gentoo-ruby

I must admit, I’ve been experimenting with Ruby on Rails with increased frequency of late. Being heavily invested both in Java on the Gentoo and professional fronts, this seems almost like… treachery :>

I must say, it has been quite refreshing. It is quite a lovely development platform. I won’t go into details about a comparison to developing on the typical Java platform, as I’m sure there are plenty of those on the interblarg already 🙂

I decided this past weekend that would try to help out with Ruby on Gentoo, and so joined the ruby herd.

My first act was to bump dev-ruby/rails from 1.2.1 to 1.2.2. If you happen to be using it in any projects, you’ll likely want to update RAILS_GEM_VERSION to 1.2.2 in your config/environment.rb.

Lastly, pingu and I are trying to get some interest in #gentoo-ruby going. According to Flameeyes, people have tried in the past, but apparently it didn’t quite stick. Let’s give it another try, shall we? If you’re working with Ruby on Gentoo, feel free to stop by #gentoo-ruby on irc.freenode.net.

Update on Xfce 4.4 unmasking

To follow up my previous post about unmasking Xfce 4.4…

Since then, I went through the process of upgrading from 4.2 to 4.4. Found a few kinks, mostly with files collisions caused by files moving from one package to another. These have been since worked out.

The major roadblock now is a number of broken dependencies for a few archs. These issues were conveniently glossed over while things are masked, but making repoman pretty upset the second they are unmasked.

So right now, all these issues have bugs filed for them, and now we play the waiting game. See the tracker for the latest breaking news.

unmask teh Xfce!!!one

The question of when Xfce 4.4 will be unmasked has been coming up with increasing frequency.

The default answer is… wait, for it, when it’s ready :>>

A more elaborate answer follows:

We have established a ‘tracker’ bug here. The team has been pretty good about keeping on top of bugs that come up with it, as in, there are no known issues currently.

So that means it’s ready, right? Not quite. Below are the outstanding issues I see:

In making haste to get a 0-day release of 4.4, some corners may have been cut. In particular, some of the xfce-extra packages are lacking the appropriate keywords, even though xfce-base/xfce4-extras depend on them. This is only for new xfce-extra packages, which have been added only relatively recently.

Icons seemed to have moved around a bit, as in, which packages owns them. This is a bit problematic, as it causes file collisions (enabled by FEATURES=”collision-protect”). While not critical to unmasking, it is pretty key for stabilizing, and the sooner it can be fixed the better.

The other things I’m concerned about is the upgrade path from 4.2 to 4.4. Some things that were extras in 4.2 are now part of 4.4. As a result, some xfce-extra packages have blockers against them. In other cases, xfce-extra packages haven’t been updated for 4.4, or otherwise are no longer maintained. These xfce-extra packages have blockers against them as well. The consequence of these point is that you will have to unmerge the packages that are reported as being blocked.

XFCE all up in your face

After being in beta and release candidates for several months, XFCE 4.4 is finally released. Not only that, but ebuilds for them are willing and ready to go. woot.

All the extra panel plugins and extra utilities have been updated as well. In addition, there have been some new additions for 4.4. Use xfce-base/xfce4-extras to get them all, or choose them piece meal out of xfce-extra/.

We’re also in the process of establishing an XFCE project for reals, instead of just being having a herd. With the creation of a project, you need a project lead. Who will it be? Well, the consensus seems seems to be your truly.

Dealing with libraries that like breaking their APIs

I’m currently working on making use of a Groovy on a project of mine which uses Hibernate and Spring among other things.

My code wasn’t particularly well tested in terms of unit tests. By this, of course, I mean there were no unit tests.

So like a good code monkey, I decided to get some unit tests going, verify they work, then move code over to use Groovy. The benefit here being that by having these tests, I can verify that things still work as expected after the migration.

I figured on trying out rMock, one of the many mocking libraries out there for Java. I get my first test case written, cross my fingers, and hit the run button (this is in Eclipse)… and what do I get, but the infamous red bar. NoClassDefFound… WTFBBQ?!? For whatever reason, it wasn’t finding certain classes from dev-java/asm.

I won’t go into further detail about how I found this problem, and found the cause, but here are my findings (versions here refer to slots):

  • dev-java/groovy uses dev-java/asm-2.2
  • dev-java/hibernate uses dev-java/cglib-2.1
  • dev-java/rmock uses dev-java/cglib-2.1
  • dev-java/cglib-2.1 uses dev-java/asm-1.5

The issue here is that asm apparently changed it’s API quite a bit between 1.x and 2.x. This problem seems to have cropped up a few times at least according to google (search for ‘NoClassDefFoundError: org/objectweb/asm/CodeVisitor’).

The asm tutorial even goes so far as to the changes… kind of. See here under the “Changes in the ASM 2.0 API Since ASM 1.x” section. A few things bother me about this.

  • I don’t see why the major changes, like renamed classes, and changed method signatures, couldn’t have been deprecated at first, and do some twiddling to proxy calls to deprecated things to the newer ones.
  • The line that goes like “In general, it would be a good idea to run tool like JDiff and review the differences between the ASM 1.x and 2.0 APIs.” Maybe it’s just me, but this seems a bit presumptious. I mean, while such a tool is probably useful, it is far from thorough. It might point out where the API changed, but would have no clue as to the significance of those changes, or how you would account for them.

A question that might come out is why would rmock expose this problem, when hibernate is using the same version of asm? Presumably it is because hibernate isn’t exercising the parts of the cglib API that use the offending parts of asm, so I must have lucked out before now.

Regretfully, this seems somewhat common in the Java world. Maybe not very common, or even common, but common enough that they stick out in my mind, causing a handful of angst and frustration.

There is some good news though. cglib-2.2_beta1 seems to have been updated to use the asm-2.x’s API. Another bit of fortune is that it’s unlikely that anything in the tree will run into this particular snaggle, just code monkeys like me using these various libraries.

For the record, I don’t get the ClassDefNotFound anymore, but instead I get a lovely NullPointerException from down in Groovy. Oh well, one step forward, one step back.

Java gets Groovier

This news is a bit belated by a few days but…. over the weekend, I spent some time getting the recently released Groovy 1.0 into the tree.

In case you hadn’t heard about it, Groovy is, among other things, a dynamic language that runs on the the Java VM. Some interesting data points about it:

  • It’s dynamic, for reals.
  • It’s about 98% syntax compatible with Java code (I made that number up, but really, it’s pretty durn close).
  • Java bytecode gets generated at runtime, no need to compile.
  • Groovy can generate Java bytecode to your typical .class files

A consequence of these points is that you can easily migrate from Java to Groovy, and have your project running exactly as it was. Then slowly, you can get into the swing of things, and take advantage of everything that Groovy has to offer as you get the hang of it.

One fear I’ve noticed of Java developers of jumping into a more dynamic language like python or ruby is late binding. You can’t really be sure the type of your objects until runtime. Sure, you can read the documentation (assuming it’s up to date) or peek at the code to be sure. With Java, binding occurs at compile time, so you can be sure that foo method expects a Bar, or that Bar has the aieeeee() method. I suspect some of this comes from defensive coding, where you become concerned about how your code will be (ab)used by others.

To combat this, a coworker and I have this idea… You use Java to define interfaces for the, uh, interfaces you care about. You can then use Groovy to implement these interfaces. By doing this, you get the best of both worlds: clients of your code get their strongly typed interface, and you get to do implementations with a more expressive syntax.

Overall, I’ve been pretty impressed with Groovy so far. Granted, I haven’t used it as deeply as possible yet, but it certainly has a lot of potential.

In case any of this sounds interesting, here’s some resources:

And in other Java dynamicism news, I’ve added the freshly minted JRuby 0.9.2 to the tree this past weekend as well. I haven’t played with it yet, so I can’t speak much on its behalf other than it’s an implementation of Ruby written on the Java VM. Perhaps that can be the topic for a future blog post.

Musings of the current state of open source Java

[Note: I’m actually referring to open source Java software and libraries, not Java itself]

I’ve been maintaining and working with Java packages for pretty close to a year and a half now. And I hate to say it, but it is only becoming more apparent that things are less than ideal.

Sure, there are a multitude of projects out there, great projects. Eclipse, Tomcat, Netbeans, Azureus, all of the Apache Jakarta project, and many more. For a developer, it’s great.

For a packager… it’s not so nice of a picture. Builds packages vary from project to project with some kind of unholy glow. While there are some defactor standards out ther e, ie using ant or maven, it ain’t pretty.

How do you build your javadocs? Is the target named doc, docs, javadocs, javadoc, or api?
How do you manage your dependencies on third party libraries? Include every jar under the sun, bundle the source in with your own, or download the from the interweb?

The latter of these is particularly irksome. As I said, the standard practice is generally to include all your dependencies. I recall reading an O’Reilly book on ant at some point, and they actually encourage this behavior. I seem to think they said something along the lines of “You can’t trust your user to use the right dependencies, so you should include them so that things will definitely work.”

I’m fairly sure most of this be traced back to influences of the Windows world. I mean, there really isn’t a concept of dependencies on other packages. You just distribute your application in a giant blob, and be done with it.

While I’m sure it’s convenient if your developing the application or distributing a single blob to a user, it certainly is a pain to package.

So for all the dependencies, we want to package the dependencies, and then package the dependencies’ dependencies, and so on. Then we want to make the application use our copy of the dependency that we just packaged. Easy enough, right?

Not quite. Here are a few of the many problems we come up against all too regularly:

  1. The API of the dependencies are unstable, as in, they break between minor revisions.
  2. The dependency has been patched in fun and exciting ways that are specific to the application.
  3. The origin of the dependency is unknown, and no upstream can be found.

Quite dreary, eh?

Fortunately, it is not always this bad. There are, of course, good applications and libraries which are sane enough to package. Maven, in some senses, has been helping improve things, by standardizing the way things are being built, and by providing a standard mechanism for getting dependencies. A step in the right direction, at the very least.

Stay tuned next time for some ideas for addressing these concerns…

Java she wanted, Java she got

It is with great pride that I am able to announce the ‘new’ Java system is now stable on all archs where we support Java! That means amd64, ia64, ppc, ppc64, and x86.

In case you’ve been stuck in a cave somewhere, and hadn’t heard anything about the new system, here are some features highlights:

  • Changes to user and system VM happens instantly. No more need to run env-update && source /etc/profile!
  • One-time changes to the VM by using GENTOO_VM, ie GENTOO_VM=kaffe ant would run ant using kaffe, instead of the user or system VM
  • Packages no longer depend on the system VM being set properly. This means that an appropriate VM will be used for building for packages. A package needs 1.5 or later? No sweat! Only builds with 1.4? You got it!
  • Now you can use all the Java 1.5 goodness you want with impunity!
  • We’ll be able to support Java 1.6 when it comes out in Decemeber in a much more reasonable amount of time.
  • Support for configuring your Java browser plugin using eselect.
  • Support for configuring your VM using eselect.

While this may seem like a small list, it is a significant improvement over the old way of things. And this list is of user-facing changes… there are also vast improvements which help a lot of the developer side of things (ie making things much more maintainable).

[edit]
On a side note, if you have been using the new system for a while, there are a few things you may want to do:

  • Make sure you’re not using the old migration-overlay. This will cause some problems if you try to emerge VMs among other things.
  • Remove the package.mask entries if you had used them
  • Remove the package.keywords entries if you had used them

In other news… I’ve been always been a bit quiet on the blogging… but, I hope to change that, at least a little bit. I have some ideas for some writeups to show off Gentoo as a Java development platform… so stay tuned 😀