Last week I stole a ticket from my boss. It’s an interesting one involving streaming of XML data to another application. There are some issues with the concept and process, but it’s a legacy system and we can’t change it (other teams are busy building a replacement). We’ve also recently upgraded this project from Rails 2.2 to 2.3 and switched from Mongrel to Passenger. All this means I’ve been tasked with refactoring all of this streaming code from a custom Mongrel handler to a normal ActionController setup utilizing a render call with a Proc in the text attribute.

The XML streaming is actually a search API for our ETL system. It queries for very large chunks of data. The streaming requirement has a few positive side effects, it allows the extracts to take place quicker and it helps us run our queries and instantiate our models in batches, keeping our RAM usage under control. Check out the really simple example below.

def search
    response.content_type = Mime::XML

    render :text => Proc.new{ |resp, out|
      out.write ""
      1.upto(10){ |i| out.write "" }
      out.write ""
    }
end

I had the migrations from the Mongrel Handler to a regular controller mostly done so I decided it was time to get some tests together. I setup the first test, the simplest thing I could do was ask for the default set. It looked something like this:

test "default" do
    get :search
    assert_response :success
    assert_select "element", 10
end

I hit cmd-r in Textmate and something was wrong, I got the following exception: TypeError: can’t convert Proc into String. After about 45 minutes of debugging I found the source of my problem, line 16 of HTML::Document. After thinking it over, it seemed like the appropriate place to make the update was around line 490 in TestProcess. The update is pretty simple: check to see if @response.body responds to call, if so run it with a StringIO object, eventually passing its contents to HTML::Document.new. Check out my ticket and patch at lighthouse. Read it over, check out the patch, run the tests and give it a +1 if you don’t mind.

Help me out here. I came across what I think is a bug in Rails core. It’s a really small issue, and an even smaller patch. Seriously, it’s only two lines of code, and three new tests — as you can see in the Lighthouse Ticket.

Oh, you want a description? The description I wrote in the ticket wasn’t enough? Fine, I’ll expand a bit here. First, let me state that this issue was only uncovered due to our semi-unorthodox deployment strategy. I say semi-unorthodox because I’ve never read of anyone else using a similar style, but I don’t see an alternative.

Here at work we build apps for two sets of users: the public and our business team. These sets of users see totally different applications; in fact, they aren’t even running on the same boxes. Think of our internal tool as a CMS and Customer Service tool. Instead of writing two applications and making them share models, we have one larger application but can control how it reacts by changing the RAILS_ENV. Our basic configuration contains three environments: development, production and admin (this is a simplification: we also have multiple test environments, but they are unimportant for the sake of this discussion).

To limit what can take place on each environment we have segregated our routes.rb like you see below.

# back end services
if %w(admin development test).include?(RAILS_ENV)
    path_prefix = (RAILS_ENV == "development") ? "admin" : "")
    map.namespace(:admin, :path_prefix => path_prefix) do |admin|
      admin.connect "/applicants/:id", :controller => "applicants", :action => "show"
    end
end

# front end site
if %w(production development test).include?(RAILS_ENV)
    map.root :controller => "applicants", :action => "new"
end

Let’s go over this routing real quick. You’ll immediately notice two blocks, the first being for “back end services” (i.e.: internal business users), and the second for the “front end site” (i.e.: the public). If you look closely you’ll see the only difference between the conditionals for each block is that the first includes the “admin” environment while the second uses the “production” environment.

The issue I ran into stems from line #3. When path_prefix is blank, RouteBuilder generates improper URLs, therefore affecting everything inside the admin namespace.

• When path_prefix is set, the admin URLs look like: /admin/applicants/1
• When path_prefix is blank they look like: //applicants/1

That extra slash is a bit of a problem with the way we deploy our “admin” environment. We use some Apache fu to map http://admin/project to the root of the application, meaning our internal users need to go to http://admin/project//applicants/1. This URL is clearly incorrect and should never be generated.

Without the path_prefix we are severely limited in our ability to run the full project in development mode. With the path_prefix set in “development” mode we can view the public portion of the site at http://localhost:3000/applicants/1 and the private part at http://localhost:3000/admin/applicants/1. It should be noted that these two routes go to two completely different controllers, one being in the global namespace, and the other existing inside an Admin module.

Now I know if you payed attention you’re probably thinking: “Andrew, you can get around this with a simple refactor, maybe try this…”

path_prefix = (RAILS_ENV == "development") ? "admin" : nil)

Yes, you’re right, that would work: having the path_prefix set to nil is a simple solution. But, a bug is a bug, and adding an extra slash to a URL isn’t an appropriate thing to do. So please, if you made it this far and you don’t think I’m a raving loon then +1 my patch.

Nah, probably not, but I do have a little something to brag about. A certain Rick Olson has kindly accepted a second patch of mine. Like the first couple of commits he merged, this new one is pretty small. It’s a simple update to the MasterFilter class in his Masochism plugin.

Masochism is a pretty specialized plugin, but extremely useful, small and easy to work with. In short Masochism provides a simple way to use a master/slave database configuration from a Rails project. It does this by delegating to different ActiveRecord connections based on the type of SQL you are generating, with reads going to the slave and writes to the master.

It’s also nice enough to provide a few extras classes under the ActiveReload namespace.The first two are abstract ActiveRecord::Base classes, MasterDatabase and SlaveDatabase, that will force the models inheritting from them work entirely in the implied database. The third is MasterFilter, a class designed to be used as an around filter in a controller. When this filter is executed your models will make sure to do all reads and writes from the master database.

Tuesday, December 2nd my company laid off about 10% of its employees, 13 people. It was a hard day for everyone, we watched a lot of great people leave. To make it ever worse 2 of the 13 were fellow developers, both highly skilled (as some would put it, ninjas) and all around cool guys. I wanted to do something to help, everyone did, but this came down from high above and none of us, no matter how much we wanted, were in a position to change it.

Not knowing what to do I came in to work like normal today to find a new blog post by one of my coworkers, Kumar McMillan. He put up a post, similar to this, letting his readers know of these 2 highly talented developers here in Chicago who are probably looking for a job. I almost smacked myself for not having this idea on my own, but instead decided to write this entry. If any of my readers (all 6 of you) know places hiring Ruby, Python, Javascript, developers (in fact I’m sure both these guys are skilled in far more than those languages) let me know, send me an email or find me on LinkedIn or at Working With Rails.

When working on larger projects the RAILS_ROOT/app folder structure can be a little restrictive. Lately I’ve taken to adding more sub-directories into RAILS_ROOT/app like, observers, mailers and presenters (check out ActivePresenter and Presenter Pattern if you’ve never heard of it). It’s a simple trivial addition to the folder structure that allows me to keep my sanity. Instead of sifting through 40+ files in the models folder I can quickly drill down to what I’m looking for.

Luckily, Rails is nice enough to have a simple mechanism to do this. Open your environment.rb file and look for the lines that read:

  # Add additional load paths for your own custom dirs
  # config.load_paths += %W( #{RAILS_ROOT}/extras )

Simply un-comment the second line and replace it with something like this:

  # Add additional load paths for your own custom dirs
  config.load_paths += %W( #{RAILS_ROOT}/app/clients #{RAILS_ROOT}/app/observers #{RAILS_ROOT}/app/presenters #{RAILS_ROOT}/app/mailers )

Very similar to moving the Ruby files around, I have also taken to moving some of the views around, specifically, mailer views. I like to keep them near the mailer model, and not in the RAILS_ROOT/app/views path.

This change is also quite easy thanks to the extensibility of the Rails framework. Near the end of your environment.rb file add the following line:

  config.action_mailer.template_root = "#{RAILS_ROOT}/app/mailers"

I finally got around to using a mocking library today at work. I chose Mocha to start, no real reason, just the first name to pop into my head. After about 15 minutes (10 reading docs, and 5 coding) I am in love. Seriously, testing has never been this easy. What would have taken me at least 30 minutes to add new fixtures wihtout breaking old tests or screwing anything else up was easily solved in one to 2 lines per test case.

class SupportControllerTest < ActionController::TestCase
  def test_applicant_activated_view
      Applicant.any_instance.expects(:activated?).returns(:true)

      post :create, :applicant => {:last_name => "Leapfrog", :email => "sos_platform@leapfrogqa.com"}
      assert_select "ul.activated"
  end
end

The first line in the above method mocks out the activated? method on any instance of the Applicant class. Isn’t that simple and clean? Without seeing the rest of the business logic I know it might be hard to accept why I need an entire library to help me with this simple boolean method, but trust me, activated? method is actually delegated to another model which is using an ActiveResource object to truly determine the result.

Recently an old friend, David Moffitt, asked me to tweak some Javascript I wrote for him and his company GTTS over a year ago. Looking at it again after such a long time was scary. The code was terrible. All functions, no classes, just a mess. I rewrote it for them as a Prototype based Class. Very clean and easy.

The Scroller object creates a simple news ticker that nicely rotates messages, stops when you mouse over, and continues when you mouse out. Yep, thats it. Check out my github project. View the comments in scroller.js for directions on how to use it.

Why the almost empty github project? The theory is I might one day write more Javascript and need a place to put it. So for now, only one file in that “project” but just you wait.

The Mack Framework is a pretty nifty combination of Rails, Merb and many other Ruby frameworks. It’s built off Rack and therefore should be simple to deploy. I really want to work with it, but what to do? Does anyone have any ideas? I need a project.

From a recent IM conversation with a good friend of mine:

Andrew Bloom
12:51 i hate working on shitty codebases
12:51 i wish i had time to do a major refactoring
Stephan Mokey
12:52 yeah, code is never done, it’s just due
12:52 we are artists

Recursion, every programmers favorite topic. Luckily that’s not what this is about, really. Today at work I was writing a Rails Generator that needed to created <a href=”http://www.ruby-doc.org/stdlib/libdoc/erb/rdoc/”>ERB</a> templates for views. It sounded pretty simple, but I hit one stumbling point. How do I make my template an ERB template that generates an ERB template of its own without interpreting all of the tags? After a bit of Googling I started to get discouraged. Then I remembered Rails already does some of this in places like the scaffold generator. I opened up the source and the templates and found an ERB tag I had never seen before <code><%%= do_something %></code>. There it was, the ‘%%’ will tell ERB to treat this as a tag for a future run. It will remove the second % and leave you with the output you expect.