Arduino + RAD Looks Like Fun

Lately I’ve been lacking any personal projects. To compensate I finding myself working on code for my employer (Leapfrog Online) during my free time. That’s not very fun, but my lack of creativity wasn’t able to provide a solution.

All of that changed tonight. For some unknown reason I suddenly came up with a project. I’m not going to say what the actual project is yet, but I will say I’ve been doing a lot of reading on the Arduino, and was thrilled to find this: RAD. I will probably end up ordering one of these starter kits when I get back from RailsConf in a week.

I went to RIT for Computer Engineering, and I’ve always loved working with hardware as a hobby. Since I’ve been out of school I really haven’t done much with my soldering iron or bit mask knowledge, mostly Rails work, so this will be a fun change.

More Testing Thoughts - When Mocks Can’t Help

The other day I found myself needing to test a class (this seems to happen a lot), but I ran into a sticky situation. I wanted to test the validity of an instance method, easy enough, and then later wanted to test another instance method that relies on the first. The problem lies in the fact that the first relies on an external service that doesn’t provide stable data, making it difficult, if not impossible to test.

A little background on the code you’re about to see. It is quite common that I must consume XML from internal services, but I have no way of knowing how much data will be returned with each request (as these feeds rely on other services). Downloading tons of XML and iterating over it (and performing actions based on the data) is slow, cumbersome and can cause annoying situations if it crashes in the middle. To help with scalability (the buzzword of the year!) the XML feed listens for 2 query string parameters, start_event and limit. Knowing this I built a small iterator object to wrap a Hpricot instance and fetch XML with only 100 rows, and to repeat until it has reached the end.

Boring business logic and methods have been removed from the following sample.

class XmlIterator
  include Enumerable

  ...

  def each
    while data = get_xml(build_url)
      results = (data/@search_term)
      results.each{|x| yield x}

      @start_event = (data/'activations').first['end_event']

      break if results.length < @limit
    end
  end

private
  def build_url
    query_string = "start_event=#{@start_event}&limit=#{@limit}"

    return "#{@path}?#{query_string}"
  end

  ...
end

As you can see, I’ve built my own enumerable object, but the each method is really a proxy back to a Hpricot doc object (produced in the get_xml method I have conveniently removed). It will fetch 100 rows, process them, and then fetch 100 more, and continue until the result set is returning less than 100, meaning it was the last “page” of data.

Now for the testing. Validating build_url was pretty simple, set some instance variables, call the method and compare the returned string. But what about each? I don’t want to hit the service to test the each method, it would really be great if I could have it look at a local file in my mocks directory. All I really care to test here is that the proxying of the iteration works as expected. I struggled with how to do this for a few hours.

My initial reaction was to mock up the HTTP service using something like WebBrick, but that sounded like a lot of work, and a maintenance nightmare in the future. I knew the solution was to change the build_url method, but how can I do that without overriding it completely, as I need to test the original implementation. This meant placing a file in test/mocks/test was out.

Then it hit me, why not just change the build_url for that one test. There are 2 ways to go about this (that I know of). Both can be seen in the example below:

require File.dirname(__FILE__) + '/../test_helper'

class XmlIteratorTest < ActiveSupport::TestCase
  def setup
    @iterator = XmlIterator.new("http://www.madeup.com")
  end

  def test_each
    def @iterator.build_url
      "#{RAILS_ROOT}/test/mocks/test/activations.xml"
    end

    ...
  end

  def test_each_again
    @iterator.instance_eval do
      def build_url
        "#{RAILS_ROOT}/test/mocks/test/activations.xml"
      end
    end

    ...
  end
end

Both @iterator.instance_eval and def @iterator.build_url will produce the same result, a new version of build_url for that specific instance of @iterator, it’s just a matter of preference which you chose to do. Personally, since it’s just one method I’m redefining I like the first version, but if I was overriding 2 or more I’d probably go with the second.

Logging in Rails

When you’re living in the Active* world of Rails (ActiveRecord, ActionController, etc.) you always have access to the ubiquitous logger method. But step outside and you’re stuck dealing with the awkward, and uncomfortable RAILS_DEFAULT_LOGGER global. Often time’s I find myself taking one or both of the following approaches depending on the class I’m writing.

class MadeUp
  class << self
    def logger
      RAILS_DEFAULT_LOGGER
    end
  end

  def logger
    RAILS_DEFAULT_LOGGER
  end
end

Now I know, neither of the methods are complex or long, but how often do you want to type that same handful of lines (and doesn’t it just make your code ugly)? Enter my latest and greatest contribution to the world: Loggable.

module Loggable
  module ClassMethods
    def logger
      RAILS_DEFAULT_LOGGER
    end
  end

  module InstanceMethods
    def logger
      RAILS_DEFAULT_LOGGER
    end
  end

  def self.included(receiver)
    receiver.extend         ClassMethods
    receiver.send :include, InstanceMethods
  end
end

Object.send :include, Loggable

Remember to require it somewhere in your code.

Mephisto to WordPress

This is the first post on the new blog. I’ve completed my move to WordPress from Mephisto. I know what you’re saying. Andrew, you’re a Rails developer, what gives?

It’s simple really, I never did much tweaking inside Mephisto. My hosting company, Media Temple provides a pretty good package called the Grid Server, but one downside is the small amount of memory provided. Because of this I can only run 2 small Rails apps at a time on my share, but they don’t count memory from PHP apps against my space.

I’m sure you see where I’m going by now. Mephisto and Mongrel took up a bunch of memory and responded kind of slow (Grid Server is a shared hosting environment after all). WordPress is quick, clean and has quite a few more features. It’s a bit more complex but the user community is quite large. I’ve managed to migrate my articles, add Google Syntax Highlighter, Google Sitemap, and Google Analytics along with some simple PHP tweaks to this cool theme over the course of 2 short nights.

Postgres Query to CSV

Today at work I was asked to generate a report from some data we’ve been recording for the past month. The query I came up with ended up producing ~3000 rows. Far to many to copy and paste into a text file and email to anyone. I knew there had to be a way to produce a CSV from PSQL, but how? Googling returned a number of informative links, but I was still a bit confused.

I ended up asking my fellow devs, and received a few possible solutions.

The first was to start PSQL with the the following flags:

psql -A -F , *connection specifics*

The second was to enter PSQL like normal and then apply these two commands:

\a
\f ,

Both suggestions 1 and 2 require you to also run the following command before running your query:

\o filename.csv

The third, and by far most elegant solution came from one of our DBAs. He recommend entering PSQL like normal and then using the following command all in one fell swoop:

\copy (select statement) to '/my/file/name' with delimiter AS ',' NULL AS ''

It is worth noting that the previous command will only work in Postgres >= 8.2

I’m posting this here in hopes others searching for this will find this a clear succinct description, but also, for my own reference as I’m sure I’ll be asked to perform something similar again in the future.

Dead Simple Mocks For Unit Tests

Yesterday I was testing a Helper (module) I had written for a Rails app I am working on. I wanted to test the helper directly, and not deal with the whole controller and request/response loop just to verify some simple methods. So I started with something like the following:

class TrackingHelperTest < ActiveSupport::TestCase
    include TrackingHelper

    def test_record_event
        ...unimportant...
    end
end

I ran into a problem almost immediately. Some of the methods in this Helper required access to the request object. At first I thought I went down the wrong path, and would need to test this Helper in the context of a controller. Then it hit me, mock the request up. But do I really need a whole class to store and share some instance variables? After pondering this for a few moments I ended up with something like this:

class TrackingHelperTest < ActiveSupport::TestCase
    include TrackingHelper

    # Mocks
    MockRequest = Struct.new(:query_parameters, :path, :headers, :remote_ip, :path_parameters, :query_string)
    attr_accessor :request

    def setup
        @request = MockRequest.new({}, "/some/location", {"HTTP_REFERER" => "somesite.com"}, "1.1.1.1", {:controller => "some", :action => "location"}, "")
    end

    def test_record_event
        ...unimportant...
    end
end

Struct’s are simple to use and allow for quick and easy mocking in unit tests. Try it out in your code next time you don’t want to build a full class to do something simple.

Random Thoughts Involving Computers

This is the output from a cool command I found on ones zeros majors and minors

history 1000 | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' \
| sort -rn | head
124 rake
106 svn
84 ss
36 cd
30 git
26 rm
26 ls
16 sc
10 mate
5 psql

How about you?

And, again from ozmm is the try command. I think I’m going to start adding this to my projects, seems really simple and helpful.

class Object
  ##
  #   @person ? @person.name : nil
  # vs
  #   @person.try(:name)
  def try(method)
    send method if respond_to? method
  end
end

GitHub and Me

This weekend a fellow member of Rochester On Rails was nice enough to share an invite to Git Hub. Check out my account, I just published two little plugins I use all the time at work and at home when building Rails apps.

The first is Fixture Manager. This plugin adds a few methods to ActiveRecord classes allowing you to save database contents to YAML (either in your test/fixtures folder, or in cms/fixtures). The plugin also provides a few Rake tasks that wrap these methods as well as a simple static class (FixtureManager) giving you two options as to how you want to save and load your data.

The second is Rcov Task. This plugin contains almost no code, its a Rake task that simplifies running your test suite with rcov (and opening your browser if your in Mac OS X).

Sake and Database Shells

If you haven’t installed Sake yet, stop whatever it is you’re doing and get it running on your system.

Ok, with that done you’ll probably want some sake tasks to actually make this new tool useful. Below is an updated version of one my favorite tasks.

desc 'Launches the database shell using the values defined in config/database.yml'
task 'db:shell' => [ 'environment' ] do
  config = ActiveRecord::Base.configurations[(RAILS_ENV or "development")]
  command = ""
  case config["adapter"]
  when "mysql" then
    command << "mysql "
    command << "--host=#{(config["host"] or "localhost")} "
    command << "--port=#{(config["port"] or 3306)} "
    command << "--user=#{(config["username"] or "root")} "
    command << "--password=#{(config["password"] or "")} "
    command << config["database"])
  when "postgresql" then
    command << "export PGPASSWORD=#{config["password"]} && " unless config["password"].blank?
    command << "psql "
    command << "-h #{(config["host"])} " unless config ["host"].blank?
    command << "-p #{(config["port"])} ") unless config ["port"].blank?
    command << "-U #{(config["username"])} " unless config["username"].blank?
    command << config["database"]
  when "sqlite3"
    command << "sqlite3 #{config["database"]}"
  else
    command << "echo Unsupported database adapter: #{config["adapter"]}"
  end
  system(command)
end

With the above task you be in any RAILS_ROOT and type sake db:shell and be automatically placed into the appropriate database shell with your username/password/host/port setup already. You can even do sake db:shell RAILS_ENV=production.

I can’t take credit for this task, in fact I originally found it on the Sake Bomb blog post by Err The Blog. The link found in the comments on that page no longer works, but some googling turned up the source code elsewhere. I’ve added Postgres and sqlite3 to the mix as it only supported MySQL originally. If you know who the original author is please let me know as I would like to credit them.

Rails is great, but…

Don’t get me wrong, I really enjoy working with Rails, but every now and then something about it bothers me. Josh Susser of has_many :through recently blogged about the sometimes differing functionality of size, length and count in ActiveRecord.

Did you catch that, _differing_. When using the count method on association collections ActiveRecord responds differently based on pervious usage. When you’re working alone or on small code bases this might not be a big deal, you might never run into it, maybe you’d even be able to use it to your advantage. The same could be said for larger projects or working with multiple developers, but it’s far more likely to cause a hard to track down bug.

I understand Rails is an open source project controlled by people with their own interests at mind, so please don’t read this as an attack against them or the project in general. I am not the first person to mention these kind of anomalies in the framework, and probably not the first person to have this thought, but would it make sense to fork the project?

I’m not sure it’s worth forking Rails to make it more “enterprisey” (read: consistent). It would be great to have a stable platform, maybe it would receive better acceptance in the office (ie: maybe my python coworkers wouldn’t pick on it). My biggest concern is that removing all these idiosyncrasies will make it impossible to port future creativity from the original authors. Again, that wouldn’t be the end of the world if such a project could attract new development, but I just don’t see that happening. Any thoughts?

I think I’m going to keep a little list going of the things that irk me about Rails. Each time I come across one Merb gets more and more tempting. I really want to dig into Merb, I’ve yet to get do anything more complex than a few basic tutorials, I just need a project…