I’ve recently upgraded to OS X 10.5 (Leopard), and all-in-all, I’m pleased with the experience. My biggest issue has been the default stacks behavior—the icon changes to the last thing added to the stack, making visual identification unnecessarily cumbersome. I worked around this annoyance (as outlined here) by changing the sort to name rather than date added, and adding a dummy folder named “_1” that will sort to the top. For extra bonus points, I customized the icon of the dummy folder. For some yet unknown reason, the most recently downloaded item still peeks through from time to time, but it’s much better than before.

Maybe it’s my Windows history showing through, but I went with the “clean-sweep” erase and install method. For a non-developer, I’d probably recommend the upgrade (and in fact I used that method for my Father-in-law’s MacBook), but I had lots of custom bits scattered about my machine, and didn’t want to be chasing any incompatibility gremlins.

So now, to get my development environment set up on the new machine… Leopard includes a fairly complete Rails stack out of the box, with a non-broken Ruby, readline support, and most of the commonly used gems. Read more here.

MySQL was not included, but the latest installer (mysql-5.0.45-osx10.4-i686.dmg) for 10.4 from dev.mysql.com downloads worked (mostly) fine. The Server and the StartupItem install and operate correctly. The PrefPane installs, but does not appear to actually do … anything. I’ll have to work on that, but I can live without it for now. After a bit of manual hacking on my database dump file from Tiger (where I was running a 5.1.x beta of MySQL), all my databases are back in place.

One last piece that I needed for my Rails apps—RMagick. I know it’s possible to install RMagick and its dependencies, um, “autoRMagickally” via a package management system like MacPorts or Fink, but I prefer not to. For some background on why not, you can read this article at hivelogic. The last time I was rebuilding my laptop and desktop near the same time, I put together a shell script to automate the process of installing RMagick. I got it back out and dusted off the cobwebs, and voila! RMagick on Leopard. (note: replace wget with “curl -O”, if you don’t have wget installed on your machine) Here’s the code:

#!/bin/sh
wget http://download.savannah.gnu.org/releases/freetype/freetype-2.3.5.tar.gz
tar xzvf freetype-2.3.5.tar.gz
cd freetype-2.3.5
./configure --prefix=/usr/local
make
sudo make install
cd ..

wget http://superb-west.dl.sourceforge.net/sourceforge/libpng/libpng-1.2.22.tar.bz2
tar jxvf libpng-1.2.22.tar.bz2
cd libpng-1.2.22
./configure --prefix=/usr/local
make
sudo make install
cd ..

wget ftp://ftp.uu.net/graphics/jpeg/jpegsrc.v6b.tar.gz
tar xzvf jpegsrc.v6b.tar.gz
cd jpeg-6b
ln -s `which glibtool` ./libtool
export MACOSX_DEPLOYMENT_TARGET=10.5
./configure --enable-shared --prefix=/usr/local
make
sudo make install
cd ..

wget ftp://ftp.remotesensing.org/libtiff/tiff-3.8.2.tar.gz
tar xzvf tiff-3.8.2.tar.gz
cd tiff-3.8.2
./configure --prefix=/usr/local
make
sudo make install
cd ..

wget http://jaist.dl.sourceforge.net/sourceforge/wvware/libwmf-0.2.8.4.tar.gz
tar xzvf libwmf-0.2.8.4.tar.gz
cd libwmf-0.2.8.4
make clean
./configure
make
sudo make install
cd ..

wget http://www.littlecms.com/lcms-1.17.tar.gz
tar xzvf lcms-1.17.tar.gz
cd lcms-1.17
make clean
./configure
make
sudo make install
cd ..

wget ftp://mirror.cs.wisc.edu/pub/mirrors/ghost/GPL/gs860/ghostscript-8.60.tar.gz
tar zxvf ghostscript-8.60.tar.gz
cd ghostscript-8.60/
./configure  --prefix=/usr/local
make
sudo make install
cd ..

wget ftp://mirror.cs.wisc.edu/pub/mirrors/ghost/GPL/current/ghostscript-fonts-std-8.11.tar.gz
tar zxvf ghostscript-fonts-std-8.11.tar.gz
sudo mv fonts /usr/local/share/ghostscript

wget http://imagemagick.site2nd.org/imagemagick/ImageMagick-6.3.5-9.tar.gz
tar xzvf ImageMagick-6.3.5-9.tar.gz
cd ImageMagick-6.3.5
export CPPFLAGS=-I/usr/local/include
export LDFLAGS=-L/usr/local/lib
./configure --prefix=/usr/local --disable-static --with-modules --without-perl --without-magick-plus-plus --with-quantum-depth=8 --with-gs-font-dir=/usr/local/share/ghostscript/fonts
make
sudo make install
cd ..

sudo gem install RMagick

I am trying so see the result of a change in a Java program….5 minutes compilation…5 minutes deployment. Arggggggggggg!

Rails Rocks! Change + refresh = done!

Short term memory is good, hopefully I will forget that experience soon.

;-)

Our server that hosts onrails.org and time.onrails.org died on Sunday. Before we could react Slicehost migrated us to new hardware and everything was up and running. Thanks guys, awesome support!

The events where the following:

• [Slicehost] Sep 1, 2007 3:04 PM, Emergency Server Reboot and HW Migration – onrails
• [Montastic] Sep 1, 2007 3:20 PM, Website status: unreachable
• [Slicehost] Sep 1, 2007 3:44 PM, Emergency HW Migration #2 – onrails.
• [Montastic] Sep 1, 2007 4:12 PM, Website status: OK

[Slicehost] indicates emails we received from Slicehost, and [Montastic] emails from the monitoring system we use. Slicehost warned us of the situation and action they are taking before our monitoring system found out that the service was down. Well, apparently they migrated us to new hardware that had another issue (bad memory) and they moved us a second time to different hardware. Well no data was lost, all our service are up and running and I didn’t have to cut short my BBQ. Thanks!

Update1: Well something is wrong with the template of our blogs. I am not sure it’s related to the update of hardware as it was working last night. More to come.

Rails makes things easy on developers—maybe too easy. It’s not uncommon to reference an association while iterating over a collection of objects, resulting in a performance-devouring N+1 queries being executed. Of course, this is easily fixed with eager loading of the association, but you get my point. Rails can be a big gun, and it’s easy to blow off your foot.

Once your application is moved into production, it becomes important to keep an eye on performance over time in order to get a feel for trends and plan capacity intelligently. Numerous Rails performance measuring tools exist, but I find that it helps to correlate performance of your application with simultaneous lower-level performance metrics of your system (CPU and Memory usage, Load Average, etc). Besides that, hey! Pretty Graphs! Enter munin, an open-source, extensible monitoring tool with a number of out-of-the-box plugins that are useful to SysAdmin type folks. There’s a pretty decent (though brief) writeup on howtoforge that explains a bit more about how to go about getting your own munin. Go ahead and check it out—I’ll chill here until you’re back.

That wasn’t too bad, huh? Okay, so at this point you should have a working munin instance. It will take a few minutes of data collection before your graphs look like anything, but going forward, you’ll have historical graphs of several key metrics for managing your server(s). It’s outside the scope of this article, but you can also set up munin to monitor multiple servers on your network, and/or alert you when critical threshold levels are passed.

Well, knowing CPU usage is great, but it would also be nice to have some idea what the average user experience is like on your site. Does your Rails app perform differently at different times of day? How long does it typically take for your app to render a page? Is the majority of the time spent in the database, or rendering? Is the response time about the same between production deployment versions? I wanted answers to these questions too, so I came up with a small ruby program that watches your Rails log in realtime, and when munin asks, provides summary information about the performance of your application. Below, I’ll go over the code section by section; for those who want it now, scroll to the bottom of the article to find the download link.

The basis of the solution is a Ruby array: we stuff values into it, then compute the average of all the values and clear the array each time munin pings us. Every time a value is added, we also check it against the maximum already seen, so we can report the maximum response time in addition to the average. We keep three of these objects around, one each for DB, Rendering, and Total response times. There is also a mode that lets you look at the current value without consuming it—useful for peeking inside without affecting the data that munin will ultimately see. Here’s the class that implements this functionality:

class Accumulator
  def initialize
    @values = Array.new()
    @max = 0
  end

  def add(value)
    @values << value
    @max = value if value > @max
  end

  def average(read_only=false)
    return_value = if @values.length == 0
      nil
    else
      @values.inject(0) {|sum,value| sum + value } / @values.length
    end
    @values = Array.new() unless read_only

    return_value
  end

  def max(read_only=false)
    return_value = @max
    @max = 0 unless read_only
    return_value
  end
end

In the next section of the code, we build our accumulators, and begin tailing the logfile to extract performance numbers. This requires the file-tail gem, available from rubyforge. Note that in my setup, this file resides in a subdirectory under lib in RAILS_ROOT. If you choose to place this file elsewhere, you’ll have to adjust the path to the logfile accordingly. Another thing to note: in our environment, the load balancer continually pings a “heartbeat” action on each node to make sure it is still responsive. As we will be hitting this action repeatedly, it is engineered to be as lightweight as possible. Therefore, any numbers from it are pretty meaningless to our overall statistics, so we don’t want to include them. To keep these numbers from skewing our results, we define an IGNORE_PATTERNS regexp (earlier in the code). If the request matches a pattern we want to ignore, its statistics are not collected.

LOGFILE = File.join(File.dirname(__FILE__), '..', '..', 'log', "#{RAILS_ENV}.log")
$response_data = { :total     => Accumulator.new(),
                   :rendering => Accumulator.new(),
                   :db        => Accumulator.new() }

Thread.abort_on_exception = true
logtail = Thread.new do
  File::Tail::Logfile.tail(LOGFILE) do |line|
    if line =~ /^Completed in /
      parts = line.split(/\s+\|\s+/)
      resp = parts.pop
      requested_url = resp[/http:\/\/[^\]]*/]
      next if requested_url =~ IGNORE_PATTERNS

      parts.each do |part|
        part.gsub!(/Completed in/, "total")
        type, time, pct = part.split(/\s+/)
        type = type.gsub(/:/,'').downcase.to_sym

        $response_data[type].add(time.to_f)
      end
    end
  end
end

So now we have a thread busy gathering our data—how can we expose the data to a munin plugin? There are multiple ways to do this, but I chose to use a small HTTP server listening to requests from the local machine only. We could build such a thing in Rails, but we really don’t need about 90% of the features Rails has to offer. Since we’re running Rails as a Mongrel cluster, we already have a perfect tool at our disposal for writing small HTTP request handlers in Ruby: Mongrel. Here are a couple of pages about how to get started writing Mongrel handlers—it’s pretty straightforward. Here’s our handler:

class ResponseTimeHandler < Mongrel::HttpHandler
  def initialize(method)
    @method = method
  end

  def process(request, response)
    response.start(200) do |head,out|
      debug = Mongrel::HttpRequest.query_parse(request.params["QUERY_STRING"]).has_key? "debug"
      head["Content-Type"] = "text/plain"
      output = $response_data.map do |k,v|
        value = v.send(@method, debug)
        formatted = value.nil? ? 'U' : sprintf('%.5f', value)

        "#{k}.value #{formatted}"
      end.join("\n")
      output << "\n"
      out.write output
    end
  end
end

h = Mongrel::HttpServer.new("127.0.0.1", PORT)
h.register("/avg_response_time", ResponseTimeHandler.new(:average))
h.register("/max_response_time", ResponseTimeHandler.new(:max))
h.run.join

The handler is generic so that it can call an arbitrary method on our collection of data arrays, so we can set up one URI for the average, and one for the maximum. With this in place, we can write a simple munin plugin script that uses Net::HTTP to query our Mongrel to get at the performance data.

Basically, a munin plugin has two usage scenarios. When called with the single argument “config”, it should output information about itself in a format that Munin understands. This includes how to label the chart, scaling information, how many series will be included on the chart, etc. When called with no arguments, the plugin should output the current values of each series. For more information on writing your own munin plugin, start with the HowToWritePlugins munin wiki page. And now, our plugin script:

#!/usr/bin/env ruby

# munin plugin to render rails response time graphs
# link to /etc/munin/plugins/avg_response_time and /etc/munin/plugins/max_response_time

require 'open-uri'
PORT = ENV['PORT'] || "8888"

def config
  title = File.basename($0).split('_').map{|s| s.capitalize }.join(' ')
  config=<<__END_CONFIG__
graph_title #{title}
graph_vlabel response time
graph_category rails
total.label total
rendering.label rendering
db.label db
__END_CONFIG__
  puts config
end

def get_data(read_only=false)
  qs = read_only ? '?debug' : ''
  puts open("http://127.0.0.1:#{PORT}/#{File.basename($0)}#{qs}").read
end

case ARGV.first
when 'config'
  config
when 'debug'
  get_data(true)
else
  get_data
end

The script will examine the name with which you linked it in to the munin plugins directory to determine which URI to query. I have also added a debug mode that will show you the current values, so you’re not consuming any data that munin needs to see for an accurate graph. The final piece is a small Daemons wrapper script to control the main log-tailing process, and you should be set. Make sure to restart munin-node so it will notice the new plugins, and after a while, you’ll see something like this:

It is worth noting that the numbers from the Rails log might not be 100% accurate, and this won’t replace the results that you can get from seriously profiling your application. Also, the information you are getting is a bit generic—all actions are lumped together, so there is not a lot of information about the cause of the performance problem. But, for insight into your production application performance, this setup should at least give you some indications about how well your baby is playing in the interwebs.

Download rails_log_monitor.rb

Download rails_response_time

Check out http://www.danwebb.net/lowpro for more info on LowPro, a very elegant approach to do Unobtrusive Javascript with Ruby On Rails. Find hereafter a small example of how to add a custom behavior to link.

  <%= javascript_include_tag 'prototype', 'lowpro', 'remote', 'application' %>  
        <div id="result_list">
          <ul>
          <% for watch_result in @watch_results %>
          <li>
            <%= link_to watch_result.created_at.to_s(:db),  
                      diff_watch_result_url(@watch, watch_result),
                      {:id => dom_id(watch_result) }
            %>
          </li>
          <% end %>
        </ul>
      </div>

LoadWatchResult = Remote.Link({
  onLoading : function() {
    $('watch_result_difference').innerHTML='';
    $('watch_result_difference').addClassName('pleaseWait');    
  },
  onComplete : function(e) {
    var source = Event.element(e);
    $('watch_result_difference').removeClassName('pleaseWait');
    $$('div#result_list ul a.active').each(function (e) {e.removeClassName('active')});  
    source.addClassName('active');
  }
});

Event.addBehavior({
  '#result_list ul li a': LoadWatchResult
});

The ‘remote.js’ provides additional behaviors creatde by Dan Wedb as part of LowPro (http://svn.danwebb.net/external/lowpro/trunk/behaviours/). The LoadWatchResult behavior we created in this example transforms a ‘standard’ link_to to a link_to_remote with additional behavior on the onLoading and onComplete of the remote call. The view stays clean.

Enjoy! Daniel

No I am not announcing the next killer scaffolding framework but I had a couple of hours available today so I just explored some of the cool features or Ruby On Rails and Flex. The part I was most interested in (today) is dynamic user interface creation and not generating and application like several scaffolding frameworks are doing. So I was able to create a UI that adapts to a given model of a Rails application. Of course I didn’t go as far as I wished, but I thought I could share it with my readers as I usually get valuable feedback. Right now the application doesn’t even manage data. That will be the next step.

In a first phase I have adapted RaildRoad Rails model class diagram generation tool to generate an xml definition that the UI would use to create it’s components. I generated an xml model for Sports a sample application provided with Streamlined and Typo a blog server.

The UI generation from the typo model is quite cpu intensive. There is not lazy instanciation of components, when you select the model, all the Tabs, lists and forms are created.

You can run the application and press view source from the context menu. Or you can see the source here. The generated xml can be seen here and here

In a second phase I created a Flex application that generates a UI from the given xml.

I was writing this blog entry while coding, so if you are more curious about how all this works, keep on reading.

Getting the schema of your ActiveRecords

We will generate an xml version of the schema that Flex can use to assemble a UI. So we need to find out info about each ActiveRecord of the application, it’s association with other ActiveRecords, and all it’s attributes. Later on would also be nice to identify the validation rules, so that we can build some validations on the fly without having to do a server round trip for some of the basic validations, (required, length, confirmation, regex). Supporting the various acts_as could also provide lots of functionality that doesn’t need to be coded over and over.

I will use the Streamlined Sports example database to experiment with. Later on we may have a look at Typo a blog server.

Let’s use the community to see how to parse the ActiveRecord. I am now checking out RailRoad (0.4.0) a class diagram generator for Rails. Railroad has the ModelsDiagram class that gather the information we need and then uses the DiagramGraph class to generate a dot format file that in turn is used to generate .svg or .png of the diagram. We are not interested in the dot generation, but will just ‘adapt’ the to_dot method to get the xml we need. So I simply reopened the class and created a new to_dot method as follows:

# http://chadfowler.com/2007/8/3/enumerable-injecting
module Enumerable
  def injecting(s)
    inject(s) do |k, i|
      yield(k, i); k
    end
  end
end

class DiagramGraph

  def to_dot
    return definition.to_xml(:root => 'active_records', :dasherize => false)
  end

  #Let organize the data in a way closer to the xml we want to generate.
  def definition
    active_records = {}
    @nodes.each do |node| 
      attributes = node[2].injecting({}) {|accumulator, value| k,v=value.split(" :"); accumulator[k] = v}
      class_name = node[1]
      active_records[class_name] = { :name => class_name, :attributes => attributes, :relations => [] }
    end
    @edges.each do |edge| 
      association_type = edge[0]
      from_class_name = edge[1]
      to_class_name = edge[2]
      active_records[from_class_name][:relations] << {association_type.to_sym => to_class_name}
    end
    active_records
  end  

end

The definition method generates a hash map with the information of the model. The to_xml is all that is needed to get an xml version of the data that the map contains.

definition.to_xml(:root => 'active_records', :dasherize => false)

Dynamically Generate a Flex UI

Let looks at the model.

Player has_many Sponsor
Coach has_many Sponsor
Team has_one Coach
           has_many Player
Sponsor has nobody!

The UI generation should be lot smarter and configurable. For now I have taken the approach to create one view for each model. This view shows all the records in a data grid and the detail of the selected record in a form view. All associations of the record are represented by a Tab Navigator. You will notice that this is not very practical for the Typo datamodel. We should have main ActiveRecords that are entry points to the application. Also the depth of the active record view is one association down but could be driven by definition i.e. Team => Coach => Sponsor, similar to the :include option of the find methods. We should be able to flatten a “to one” relation and have all the attributes of the association in the same view than the source ActiveRecord.

You can run the application and view the source.

The main application, FlexScaffolding, adds dynamically a ActiveRecordsView for each ActiveRecord in the model to the Tab navigator. That’s too many tabs for the Typo model…

    private function generateView():void {
          for each (var activeRecord:XML in definition.children()) {
               var arView:ActiveRecordsView = new ActiveRecordsView();
               arView.definition = activeRecord;
               mainView.addChild(arView);
          }        
    }       

The ActiveRecordsView.mxml does the main work of generating the UI. It’s not very big (66 lines), but is also limited for now. I let you check out the source code if you a curious. Next on the list is to map too more data types (ie a “text” ActiveRecord attribute should be mapped to a TextArea, date and datetime should be support). I need to add configuration options where the defaults can be overridden. I need to create a Rails controller that uses the same xml model to expose the data to the view. There shouldn’t be the need to hand code RESTFul controllers. Or maybe the UI should be build from the routes…

That’s all for now. Enjoy! Daniel Wanja

Time.onrails.org – a simple time tracking application has been updated.

• Time shifting – Let’s assume you entered time from 08:00-12:00 13:00-17:00. Then you notice that you effectively took a longer lunch. Simply edit the 12:00 to 11:45, then tab to the next field and change 13:00 to 13:15 and press enter. In the same way, if you didn’t come back from lunch at all that day. Simply clear out the 13:00-17:00 field and press enter. This will delete this time slot.

• Running Timers – When a timer is running (i.e. 11:30- ), the time line (day) is highlighted in green. That way it’s easy to detect which timers are still running.

• IE6 Support. Time entry was pretty broken with IE. This should have been fixed. I haven’t installed IE7…so I assume it doesn’t work there.

• Rails Migration. Behind the scenes Rails was migrated from 1.1.6 to 1.2.3.

Please let me know if you find anything unusual.

Enjoy! Daniel.

I recompiled the Log Visualizer with Adobe AIR. You can download it here.

I tried it under Windows XP (Parallels) and it seems that the File.browseForOpen doesn’t fire the Event.SELECT event under Windows. So the bug is that you can open a file, but the application doesn’t know when you selected it. I was contacted by Logan today who wanted to know if there is a Windows version. So sorry for the Windows users out there for the moment. Note that the Apollo version of the Log Visualizer was working under windows.
Download: RailsLogVisualizer0.5.air

The invitations are going out! So, if you haven’t yet, go signup for the eBay Desktop beta (code name Project “San Dimas”“) at http://www.projectsandimas.com/.

The eBay Desktop is an Apollo Application, oops, Adobe AIR application, that is developed by our friends Tony Hillerson and Sean Christmann from EffetiveUI for eBay. As they had their hand full applying the finishing touch to the beta, they asked Lee and myself to help with the beta Signup program and administration site which is a Ruby on Rails application. How cool is that! Thanks guys for the opportunity to work on such a visible project…

Now if you know Sean you will understand why using the eBay Desktop is fun. When Sean is not coding he is playing games, and he want’s any application to be fun and playful. Even when he gives talks he manages to demonstrates how to program a WII controller. It’s cool that Alan Lewis gave them the creative freedom to create some new user experience concepts on top of the existing eBay apis.

Ken, my desk neighbor at one of my clients asked why would you need something like Adobe AIR? Well in the case of the eBay Desktop, you are provided with an immersive experience, having a dedicated application will encourage the user to stay within that application and not get (too) distracted. The desktop also allows to save custom searches and additional information locally, thus providing functionality that may not be available by the existing eBay apis. It could also allow for functionality like drag&drop of images and text from the file system to create online auctions, use a connected camera to snap pictures, scan bar codes to retrieve product information, and so on…

PS: Hey Sean what’s your blog…or are you too busy coding?

From my perspective (as someone who has been doing Rails development full time for the past two years or so) it’s hard to believe, but there still are web developers who aren’t familiar with Rails. I was recently approached by an old friend about doing a training session on Mono. The last time we had been in close contact, I was exploring Mono and excited about the possibility of a .NET environment on Linux. Of course, that was before I discovered ActiveRecord and became a card-carrying Rails fan club member.

I told him that while that would probably be fun, I’d rather talk about something I didn’t have to learn for the presentation, and by the way, had he ever considered having a session on Rails? Well, I’m just back from the first segment of the course, which consisted of installing a Rails development stack in an Xubuntu virtual machine. Due to bandwidth limitations and a delayed start, that was all we accomplished tonight, but the remainder of the course (actually learning about Rails) will be in two weeks. We will provide a ready-to-go virtual machine to anyone interested in an introduction to Rails, and there are a few seats still available.

So if you or someone you know will be in the Denver area at 6:30 on June 28 (the day before the iPhone!), and have a couple of hours to be indoctrinated introduced to Rails, drop me a line [my gmail account is rubysolo].