Cobalt Edge

 
Filed under

Rails

 

Tricky Rails Callbacks and Deleting Associated Records

UPDATE: As it turns out, the below solution was totally rookie!  The whole issue is still there, but the need to solve it with the use of update_all (which is a bit brute force and generally something to avoid for just updating a couple attributes on a single record, it avoids validations, etc, etc.), was not necessary.  As it turns out, there are association callbacks. I'm kind of surprised I hadn't seen this or thought of it.  But, there are callbacks to be notified when associated records are added, or removed (and both the before and after variations). So, instead, instead of setting the update_lowest_and_highest_rates as an after_update, specify an after_remove callback on the association:

  has_many :fares, :after_remove => :update_lowest_and_highest_rates

Then implement the update method like:

  def update_lowest_and_highest_rates

    fares.reload

    set_lowest_and_highest_rates

    save

  end

And note, set_lowest_and_highest_rates, is the same method as the create_lowest_and_highest_rates, just renamed.  It's renamed because now you'd use before_validation or before_save or whatever is needed in your case, instead of the _on_create variant of that.  The rest of the problems are still there (needing to refresh the fares association, and the general aspect of all this, etc.), but this is a much nicer solution I think.

I recently ran into some tricky issues related to deleting records on an association combined with ActiveRecord callbacks.  The issue may seem like an edge case, but from googling for solutions, clearly it's not an uncommon need.  An example will illustrate the problem...

 
I'll use a sort of pseudo-example from DealBase.  Let's take a flight "Deal" model.  It has_many "Fares", which are just prices for the deal from one airport to another.  A Deal accepts_nested_attributes_for Fares.  This lets us edit a deal and all its fares in one form, etc.  As part of this, you can select to delete a fare.  We're using the standard mechanism of the _delete parameter and a checkbox, etc.  So far, so typical.
 
In order to optimize searching and so on, we keep track of the lowest and highest rates from the fares in the deal record (i.e. we denormalize these values).  To determine these low and high rates, we use an ActiveRecord callback to compute them.  This is where the problems come in, at least when using nested attribute support.  Deletion of records from the association (Fares) does not occur until after the Deal record is saved.  What that means is that you can't calculate the low and high rates safely in a before_save callback, you need to use after_save.  That is a bit of a pain in that it means you instantly are doing more DB calls to update those values since you can't do it as part of the primary update.  For us, the rate of writes is small, so that isn't a big deal.
 
Now the second problem of course is that when you do update those values, you're going to cause another save of the record, and that will kick off the callbacks again.  You've now created an infinite loop.  There's no way that I know of, in Rails 2.3.x, to do an update without callbacks (older versions of Rails had update_without_callbacks).  As per the Rails Guide, there are however a few methods that skip callbacks.  One of those is update_all.  It's a bit of a brute force approach, but it is the solution I wound up with, at least for skipping callbacks.
 
The third problem is that within your after_save callback, your association won't be up to date if a deletion has been performed (it'll still contain the item that is marked for delete).  This makes sense, given you are still working with the same instance of your primary object.  Luckily you can just call reload on that.  If we put all these things together, the final solution is relatively simple, although still feels a bit hacky:
 
class Deal < ActiveRecord::Base
  has_many :fares
  
  accepts_nested_attributes_for :fares
  
  before_validation_on_create :create_lowest_and_highest_rates
  after_update :update_lowest_and_highest_rates
 
  ...
 
  protected
 
  def update_lowest_and_highest_rates
    fares.reload
 
    low, high = (fares.minmax_by {|fare| fare.rate}).map(&:rate)
 
    Deal.update_all({:lowest_rate => low, :highest_rate => high},
                    ['id = ?', self.id])
  end
end
 
There are some downsides, such as validations not getting called on the denormalized values when doing the update_all.  You also likely need a before_validation_on_create to do the same functionality (if you have validations for the denormalized values) when first creating your Deal, in which case you don't have to worry about the delete situation.  I'd love to hear of a better solution.  Also, there is the without_callbacks gem that provides some really nice syntactic sugar for this kind of thing, but hasn't been updated since 2008, and looks like it may not work with the latest versions of Rails.

Loading mentions Retweet
Filed under  //   Rails  

Comments [0]

Setting up Integrity for Continuous Integration

Recently I switched the DealBase continuous integration server to use Integrity instead of CruiseControl.rb. This happened because I'd been having some sporadic failures under CC.rb that didn't seem explainable (no code would change, and tests would fail at random), and also due to some changes in Cucumber between versions, it all conspired to switch.

It should be noted that, as you might guess, the "random" failures were not exactly random, but suffice it to say that the root cause re-inforced my notion to switch. My main beef with CC.rb has more to do with it being somewhat in bed with Rake and wanting to run your CI build via a rake task, and some of the issues (or impurities?) that come up with that. But I'm boring you...

Anyway, Integrity... Setup is relatively easy, and is covered fairly well on their setup page/docs. But the following is what I did, which I'm documenting for myself and anyone else who may find it useful.

I setup my server to use with Nginx and Passenger. I tried using Integrity with Nginx and Thin, but wasn't able to get Integrity to work right (similar to the results defunkt had when they tried it at GitHub). We've standardized on Nginx+Passenger, so this was good anyway. I nuked the Nginx I had already and proceeded (all this being done on an Ubuntu Hardy VPS system at Slicehost):


sudo gem install passenger
sudo passenger-install-nginx-module

I let it install Nginx, and picked the default location. I then re-added our HTTP Basic Authentication and a few other Nginx tweaks as I had in the prior Nginx configuration. Next up was to install some prerequisites and Integrity itself:


sudo apt-get sqlite3 libsqlite3-dev
sudo gem install integrity do_sqlite3 thin
integrity install /home/ci/integrity --passenger
cd integrity

Next you'll want to tweak config.yml to customize the domain where you'll access your CI server. I left the rest the same. Then setup Integrity's database:


integrity migrate_db /home/ci/integrity/config.yml

I also added the "integritray" plugin, so that I could continue to use CCMenu to monitor my builds. See the integritray GitHub page for simple install.

Now point Nginx/Passenger at your install, by adding the appropriate server block for a Passenger Rails app, such as:


server {
listen 80;
server_name your.ciserver.com;
root /home/ci/integrity/public;
passenger_enabled on;

auth_basic "Restricted";
auth_basic_user_file /opt/nginx/conf/htpasswd;
}

Finally, fire up Nginx, and surf to your CI server domain/URL. You should see something like this:

Click the "create your first project" link, and enter details about your app. The "Git repository" field should get your GitHub or other git server URL (e.g. your clone URL). For a build script, the following is what I used:


rake log:clear && RAILS_ENV=test rake db:reset && spec --options spec/spec.opts spec/**/*_spec.rb && RAILS_ENV=cucumber rake db:reset && cucumber --strict -q --format pretty features && rake ci_tag

This is one nice thing about Integrity - it's pretty much any command you can give it. Sure, you might want to wrap that up in a Rake task, or a shell script or however you want to do it. I just entered that raw so it's overly obvious exactly what it's doing. I also found this to work better than doing it as a Rake task, as somehow I wasn't getting the environment to switch properly under Rake. The "ci_tag" rake task is my task for tagging/labeling succesful builds in Git, etc.

Next, you'll want to setup a post-commit service hook on GitHub. You can get your Push URL by clicking the edit link for your project in Integiry:

Note, that works even with HTTP Basic Auth, just add your user credentials to the Push URL before pasting it in on GitHub.

Finally, fire off a build. And note, one downside of Integrity is that it doesn't indicate (in the web UI) that a build is underway. It just says it hasn't been built yet. The integritray item and thus CCMenu will show you that it's building though.

Lastly... some have asked why not XYZ CI server? A few notes on this:


  • Hudson: this looks awesome, but also somewhat overkill for our needs. I have one project to build, and I'm doing it on a very inexpensive slice that has only 256MB of RAM - I doubt I could even start Hudson in that little RAM, being it's a Java web app, etc. Secondarily, I'd prefer to have the app be Ruby so I can hack on it (I've made at least very minor tweaks to every CI server I've used to date).

  • cruisecontrol.rb - this is what we were using, and it worked well, with minor exception to some random failures and the Rake-oriented build process. I'd really say this is minor though, and would suggest folks try it out. You can of course refer to my previous writeup on setting up CruiseControl.rb :)

  • CI Joe/cijoe - this is actually what I started with when I looked at exploring alternatives. I had problems getting the build working properly, which seems odd. In hindsight that may have wound up being due to some problems during our switch to a newer version of Cucumber. But, one thing I didn't like is that there is no state maintained, so if you stop and start cijoe, it loses track of all its previous builds. This may or may not matter much to you, but I didn't like that. I also didn't want to spend time writing notifiers/CCMenu integration type stuff for our needs. I will ay that cijoe setup/install is pretty cool.

  • Run Code Run doesn't have a viable plan for us yet, plus I've heard having it run custom rake tasks and such doesn't work (wrong?). Furthermore, I didn't really want our private code on their servers and didn't see a need to outsource this.

  • others... either hadn't heard of them, they didn't work well with Rails, or whatever - upshot, Integrity worked, got it up and running fairly fast, and didn't need to spend any more time on this.

Loading mentions Retweet
Filed under  //   ContinuousIntegration   Integrity   Rails  

Comments [0]

Passenger Fails to Regenerate Cached/Aggregate Assets for Rails

At DealBase, we've been testing Nginx with Passenger, and have mostly had good results. There are two issues that have come up, hopefully only one of which may broadly affect others.

The first issue, which likely affects anyone using this, is that it appears that if you combine and cache CSS or JavaScript via tags like this in Rails:


<%= stylesheet_link_tag :standard, :cache => 'standard' %>
<%= javascript_include_tag :jquery, :cache => 'jquery_all' %>

The ":standard" and ":jquery" symbols are expansion symbols for multiple CSS/JavaScript files defined in a Rails initializer. On the first request Rails gets, it's supposed to combine all the files as per the expansion symbol definition, and then produce a "cache" file, so you have a single file that is included in your HTML.

This worked fine for us under Mongrel, but it didn't seem to regenerate under Passenger after doing a deploy, even with a restart to Passenger. We had to do a second restart of Passenger, and then hit the server at least twice, if not more to see it get picked up.

Thanks to a tip from Engine Yard, one solution can be found on the overstimulate blog, where they detail how you can add a rake task that you call during deploy to regenerate those cache files. This is really even a nicer solution under Mongrel and others, as it will mean it doesn't occur during your first request.

The second problem seems confined to my MacBook Pro laptop (no problem on my MacPro tower). That is, I simply cannot get Nginx+Passenger to work. It installs fine, Nginx runs, but I get some odd permissions problem from Passenger:


2009/05/12 11:57:52 [alert] 19611#0: could not create /var/folders/7m/7m7ezMSTHdiBHz5bzVyNDE+++TI/-Tmp-//passenger.19596/control_process.pid (13: Permission denied)
2009/05/12 12:01:19 [crit] 19611#0: *1 connect() to unix:/var/folders/7m/7m7ezMSTHdiBHz5bzVyNDE+++TI/-Tmp-//passenger.19596/master/helper_server.sock failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: dealbase.dev, request: "GET / HTTP/1.1", upstream: "unix:/var/folders/7m/7m7ezMSTHdiBHz5bzVyNDE+++TI/-Tmp-//passenger.19596/master/helper_server.sock:", host: "dealbase.dev"

The first line of those two lines occurs when I start Nginx. The second happens when I try to surf to a page from my app in my browser (which makes sense given the first error :) If anyone has any suggestions on this one, let me know!

Loading mentions Retweet
Filed under  //   Nginx   Passenger   Rails   RubyOnRails  

Comments [0]

Front End Rails Developer Job at DealBase.com

At DealBase, we have an opening for a part-time front end Rails developer at DealBase.com. The opening is for US residents only, and for individuals, no agencies or recruiters please. Most likely you'd be working remotely/telecommuting. The job posting, which is posted in several places, such as Rubynow, Working With Rails, and Rubyjobs.in, covers all the details, but I'll relist it here for ease:


DealBase.com, a startup hotel deals site, is looking for a stellar front end web developer who will adapt our current look/feel to new features, leverage JavaScript for useful and fun features, and is eager to apply their skills to enhance the user experience of our site. We're looking for you to share your knowledge and make an impact, be passionate about your work, and up-to-date on the latest technologies. If this is you, and you enjoy working with a small, distributed, agile team, then we'd love to talk with you.

Requirements for this position:



  • Deep knowledge of XHTML and CSS

  • Familiarity with browser capabilities and restrictions for all major browsers

  • Solid JavaScript skills

  • Experience with/demonstrated use of Git

  • You use and demand MacOS X as your primary development environment

  • Comfortable at the command line

  • Basic skills for image editing and optimization for the web

  • Exposure to and basic knowledge of Ruby on Rails

  • Great communication skills

  • Attention to detail

  • Ability to work both independently and on a team

  • Eagerness to share ideas and problem-solve creatively

  • Experience working on consumer oriented web applications/consumer focus

  • Quick learner, and good at digging in to problems

  • Agile development practices

  • You are based in the US.

  • Individuals only (no multi-person firms, agencies, etc.)
  • Nice to have:



    • jQuery experience

    • GitHub experience

    • MySQL experience

    • Use of test frameworks, TDD, and BDD

    • experience with Linux

    • If you'd like to work with us at DealBase.com and think you're a good fit for this position, send us a resume and sample work, or let us know where we can see your resume and work/code, by emailing jobs@dealbase.com. Please note, we are only considering candidates based in the US.


I'm excited to find a great developer to work with. DealBase has been an awesome company and app to work on, and we're already experiencing great success. We have some pretty cool features planned, and it'd be ideal to get some real CSS and JavaScript ninja skills making those features even better. So, if this is you, please do get in touch, making sure to send email to the right email address as outlined in the job description.

Loading mentions Retweet
Filed under  //   DealBase   git   JavaScript   Jobs   jQuery   Rails   RubyOnRails  

Comments [0]

Deploying per-server crontabs with Capistrano

There's been a couple cool writeups/solutions to deploying your crontab files when you deploy with Capistrano, which I think is great. I can't find the first one I saw (mention in comments and I'll update), but on GitHub, javan has the whenever gem that is really more about allowing you to define crontabs with Ruby/Rails' time methods so you don't have to remember the crontab file syntax which none of us ever seems to be able to remember. You can of course integrate this with Capistrano (and that's covered in his Readme). The point of all this: no more having to remember to go put in or uncomment a crontab entry once you deploy a certain build, and keeping your crontabs under version control. However, for us, none of the solutions out there worked quite right, and I just use what I find to be a simpler setup.


First, we have multiple servers with different crontabs per server. Also, we have some environment variables that get defined within the crontab so that they work properly on our Engine Yard slices. I just found that, while yes, I sometimes don't remember all the crontab syntax perfectly, I also don't do this often enough for that to be an issue, and would rather just have the real deal right there, so I knew exactly what I was going to install on my server. Lastly, I didn't really want to have yet another gem dependency for something pretty straight forward like this (IMHO).


So, get on with it you say, what's the solution? Two pieces. First, I create a crontab directory within my Rails app's config directory. In that I store crontab files named by the hostname of the server - the same thing you'd get by doing a hostname on the server. You could add an extension or whatever you want, but the hostname is what makes this work easily, so you want that somewhere in your file naming convention. We only have a couple servers and I know them well, so I just went with pure hostname for now. The contents of each file are exactly what you'd see in the crontab file on the server, for the user you set it up under.


Second, a simple Capistrano task to affect the given crontab file on the server, with an after hook to run it:



task :write_crontab, :roles => :app do
puts "Installing server-specific crontab."
run("cd #{deploy_to}/current/config/crontab; crontab `hostname`")
end
after "deploy:restart", "write_crontab"

That's it. You can obviously tweek this for your own setup, for example, maybe you need to run it on all roles, or different roles, or what not. Your run command might need to be more robust (or run a shell script or rake task) for example if not all servers have crontabs or you have something more dynamic. But, as you can tell, setting this kind of thing up is pretty straight forward, and it's great to keep your crontab setups in version control. Thanks to you guys that stimulated the idea in the first place.


Loading mentions Retweet
Filed under  //   Capistrano   Rails   RubyOnRails  

Comments [0]

MySQL Performance Issues and acts_as_versioned

Recently we ran into an interesting performance issue with MySQL. We have an automated process we run at night a few nights a week that does data harvesting for hotel rates and such. This data is versioned so that we can look at historical values. However, this script had begun to really crawl. Originally it took a couple hours to run. But it had gotten to the point where it could take almost a day.

I tracked this down to being a SQL MAX call used by acts_as_versioned to determine the next version for one of these records. The problem is that it had to sift through nearly 10 million records. In testing this on my local machine, just one of these SQL queries could take 45 seconds! Think about doing this across oh say 100,000 hotels, ya, not good.

The good folks at GitHub ran into this same thing (with a table of 36M records) on nearly the same day. Their approach is similar to the approach I'll be taking on another table (which isn't currently affecting us this way, but will have different benefits), which was to split it into two tables, one with older data. I could have done this, and would have, but the reality was that we simply didn't need to keep these versions, as we weren't using the data. So, luckily, I was able to just no longer version this particular model, and throw out that table. After doing that, I ran the script and it took just over an hour. Yea!

So, this is something to note if you use acts_as_versioned with models that have frequent changes and a decent number of those models to begin with (think multipliers). One of the things I'll be looking into in the future is whether that MAX needs to get done, or whether acts_as_versioned can be smarter about how it does it. On first glance you'd think you could just use the version number on the original model itself, but that number isn't guaranteed to be the latest number, since you can rollback versions and so on.

Loading mentions Retweet
Filed under  //   MySQL   Performance   Rails   RubyOnRails  

Comments [0]

My Setup and Software

I too read Al3x's interview the other day, and like John Nunemaker, figured I'd share my setup, as I enjoy reading what others use and often can pick up a few interesting tools or tidbits.

Unlike Mr. Nunemaker, my desk is too messy, IMHO, to photograph right now :) However, many similarities aside from that. On with it...

I use a 17" MacBook Pro with 4GB RAM as my only machine these days. Like Alex and John, I really like having just a single machine, and I no longer work for a corporation where I'd worry about that. DealBase is cool and wouldn't try to make some wacko claim to some work not relevant (and we've explicitly discussed my use of a single machine, etc.). I have my MBP open on a laptop arm from Ergotron, and then my primary monitor is a 30" Dell. Really love the big monitor. I do my main work o the 30", and then the laptop screen has TweetDeck, iChat, Things, some Fluid apps, and other things that I tend to more glance at, and aren't primary work items.

Further, I use a wireless Apple keyboard, and like John, I just love this thing. I can't tell you how long I'd been looking for a keyboard that was just a keyboard (but with arrow keys). I hate normal keyboards that take up so much extra space on the right side (my mouse side) with stuff I rarely use - which only exacerbates problems with having my arm/elbow canitlevered further out to use the mouse, sometimes causing arm strain after long days of coding. I use Logitech MX Revolution cordless mouse, which I like quite a lot.

Transitioning to music... I use JBL Creature speakers, and listen to a variety of things, or nothing. Pandora, via a Fluid app, iTunes (my own playlists, or various Ambient "radio" stations), etc. Either that, or we have a whole-house NuVo Concerto audio system, so sometimes I have that on either with XM satellite radio, or to a playlist from the iPod we have hooked into it. The NuVo setup is nice because it fills my office with sound a bit better (via in-ceiling speakers), but I have more variety via the computer.

As with Alex and John, I am absolutely in love with my iPhone 3G. It is even better than expected. It has essentially replaced my 80GB iPod in the car, typically because it's more up to date, and I like it's UI better; I can remotely work on servers if I have to via iSSH, play games if I'm bored, use InstaPaper to read things I've set for reading later, sync with Address Book and iCal, and of course Twitter, via Tweetie. So, yes, I use Apple's Address Book and iCal, for great sync, simplicity, etc.

Ok, onto dev stuff. My primary work is on Rails-based web-apps, although I dabble with other things as well. DealBase is my day job, and I'm also involved with Bring Light.

Yet again, like Alex and John, I spend the bulk of my time in TextMate, iTerm (a better Terminal, IMHO), and Safari. And actually, I do my development testing in nightly builds of WebKit/Safari, and all my other browsing in standard Safari. I do pull up Firefox for testing, and to use YSlow and sometimes Firebug (although I've been finding the dev tools in WebKit nightlies work well). I've used Emacs - did so for about a year when working with Linux as my desktop. I ditched it back then in favor of Visual SlickEdit, but these days TextMate just rules. I don't get the Emacs passion - why do you want to press two keys for everything, especially the most common things? Yes, I know, you can setup different bindings, etc., but come on the most basic things like saving, opening, copy, paste, etc. should be "single" key (and by single I mean some meta+key) strokes by default. I do fire up vi all the time at the command line on remote servers, and even occasionally on my MBP for some real quick edit. Also, I spend the bulk of my day in my text editor, so yes, appearance matters, and TextMate kills others. I've also used a lot of IDE's in the past, from IDEA, to Eclipse, to Visual Studio. Visual Studio is actually quite good if you have to suffer in that world, but I find Eclipse just plain crappy. IDEA was great for Java, and their Ruby setup will be something to keep an eye on, but generally, the setup I have now works well.

I have all my code for nearly everything I do (e.g. both private and open source/public) on GitHub, and truly love it. Git has been a huge win, and gives me the best of, as well as improving SVN and Perforce. I'm using GitX for most of my commits and history browsing these days.

I use RSpactor for continuously running our RSpec suite, and we also use RSpec stories (but haven't converted to Cucumber yet). I recently added speech output to RSpactor, and that is my preferred notification instead of Growl. We use Pivotal Tracker for tasks/stories/features as well as bug tracking. We used to use Lighthouse, but having it all in one place was nicer, and Tracker wins big time in my opinion. If you want GitHub post-receive hook for Tracker, I recently whipped that up, and its been a real nice addition. We too use Hoptoad for exception notification, and really like it. Also, New Relic is in use at DealBase. I also like viewing Google Analytics with Analytics Reporting Suite, a slick AIR app.

I really like Navicat as a GUI for database stuff. It's proprietary/pay software, but honestly, it's worth it to me. I can do all this stuff command line fine, but the GUI simply makes it a heck of a lot faster to view the results, quickly re-sort on a column, mess around with queries, etc. Also, it has great SSH support, so I can tunnel into all my server's DB's with ease.

I have CruiseControl.rb setups for all my Rails apps, and make use of CCMenu for a nice little status menu item showing me what's going on with those.

I pretty much can't live without LaunchBar. Same goes for 1Password.

Skitch is quite handy for showing sharing and annotating screen shots, and we use Google Docs and Gmail. Speaking of email, I am a huge fan of Mailplane, which is a Mac app for Gmail. Integration is superb, and I can quickly switch around my 15 or so Gmail accounts with ease. I find it superior to a Fluid app for Gmail, since the integration is better and it handles multiple accounts.

I host most of my own web apps on Slicehost, and DealBase is at EngineYard.

I also use Backpack some, although not nearly as much as I used to, and access it about 99% of the time via Packrat. MarsEdit is my blog authoring tool of choice. NetNewsWire is my RSS reader.

All of my photography and photo processing, etc. are done in Adobe Lightroom. I use the Flickr plugin for it as well.

Various other bits:



  • TextPander

  • WeatherDock

  • Pukka

  • Flickr

  • Del.icio.us

  • xScope - a great screen ruler app

  • Photoshop CS3 (look for my name in the about box too :)

  • JungleDisk - I do some backups with this

  • SuperDuper! Still my favorite backup, although I use TimeMachine too

  • CSS Edit and XyleScope sometimes

  • Last.fm - is running all the time, but I really don't actually make use of it, kinda silly.

  • Acrobat Pro and Reader

  • XCode (or TextMate) if I'm working on an Objective-C/Cocoa app.

  • iStat menus

  • YouControl Tunes

  • p.s. One other bit I can't live without but really isn't computing hardware/software, is my espresso setup. I use an Expobar Brewtus II machine, Macap MC4 stepless doserless grinder and a variety of cups (mostly Nuova Pointe and Illy). I use only totally fresh beans from a variety of places (favorites include Blue Bottle, Ecco Caffe, PT's, 49th Parallel (unfortunately not often, since shipping from Canada makes it a bit cost prohibitive), etc.). Coffelab tamper and Bumper stand and knock box. My espresso bar is kept clean (unlike my desk). The pictures are a bit older, so don't show bottomless portafilter in use these days.

    Whew, that's more than plenty. What's your setup?


    Loading mentions Retweet
    Filed under  //   ContinuousIntegration   CruiseControl   DealBase   environment   Espresso   git   Gmail   iPhone   laptop   Mac   Nuvo   Office   Pivotal Tracker   Rails   RSpactor   Ruby   TextMate  

Comments [0]

Using View Helpers and Controller Actions from Rails' script/runner

Recently I coded up a controller and view that produces a large data file. It was done this way because it needs to generate the proper URL's and take into account a fair bit of stuff at the view level in our application (like pagination, and the custom URL's we have, so extensive use of our URL helpers and other things at this level). The action takes a long time to run (about 3 minutes), so of course I page cache it. However, 3 minutes breaks the web server and/or Mongrel timeouts in our environment, so won't get served up in production and staging environments.

The solution, and I'd love to hear other ways, as I certainly won't claim this is the best way, was to create a small script/runner script that simply executed this route within our app. However, script/runner doesn't normally give you controller and view layer access, plus it doesn't have the context of a web request, so it won't know say the host it's being run against and so on. However, one can leverage an Integration::Session and manually set the host to get that. Thus, the script becomes as simple as:


#!script/runner

require 'action_controller/integration'

session = ActionController::Integration::Session.new
session.host = 'www.yourdomain.com'

session.get_via_redirect '/controller/action'


Note, I'm using get_via_redirect here because the particular action I call does a redirect after it expires the cached page of the action it's redirecting to. If you don't have a redirect going on, then you can just call get instead. This does not output anything of course, but for us, just caches the resulting page, which is exactly what we want.

Loading mentions Retweet
Filed under  //   Rails   RubyOnRails  

Comments [0]

DealBase - Check out the new features

Since doing our initial public access to the DealBase.com site, you know, the web site that has the most hotel deals on the web (50x more than anyone else), we've been working hard on improving the experience for users. Today we rolled out the latest major revision to the site, which includes some pretty cool stuff. First, let me list these things out, and then I'll cover a few interesting technical bits that occurred along the way.


  • New home page: yes, a brand new, much nicer home page. This really helps tell you what's great about the site. It also has a new search box that I call the "omniscient search" - a single search field that auto-completes on our deal locations and hotels, which is much nicer than the separate search boxes we had before. This same search box is also used everywhere else on the site.

  • Deal filtering: this was a big one. Seems fairly simple on first blush, but lots of interesting stuff going on in the background. You can now filter deals on any deal listing page, by various criteria. You can combine filters too. So, for example, you can narrow down the deal listing by dates, prices, and hotel ratings, allowing you to, for example, look for 4 star hotel deals valid from April through August of 2009, in a certain price range, for a given city. Very handy. Here, take a look at the deals for Hawaii page as an example.

  • Deal sorting: in addition to the filtering, you can now sort the deal listing a myriad of ways, from most percent savings or most dollar savings to high or low rates, or most recently posted, etc. Combining this with filters allows you to really narrow down what deals are best for you.

  • Various UI/visual improvements. Meagan, our talented designer, has done a lot of work here.

  • Speed improvements. Various database queries and other operations for the site have been sped up, sometimes in small amounts, sometimes in extremely drastic ways.

  • More deals: we should probably have about 10,000 deals on the site by the time you read this. This is very exciting for us, and shows how serious we are. These are all very real deals, no link bait, no BS. These are true deals, checked by our team of editors. This gives us about 50x more deals than any other hotel deals site.

  • Comments! You can now comment on deals. No login required (just like the rest of the site). Comments do get reviewed, and we'll be watching for spam and so forth, but this is a great way to tell other people about a good deal or a hotel you like, etc. Comment box is at the bottom of a deal's page. For example, check out this wild personal fireworks show deal at the Ritz-Carlton in New York.

  • Chrome browser support. DealBase works and looks great in Chrome.


And now, since this is a geek blog anyway, a few technical bits...

  • We're running Rails 2.1.2, which is the latest release of Rails as of this writing. We try to stay up to date regardless, but this was a key release, as it fixed some tricky ActiveRecord named_scope issues when using SQL JOINs. Our filtering and other work requires various JOINs and the fixes here prevented us from having to explicitly hand craft a bunch of queries. Thanks Rails team.

  • My current favorite gem is Ryan Bates' scope-builder. This is just so nice for building up big, conditionally chainged named_scopes. As you can imagine this is heavily used in building up combined filtering and sorting of deals.

  • More jQuery goodness. I continue to love jQuery, and use it extensively. It is used heavily in the filtering features, pulling in some nice slider UI elements, and also using it for the "updating" status and dimming effects when the AJAX filtering operations are running.

  • One issue we ran into with Chrome was using the :cache feature of Rails' javascript_include_tag. If we used this to combine and create a cache file of a bunch of separate JavaScript files, Chrome failed to properly load/parse the resulting JavaScript file. This broke pretty much everything JavaScript wise in Chrome, but the simple fix was to not use :cache to achieve this.

  • As a helpful, and economical testing tool, we've been using CrossBrowserTesting.com to give us VM's of a slew of different OS and browser combinations. I tend to run VMWare Fusion and do a lot that way, but it's also a pain to keep up a bunch of different VM images, or have to fire that up for a quick test, etc. We're also using BrowserCam.

  • Finally, another shout-out to the Hoptoad service/folks. This continues to be an outstanding service for us. It works really really well, and it's free, so to me it is the winner amongst the competitors.

  • We've done a fair number of modifications to our tagging plugin (acts_as_taggable_on_steroids), although they're all particular to our app, so not sure if any will get contributed back. Things like enforcing all our rules about tag naming and so on. The same goes for the will_paginate plugin. But in this case, I'm hoping to contribute these back as soon as I can properly contribute the patches and ensure they'll work in any app.


I'm sure there's more, but that's what I can think of for the moment. It's been a busy couple weeks, and I'm really excited about the state of the site these days. We've been getting some great feedback, and I've had a few friends book deals they've found on the site (one friend saved $800 on a trip!). If you have feedback, don't hesitate to add a topic or question in our Get Satisfaction feedback system. This tells us what things you'd like to see, or any problems you're finding, etc.

Loading mentions Retweet
Filed under  //   DealBase   Rails   RubyOnRails  

Comments [0]

Speed Up and named_scope acts_as_taggable_on_steroids Finds

I use the acts_as_taggable_on_steroids plugin for tagging. I've been happy with it, but recently have been adding a lot of searching, sorting, filtering, etc. functionality to an app, and needed the find by tag functionality to work as a named_scope, so that I can have it within a chain of many named_scope finders.

This turned out to be trivially easy to do (without having to copy the SQL and put that into my named_scope). To add a "tagged_with" named_scope to your model that is already acts_as_taggable, you can just do this:


named_scope :tagged_with, lambda { |tags| YourModel.find_options_for_find_tagged_with(tags) }

I've been doing a lot of benchmarking and performance improvements to our SQL as well, and decided to see if this was any different in performance compared to just doing YourModel.find_tagged_with that acts_as_taggable_on_steroids adds. As it turns out, the named_scope version, which is really identical at the core, is faster, especially if called more than once (per thread/Rails request)! Here's the benchmarks to prove it:


Single query/1 Iteration:
user system total real
named_scope 0.040000 0.000000 0.040000 ( 0.045085)
find_tagged_with 0.020000 0.010000 0.030000 ( 0.108233)

10 Iterations:
user system total real
named_scope 0.030000 0.000000 0.030000 ( 0.040282)
find_tagged_with 0.420000 0.030000 0.450000 ( 2.040245)

I repeated these multiple times and got the same/similar results each time. So, for a single query, it's only about 2x faster, but if you start issuing this same find multiple times per request then I believe it's the Rails query caching that kicks in with named_scope, but apparently not with the generic find with all the options (I'd love to hear some commentary on this from someone who knows the details).

Regardless, using a named_scope is nice because now you can more easily chain a tag find together with other named_scope items.

Loading mentions Retweet
Filed under  //   Rails   RubyOnRails  

Comments [0]