Cobalt Edge

 
Tags



January 29, 2010

Rails Action Caching With Query Parameters

First, if this is known to people, great, if not I hope it helps others, as I wasn't able to find this in all the Googling I did.  Furthermore, since this turned out to be such a simple solution, I'm curious if there are holes...

I wanted to setup more significant caching for some heavy use types of pages on DealBase.  Various things have made this challenging to date, but the last thing I ran into was dealing with the fact that query parameters change page results (duh), but that of course Rails' page and action caching ignore query parameters.  There isn't an easy (or?) way to get around the page caching part unless you start mucking with Nginx rules as well I think.  But, action caching has a solution.  I had done a ton of Googling on this, and I knew about adjusting the cache_path, as well as some other bits, but we have cases where there are a lot of parameters.  Plus, I didn't want to worry about what happens if I add a new search/filter type of parameter later on, and having to remember to add that to the list of things the cache_path differed on.  

As it turns out, you can simply do regular action caching, with full query parameter support, very easily with:

caches_action :my_action, :cache_path => Proc.new { |controller| controller.params }

That will do the regular style of action caching, but stick all your query parameters on (in alphabetical order so you don't have to worry about different query parameter order not retrieving the same cache results).  Now, it's quite likely that you'll want something a bit fancier.  For example, here's something closer to what I use in reality:

caches_action :action_one, :action_two,
  :cache_path => Proc.new { |c| c.params.delete_if { |k,v| k.starts_with?('utm_') } },
  :expires_in => 4.hours,
  :unless => Proc.new { |c| c.request.xml_http_request? || c.send(:current_user).try(:admin?) }

This just adds in some conditions, as well as removes some query parameters that I don't want to differentiate the cache on ("utm" keywords for Google Analytics, etc.).  In this particular case we're not caching the page at all if it's an AJAX request or for our admins.  

Finally, one comment is that you do need to be careful with things like pagination.  In many cases, page 1 is going to get viewed a ton, and maybe page 2, or page 17 for that matter are rarely viewed.  You could have a case where you make updates to the content that then makes a change such that the different pages are out of sync and thus you have duplicate items on pages or missing items.  You could skip using expires_in, and use cache sweepers if that works for you, but factor in how often updates are happening and whether that might nullify the advantage of caching (i.e. if you do frequent updates, it'd expire your cache a lot and maybe too often to benefit from it if you have high rate of updates).

What say you?  Anyone else doing this, any other issues?

Loading mentions Retweet

Comments [0]

January 19, 2010

Evaluating RubyMine IDE for Rails Development

I have spent roughly the last month using the RubyMine IDE for about 80%+ of my daily development work.  I initially grabbed it to try out the debugger on a particularly nasty problem, and then decided to give it a real evaluation.  My daily work is on DealBase, and covers the full spectrum of Rails development.  The testing was done mostly using a quad-core Mac Pro with 8GB of RAM and 30" and 24" monitors (dual monitor setup), with RubyMine running essentially full screen on the 30".  I also did some work on my MacBook Pro (Core 2 Duo, 4GB RAM).  Both machines use Snow Leopard.  My current and longtime editor has been TextMate, although I have far more time spent historically in environments like Visual C++/Studio, IDEA, a bit of time in Eclipse, a bit in Emacs, BBEdit, CodeWarrior, regularly use Vim, etc.  To date, the speed and light weight of TextMate, combined with it's slick column editing feature (that's one killer feature), have kept me from switching.  But, there may be some movement here...

I kept notes during my eval, and simply broke it down into pros and cons for me, along with a list of "other" comments.  Just going to dump these here, and then I'll tell you the outcome at the end...

Pros
  • Debugger, oh, to have a real debugger back, is quite nice.  This is what motivated me to try it to begin with.  Super easy to setup and get working to, basically no effort at all, which is much appreciated (but do need to see if can be done with Passenger setup instead of Mongrel, since we don't use Mongrel).  I've used command line ruby debugger, but once you've used a solid GUI debugger, the command line ones are just such an inferior way to work.
  • Really like how it puts the folder name in front of the file name when there are multiple files in the project with that name
  • Puts squiggles under misspellings or other code errors - quite handy.  It also puts this under the file's name in its tab, so you can quickly see which files you have open that have errors in them.
  • Hold Cmd while hovering over the name of a partial or similar, and then click to open that file
  • Word selection via double click includes the "@" on field names for Ruby code
  • It's quite smart about correlating Cucumber step statements to their definitions: it will actually show you a step statement doesn't match when that non-match is because the regex in your definition is not matching.  For example, if you had something like  "from (\w+ airport)" in your definition, but you put "from San Francisco airport", it wouldn't match, and RubyMine figures this out (I'm impressed!)
  • Having an irb or script/console REPL avail right there is quite nice.  I'm still experimenting with whether I organize my windows such that this is always at the front or not.
  • I'm just starting to get used to using them, but the "view for" and "action for" little markers in the gutter of controller/view files will take you to the corresponding view/action, a nicety.
Cons
  • TextMate bindings didn't map Cmd-Enter the way TextMate did.  Remapped it and it works fine now.
  • Doesn't work (well) with MacOS X Spaces - prevents switching to another space when you select another app
  • Sometimes grinds to a halt while it re-parses your project or what not
  • File tabs: 
    • They go to multiple rows too easily.
    • They just don't look right/good.  I'm not sure if it's because they're non-native tabs or what.
    • I understand, but don't like how the rows swap when you change to a different tabs.  You lose track of where a file is in the tabs.
  • You can't really just open up an empty, new window at whim.  I use this a lot as a scratch pad type of thing in TextMate.
  • Find in project is nowhere near as fast, or presented as usefully as Ack in Project for TextMate.  It does provide more options, such as filtering files by directory or type and so on, but the speed is generally far more important.
  • Too slow on a MacBook Pro 2.4GHz Intel Core 2 Duo with 4GB RAM - too many operations seemed to bring it (and my machine) to its knees.  Seems ok on my quad-core tower with 8GB RAM.
  • GoTo File (Cmd-T in TextMate) is odd in that it'll match to image files.  I would expect these to be automatically culled out (they are in TextMate), as I'm not looking to open some PNG file for editing.
  • Also seems to crash enough that it bothers me.  I can tolerate an occasional crash, although with TextMate I've run it for weeks on end with no crash, using it constantly.  I've had several either hangs or crashes with RubyMine in the short couple weeks I've been trying it out.
  • Can't run parallel specs/cucumber in it and get the proper processing/handling.  Our test suite is extensive enough, or well, takes long enough to run, that I really need to take advantage of multi-core machines and run this stuff in parallel.  It'd be awesome if they could integrate parallel specs.
  • "Synchronize files on frame activation" doesn't work, or doesn't do what I thought: have to tell it to reload log files from disk.  Even closing the tab for the file, and re-opening it, doesn't re-synchronize it from disk.  This really sucks when viewing log files, and is not even good for regular files - prompt me if the file changes on disk and see what I want to do, before I've spent a lot of time editing something only to realize I was editing a stale version.  
  • Has Google Chrome browser listed in Web Browsers prefs, but I can't select it or type in its location, etc.
  • Has the usual Java UI ugliness - Java file dialogs instead of native dialogs (and thus you see files and stuff you wouldn't normally see - not all bad, but ugly), just doesn't look nearly as good as a native app.
  • If I specify WebKit as my browser (which gets opened when starting to debug), a) it doesn't bring it to front, and b) it gets the URL wrong.  I see this in the address bar: "file:///Applications/RubyMine%202.0.app/bin/http:/0.0.0.0:3000"  I fared better with Chrome, but it launches a second instance of Chrome (not just a separate window, an actual separate app process).  This may be a Chrome behavior though?
  • I added a Tool for "/bin/zsh -i" to have just a regular shell in side, but the coloring and various escape codes and such don't work, and that winds up breaking enough stuff for me (that I don't want to re-jigger for this), that that's a downer.  I'd hoped that I could sort of just "live" in the IDE, like many Emacs users do.
Other 
  • Column editing is partly there, but not as good as TextMate.  If you make a column selection, and then hit paste, it doesn't replace it, it just inserts your paste at the end.  This makes it hard to say bulk rename something that's aligned vertically.
  • The refactoring bits were just ok.  This is a really hard feature in general though.  Simple refactorings I used, like renaming a method worked fine.  But, changing the name of a model class did about 1/4 of the work needed and was questionable as to whether it was an advantage over just doing it all myself.  It got all the obvious, low-hanging fruit, but it seemed to completely ignore Rails helpers and views.  It misses a bunch of things that I'm not surprised at of course, such as table names in SQL conditions (e.g. in named_scopes), model name used anywhere in JavaScript or CSS (which isn't uncommon in Rails code), and then the one that I'd like to see it do a better job on, but that is hard of course is on associations.  So, in my application, I was refactoring our "Deal" model to be "HotelDeal".  We have a "Hotel" model and it has_many deals.  Thus in the code you'd routinely see things like "hotel.deals" to reference that association.  It didn't handle any of those.  Not surprising, but starts to devalue the refactoring feature.
  • Their TextMate keybindings don't get Cmd-Enter right, but you can fix it.  You can remap the keys for "Split Line" and "Start New Line" (swap the two for example - "Start New Line" is the one you want to be Cmd-Enter).
  • I'm not quite willing to call this a con, but I find that I still prefer to use GitX to review and commit my code, instead of what's built in to RubyMine.  GitX has a nicer UI for it, just seems more straightforward, etc.  But, it is at least quite nice to have Git support right in RubyMine.  The two key changes I'd like to see that would potentially fix this for me, would be to show diffs by simply selecting one of the changed files (just like GitX does).  This is key for me because I review all code changes prior to a commit.  Second would be to show what branch I'm on.  I always work on a branch, and it's nice to see the branch as a double-check prior to committing.
  • Hoping for a TextMate theme importer (I think that's coming?), so I can import my customized Argonaut theme.

Verdict?

So, what's the upshot?  Well, I certainly won't ditch TextMate, but to my own surprise, I find I'm getting quite comfy in RubyMine.  When you need the debugger, it's really nice to have, and the other thing that's really been growing on me is their syntax checking - the squiggly lines under misspelled variable or method names, or simple syntax errors, etc.  It just saves time.  Log file viewing is clearly a problem, so I use TextMate for that (or just tail it, depending on my needs), and I don't find it to really be usable on my MacBook Pro, mainly just feels to sluggish, but I don't often use that for heavy dev work anyway.  I'd like to see it (or the debugger) work better with launching non-standard browsers.  I had had the hope that it could be like Emacs is for some folks, where I could really do darn near all my development things from inside it.  You could, but they aren't up to the same level as the separate bits yet (real zsh shell and GitX, in particular).  

My expectation is that I will buy a license, as the $99 price is worth it just to have the debugger when needed, but I might actually wind up using it regularly.  I think their pricing is actually a bit low, but it no doubt hits a sweet spot that likely motivates more folks to buy it.  My experience has been better than expected (I thought I was done with IDE's), and I'm also now eagerly awaiting Aptana's release of RadRails 3.  Their little sneak preview videos have been very promising.  But, I generally hate Eclipse (I've had plenty of time with it back when doing Java work, and always preferred IDEA, so there's my bias out in the open), so I'm wondering how that'll impact things.

Thus, I'd suggest that if you're a Rails developer, you give RubyMine a whirl.  It's really pretty impressive and I look forward to trying it a bit more, and also seeing what improvements come in the future.

Loading mentions Retweet

Comments [0]

January 14, 2010

Liberation by Boxcar: or Alternatives to Twitter SMS

The iPhone app Boxcar has liberated my iPhone from the stress and worry caused by Twitter SMS.  What? 

All of the server monitors I have setup, but most importantly, the ones for DealBase, will send me an SMS if there's a problem.  I also get Hoptoad exception notifications via SMS for DealBase.  These are things that are timely, and important.  I'd had one bad case where the servers were reporting down (they weren't actually, was a DNS issue), and this wound up unfortunately setting me up to be hyper-vigilant in checking SMS.  Unfortunate stress and worry.

However, what compounded this was that I was getting some tweets (friends mostly) via SMS.  Well, obviously, the tweets far outnumbered any actual server issues, but due to the above vigilance about checking SMS's, these tweets would make my phone ding and make me immediately worry, "did the server go down?!"  I wound up starting to cut down whose tweets got sent via SMS.  But during this, I also realized that I actually didn't want any tweets via SMS unless they were a "direct" or a "mention".  If they weren't, I'd just read them when going through my regular Twitter feed.  However, Twitter doesn't have an option for just getting any directs or mentions via SMS (they really should).  That led to Boxcar...

Boxcar is a fantastic iPhone app that monitors your Twitter (or Facebook, and other things) stream for directs or mentions, and sends you push notifications when one is found.  It's fast, staying extremely current with your feed, and works great.  With this addition, I've been able to off SMS for everyone I follow on Twitter.  I've been doing this now for about a week, and realized this morning how liberating it has been!  

Normally, I would wake up, instantly check my phone (which serves as my nightstand alarm clock now, thanks to the excellent Kensington Nightstand Charging Dock - click the little photo icon to see how it'll look in use), to see if any server down SMS messages came in that I didn't hear while sleeping.  Usually I'd have a few SMS messages, which would instantly raise my stress level, only to pretty much always find they were tweets.  With Boxcar, now my mornings are great.  I still check my phone of course, but yep, no SMS's anymore.

So thank you Boxcar, great product!  While I can't more heartily recommend Boxcar, I'll point out Prowl which some folks also recommended.  My main issue with Prowl is that it requires an always on Mac to grab whatever data you want as a push notification, and send it to the Prowl servers.  There are workarounds for some things for this, but it was just way more hassle and such than I wanted, for what I needed.  But, you may find it useful if you have data you want as a push notification that you don't have another way to get.

Loading mentions Retweet

Comments [0]

January 6, 2010

Workaround for jQuery Autocomplete Plugin mustMatch Fail on Commas

The jQuery autocomplete plugin is quite nice, and we use it a fair bit on DealBase.  However, what I found was that if you set the mustMatch option, your text cannot contain commas, colons, and a few other punctuation (and maybe other) characters.  For some reason that I've yet to track down (debugging through many nested anonymous functions and callbacks has yet to yield an answer), it thinks it changes at some point to not think it's a match. A good example of this is if you allow a user to type in a city name, but you want to show them matching cities with the state and/or country included, to disambiguation.  E.g. you may have Portland, Oregon and Portland, Maine (not to mention the few Portlands in Australia :)

What I found fixed it for me was to set the matchSubset option to true.  This allowed it through, retaining the full text of something like "Portland, Oregon".  According to the plugin's docs, matchSubset should only come into play when using the cache, which we don't use (because we limit results to 20 items, and often there are more than 20 matches, e.g. a user types "new" or "las" or something - many cities will match).  

I'd still like to track down the real bug here, but since others have run into this, and I've yet to see any other solutions, I wanted to post that, in hopes it'll help some other person suffering from this.

Update: even setting matchSubset to true, did not solve the problem where you have a dash/hyphen in the string, and the part that matches is to the right of the dash.  Removing the dash is the only thing I've figured out to make it work.

Loading mentions Retweet
Filed under  //   JavaScript   jQuery  

Comments [1]

December 31, 2009

Final Thoughts on Coders at Work

I finished reading Coders at Work about a week ago.  Truly a great book as others have also said.  I was surprised by some of the things I read in the interview with Donald Knuth.  He is such an icon, and his books are really intense, and as such, I think I expected something (someone?) different.  But, he seems quite approachable, down to earth, and of course very interesting as well.  A couple great parts stood out in Knuth's interview.  

When he was asked about the reading of his books and how challenging they can be (due to the math involved, etc.), he responded, "I sometimes wonder if I can read them..."  How awesome is that?!  Later on in the same answer, he says, ..."and I write it down and put it in the book so that I don't have to have it all in my head."  It's nice to know he doesn't necessarily have his books memorized.

In another section, when asked about a technique he's using and whether it'd be useful to others, he answers, "I have no idea.  I'm not sure how it would work if I was in a team of 50 people..."  To me, that's refreshing in this day and age when everyone is so adamant about the methodology or process they use to develop software being the one right way, etc.  One way does not work for everyone.  I've seen this getting a bit out of hand in the Ruby community at times.  There can be a fine line between being very passionate about a technique, and preaching it or, trying to cram it down everyone else's throat as THE way to do it.  I think ideas hold a lot more weight when you can be excited in showing someone something, and then let them absorb it and come to the same realization themselves (or not - even if they don't, they'll probably have more respect for it working for you, instead of being completely turned off).

Lots of other good bits.  One final bit about the book in general that surprised me personally, is that I generally stay away from history.  I just don't generally find it that interesting, and in the software and compute world, often find it not pertinent to work I'm doing.  But this book presented history in a truly enjoyable way.  It has motivated me several times to look into various topics.  I'm currently on a Clojure kick (as I'm motivated to re-learn LISP, as well as see more about functional programming in general (thanks Armstrong and Jones)); and I plan to crack open the Knuth books shortly as well (to date I've been one of those folks who owns them, but they just sit on my shelf).  Listening to various luminaries talk about the non-trivial nature of them, as well as Knuth himself, simply makes them feel more approachable.  

And, I should have a bit more reading time, since I managed to fracture my ankle and tear some ligaments on the 26th while trail running.  So, I'm doing a lot of couch surfing these days.  

Loading mentions Retweet

Comments [0]

December 10, 2009

Pivotal Tracker GitHub Post-Receive Hook Now Smart About People

That title is a bit cryptic, but here's the deal.  In the past, the GitHub post receive hook for Pivotal Tracker (http://github.com/chris/tracker_github_hook) was configured using a single API key.  This meant all the comments it added to Tracker were as the person who corresponded to the API key - regardless of who actually made the commit to git.  On multi-person teams, that sucks, as you want the comment to be from the person making the git commit.  Well, that's fixed now.  The fix is backward compatible, so if you simply update your version, nothing will break.  

To take advantage of the new ability, you need to get the new version of the code, and then update your configuration file to associate the Pivotal Tracker API token of each person who may make a commit with their git email address.  To do so, add a new user_api_tokens value to the configuration file, listing each person (arbitrary name) and their API token and email address.  The email address needs to match that which is associated with their git/GitHub commits.  Here's an example:

user_api_tokens:
  chris:
    email: 'chris@example.com'
    tracker_api_token: deadbeef340e3bafa96f5b2cac6986ed
  zach:
    email: 'zach@example.com'
    tracker_api_token: badd00d27202feed12de3120903ea83a

Now what will happen is that anytime a commit comes into the post-recieve hook, it'll try to correlate the email of the author of the commit with an entry in user_api_tokens, and if found, it'll use that tracker_api_token to call the Pivotal API, and thus associate the comment in Tracker with the person whose API token it is.

For any email that isn't found, the default/top level tracker_api_token will be used (this can be the same as one of the user_api_tokens), so you still want to keep that one in there as your default API token.

Loading mentions Retweet
Filed under  //   GitHub   PivotalTracker  

Comments [0]

December 3, 2009

Coders at Work is a Great Book

I'm now about 3/4 of the way through Coders at Work, which is a series of interviews with software lumninaries (a good range, like Douglas Crockford, Donald Knuth, Brad Fitzpatrick, etc.).  I've been truly enjoying this book, one of the better computer/software/geek books I've read in a while.  There have been various points of debate further covered by Lambda and so on; big ones tend to be C++ hate and testing.  I've liked reading a lot of the different points of view on what one needs to know to be a great programmer, as well as debugging techniques (a lot more print statement use than I expected!).  

Since I just finished the chapter with Dan Ingalls, a couple points that caught my attention:

He's never used C or C++.  For someone with his experience, it caught me by surprise.  My biased view of things is that C is that "everyone" has done C.  I realize these days that's very much not true, in that you have plenty of folks who've only used PHP for example.  But, if you've got say 10+ years of experience, it's amazing if you've avoided C (or C++).  I suspect it will still be some time before I've written more LOC in a language other than C/C++ (for me, more C++ than pure C).  

And, this question and answer, to quote directly from the book, I loved.  Not because it means we keep programming to us programmers, but because he nails the fact that people want to focus on what interests them, and collaborating so that you leverage your expertise and don't waste your time on something that someone else can do more efficiently and/or effectively, is great.  Oh, and I love the last sentence in his answer, awesome:

Seibel: Related to that, as more and more fields rely on computing in more and more ad hoc ways, there are folks who want to find a way for "nonprogrammers" to program.  Do you think that'll happen, or will domain experts, say biologists, always have to team up with programmers to build custom software to solve their problems?

Ingalls:  I think there will be that kind of collaboration because the biologist isn't interested in programming it.  He's interested in finding out this or that.  Then there's somebody who understands how this stuff is being wored on on the computers who can help him do that.  I think the thing that lets a nonprogrammer program is an application. 

 Get the book, it's good stuff.  A few notes about me, in relation to common themes:
  • I like testing, and believe in automated test suites, etc.  I like TDD, BDD, and TFD, etc., but I'm not insanely strict about it either.
  • I've done a ton of C++ (e.g. Photoshop and other desktop apps), and at this point, especially after things like templates and so on, find it repulsive.
  • I don't believe you have to know the hardware or assembly language to be a great coder - unless what you're coding is particularly impacted by that.  Ingalls talks about this a bit too.
Oh, one last thing the book has inspired, or really, has further motivated me on, is to [re]learn some other, what I'd consider non-mainstream languages (depends on your POV).  In particular Lisp, or in my case, I'm poking around with Clojure (I did scheme stuff in school, and loved it, but it's been a long time since I've done any lisp or scheme code, nor have I written a significant app/program with it) and Simon Peyton Jones' interview inspired me to look at Haskell.  I've been quite interested in functional programming in general lately.  I've spent some time with an Erlang book, but honestly the syntax just repulsed me and thus I'd rather look at something else (Armstrong's interview was also interesting, not that any of them weren't).  I'm after the functional stuff more for the principles they pursue, and don't really expect to use them specifically, so I'd rather spend time with one where the syntax is more appealing.  Fun.

Loading mentions Retweet

Comments [0]

October 8, 2009

Switching to Posterous

I'm moving my blog to Posterous, or at least trying it out.

Loading mentions Retweet

Comments [0]

October 2, 2009

DealBase Now With RSS Feeds For Hotel Deals

I've been extremely busy with DealBase work, so have lagged on blogging, but have some interesting things coming up and hope to post more soon. Recently I added RSS feeds to all of the city/destination and hotel pages on DealBase. This is just one more way for you to keep up to date with all the deals we add to the system. Our other primary way, aside from browsing the site of course, is to sign up for personalized alert emails. This lets you follow one or more cities, and set criteria like class of hotel (i.e. you only want to see 4 and 5 star hotels), maximum price, and so on. You can set a schedule for how often you want the emails (once a week, twice a week, daily, etc.). We'll send you up to the best 10 deals that match your criteria. You can also simply subscribe by just supplying your email address and nothing else and we'll set your initial preferences for our top 10 destinations. We'll continue to enhance this alert service, and I'd love to hear any suggestions people have for additions, changes, etc.

Development of the alert emails has been interesting. One of the most useful tools throughout has been Litmus' email testing service. It's pretty slick. They cover quite an array of email clients, both web hosted (Gmail, Yahoo, Hotmail, etc.), as well as desktop (Outlook, Apple Mail, Notes, etc.). They also do spam testing. When you begin a test, they supply you an email address specific for that particular test run, and then you just email them the real email you want tested. You can also paste in an email, but I prefer to do the sending approach to as closely mimic the real thing as possible. I have my IRB and various other bits configured to make it a single command to fire off test cases to a Litmus supplied email address, and the whole process works great, and is truly worth Litmus' cost.

I started off mentioning RSS, and back to that... The RSS does work differently than the email alerts (obviously), and to some degree the web page. It is like selecting the "Most Recently Posted" sort on a page (see below image). Deals are supplied to the RSS feed as they're posted, just like a blog would do. Just one more way to stay on top of deals. Oh, and don't forget we're on Twitter too.


Uploaded with plasq's Skitch!

Loading mentions Retweet
Filed under  //   DealBase  

Comments [0]

August 20, 2009

Further Continuous Integration travels: Hudson, and Back to Integrity

[Update: I resolved the Git issue, and have now switched to using Hudson. The advantages of it's in-progress display, ability to more closely monitor and/or kill a build, and my impression of it being more stable, won the day.]

After switching our CI server to Integrity, there were a few blips, one of which is that we were hitting swap (memory). Hitting swap is not surprising and I'm surprised it hasn't been a problem sooner, since memory is our #1 battle. I figured if I was going to up the memory on our CI server, I might also try Hudson, as that was the main reason for not trying it previously.

The short story is that I tried it, and we're back to Integrity, but I learned some interesting things. The following is both some notes on installation, as well as some reasons why it didn't work out...

Installing Hudson

Our CI box is a slice at Slicehost, running Ubuntu 8.04 LTS, and I'd just upped it from 256MB RAM to 1GB. Also of note, I've built and install Git 1.6.4 for this system. The following are brief notes on getting Hudson going as it wasn't quite as simple as most folks made it out to be...

First, I followed the directions on the Hudson site for installing Hudson from the Ubuntu package. This amounts to adding the package repository, updating apt-get, and then doing:

sudo apt-get install hudson

This installs all the Java 1.6 stuff you need (and mine seems fine with 64bit Java), and other dependencies. After the install, Hudson is running, on default port 8080. Next up I added a server block/configuration in Nginx for Hudson (and unlinked/removed Integrity). I then went to Hudson in my browser. What I got was an error message about the xstream library. Fixing that was easy, as it turned out: downloaded the latest (1.320 at the time) hudson.war from the Hudson site, and replaced /usr/share/hudson/hudson.war with it. Restarted Hudson, and voila, now it was up and running.

Also, Hudson runs as the user "hudson", so I needed to add an SSH key for that user, and then add the public key to GitHub. And, setup a ~hudson/.gitconfig as needed. Finally, as I found later, do a git clone or an SSH to github so that you get past the whole SSH authenticity question when you first SSH to an unknown server. Note, the Hudson user is not an interactive user, i.e. you can't directly login as that user, so to gen the SSH key, you'll need do something like su to root and do, sudo -u hudson ssh-keygen -t dsa.

Configuring Hudson

Before adding a project, I needed to config some plugins. I went to the Manage page, clicked on Installed, and turned off the Subversion plugin and restarted. Next was going back in to manage plugins, and installing the Git, Github, and Rake plugins, and again restarting. Both restarts showed an Nginx bad gateway error, but simply refreshing got it back (probably just needed more time for Hudson to restart).

Then to configure a build, from the main Hudson page after a fresh install, click the "create new jobs" link. In the ensuing form, enter a project name, and select "Build a free-style software project".

Next:


  • checked "Discard Old Builds" which then shows you options (so you can put in keep for X days, or X number of builds)

  • Added URL for Github project, http://github.com/yourproject/yourproject

  • Select Git as the SCM, and entered by git@github.com:yourproject/yourproject.git URL for the repository

  • Turned on merge after build option. This will supposedly add tags for the build to your code base and then merge them back in. More on this in a bit.

  • Next I configured the build steps for my project. All I really did here was take the same build steps I used for Integrity, and added them as individual shell and rake tasks.

At this point, I fired off a build (truth be told, I started with just a single build step to vet it), and things worked, with the exception of the very last step, where I push the Git tags back to GitHub. This is what I saw:


[workspace] $ /bin/sh -xe /tmp/hudson1444107192962944065.sh
+ /usr/bin/git push --tags
XML error: syntax error
error: Error: no DAV locking support on https://github.com/dealbase/dealbase/
error: failed to push some refs to 'https://github.com/dealbase/dealbase'

I've now been searching for answers to this, and haven't yielded anything. I've tried the git push directly from a shell, with the same result. If I do this as a different user (e.g. under my user) it works fine. This git push is attempted both as part of a rake task (the ci_tag task), as well as I tried making it just a straight shell command in Hudson.

After a lot of googling, and asking, and no resolution in sight, I've gone back to Integrity...

Comparisons and Notes

First off, I think Hudson looks pretty stellar. There are a TON of plugins for it, and it seems quite mature and polished. The essentially 100% configuration via the GUI is slick. Install, despite a few hoops above, was actually pretty painless. So, here's just a few notes/opinions:

Hudson



  • You will need some memory to run Hudson, more than with Integrity or CruiseControl.rb. From what I can tell, you probably want a system with 1GB or more. Various other folks I talked to all had 2GB or more systems, and their Hudson processes were taking 1.5GB or more. This is partly just a Java thing. It should be noted, the others were running more than one build with Hudson, and mine seemed to work fine on a 1GB total memory system (didn't seem to hit swap).

  • Hudson allows you to kill a build while it's running (nice!).

  • Hudson works with CCMenu/CCTray out of the box.

  • The git integration has more options in terms of picking branches, doing merging, and various more involved operations, but doesn't have GitHub post-receive hook support out of the box (there are plugins up on GitHub for it, but not listed in the standard plugin list).

  • Hudson runs as user "hudson", which is a user that has no shell. You could change this, although the idea is you shouldn't have to. However, this complicates setting up SSH keys and various things. And, of course, I had the issue with Git as mentioned above. I could probably change this to run as my user and so on, but part of all this for me is not having to change a lot of defaults and start messing with core configurations/designs of the system. In part, I just don't have time to do that, and to maintain it (these kinds of changes often cause problems when you upgrade versions, etc. - also known as you may forget to redo these changes if an upgrade undoes them :).

  • The UI and web app itself is quite nice, understandable, well done in general for something like this.

Also, thanks to the Tea-Driven blog for motivating me to try Hudson (and for some tips on Testjour - more on that in another post).

Integrity

I really only have one main complaint about Integrity, and that has to do with indication of a build in progress. It essentially doesn't indicate it - it says it hasn't been built yet, but a build may actually be running. The CCMenu/integritray plugin will show you that a build is running, so this mostly solves my problem, but this seems like a core failure. I may have to look at fixing this. And that is a win for Integrity in that it's Ruby, and thus I'll be more apt to go fix this (while I spent MANY years doing heavy Java work, I don't have interest in working in that code anymore for a task like this).

CruiseControl.rb

One thing that remains an advantage for CC.rb is that it has "build artifacts" - i.e. you can create files and such as part of your build and have those known to CC.rb, where it then links to them in the summary of your build. You may or may not need it, but it's also very handy for simply showing you the Git tag you created on a successful build. I just touch a file in the build artifacts dir with the name of the Git tag, and then I don't have to dig through the output of the build to find my Git tag.

That's all the time I have for today, hopefully this is also helpful to others...

Loading mentions Retweet
Filed under  //   ContinuousIntegration   Hudson   Integrity   Testing  

Comments [0]