Cobalt Edge

[Update: I resolved the Git issue, and have now switched to using Hudson. The advantages of it's in-progress display, ability to more closely monitor and/or kill a build, and my impression of it being more stable, won the day.]

After switching our CI server to Integrity, there were a few blips, one of which is that we were hitting swap (memory). Hitting swap is not surprising and I'm surprised it hasn't been a problem sooner, since memory is our #1 battle. I figured if I was going to up the memory on our CI server, I might also try Hudson, as that was the main reason for not trying it previously.

The short story is that I tried it, and we're back to Integrity, but I learned some interesting things. The following is both some notes on installation, as well as some reasons why it didn't work out...

Installing Hudson

Our CI box is a slice at Slicehost, running Ubuntu 8.04 LTS, and I'd just upped it from 256MB RAM to 1GB. Also of note, I've built and install Git 1.6.4 for this system. The following are brief notes on getting Hudson going as it wasn't quite as simple as most folks made it out to be...

First, I followed the directions on the Hudson site for installing Hudson from the Ubuntu package. This amounts to adding the package repository, updating apt-get, and then doing:

sudo apt-get install hudson

This installs all the Java 1.6 stuff you need (and mine seems fine with 64bit Java), and other dependencies. After the install, Hudson is running, on default port 8080. Next up I added a server block/configuration in Nginx for Hudson (and unlinked/removed Integrity). I then went to Hudson in my browser. What I got was an error message about the xstream library. Fixing that was easy, as it turned out: downloaded the latest (1.320 at the time) hudson.war from the Hudson site, and replaced /usr/share/hudson/hudson.war with it. Restarted Hudson, and voila, now it was up and running.

Also, Hudson runs as the user "hudson", so I needed to add an SSH key for that user, and then add the public key to GitHub. And, setup a ~hudson/.gitconfig as needed. Finally, as I found later, do a git clone or an SSH to github so that you get past the whole SSH authenticity question when you first SSH to an unknown server. Note, the Hudson user is not an interactive user, i.e. you can't directly login as that user, so to gen the SSH key, you'll need do something like su to root and do, sudo -u hudson ssh-keygen -t dsa.

Configuring Hudson

Before adding a project, I needed to config some plugins. I went to the Manage page, clicked on Installed, and turned off the Subversion plugin and restarted. Next was going back in to manage plugins, and installing the Git, Github, and Rake plugins, and again restarting. Both restarts showed an Nginx bad gateway error, but simply refreshing got it back (probably just needed more time for Hudson to restart).

Then to configure a build, from the main Hudson page after a fresh install, click the "create new jobs" link. In the ensuing form, enter a project name, and select "Build a free-style software project".

Next:

• checked "Discard Old Builds" which then shows you options (so you can put in keep for X days, or X number of builds)


• Added URL for Github project, http://github.com/yourproject/yourproject


• Select Git as the SCM, and entered by git@github.com:yourproject/yourproject.git URL for the repository


• Turned on merge after build option. This will supposedly add tags for the build to your code base and then merge them back in. More on this in a bit.


• Next I configured the build steps for my project. All I really did here was take the same build steps I used for Integrity, and added them as individual shell and rake tasks.

At this point, I fired off a build (truth be told, I started with just a single build step to vet it), and things worked, with the exception of the very last step, where I push the Git tags back to GitHub. This is what I saw:

[workspace] $ /bin/sh -xe /tmp/hudson1444107192962944065.sh
+ /usr/bin/git push --tags
XML error: syntax error
error: Error: no DAV locking support on https://github.com/dealbase/dealbase/
error: failed to push some refs to 'https://github.com/dealbase/dealbase'

I've now been searching for answers to this, and haven't yielded anything. I've tried the git push directly from a shell, with the same result. If I do this as a different user (e.g. under my user) it works fine. This git push is attempted both as part of a rake task (the ci_tag task), as well as I tried making it just a straight shell command in Hudson.

After a lot of googling, and asking, and no resolution in sight, I've gone back to Integrity...

Comparisons and Notes

First off, I think Hudson looks pretty stellar. There are a TON of plugins for it, and it seems quite mature and polished. The essentially 100% configuration via the GUI is slick. Install, despite a few hoops above, was actually pretty painless. So, here's just a few notes/opinions:

Hudson

• You will need some memory to run Hudson, more than with Integrity or CruiseControl.rb. From what I can tell, you probably want a system with 1GB or more. Various other folks I talked to all had 2GB or more systems, and their Hudson processes were taking 1.5GB or more. This is partly just a Java thing. It should be noted, the others were running more than one build with Hudson, and mine seemed to work fine on a 1GB total memory system (didn't seem to hit swap).


• Hudson allows you to kill a build while it's running (nice!).


• Hudson works with CCMenu/CCTray out of the box.


• The git integration has more options in terms of picking branches, doing merging, and various more involved operations, but doesn't have GitHub post-receive hook support out of the box (there are plugins up on GitHub for it, but not listed in the standard plugin list).


• Hudson runs as user "hudson", which is a user that has no shell. You could change this, although the idea is you shouldn't have to. However, this complicates setting up SSH keys and various things. And, of course, I had the issue with Git as mentioned above. I could probably change this to run as my user and so on, but part of all this for me is not having to change a lot of defaults and start messing with core configurations/designs of the system. In part, I just don't have time to do that, and to maintain it (these kinds of changes often cause problems when you upgrade versions, etc. - also known as you may forget to redo these changes if an upgrade undoes them :).


• The UI and web app itself is quite nice, understandable, well done in general for something like this.

Also, thanks to the Tea-Driven blog for motivating me to try Hudson (and for some tips on Testjour - more on that in another post).

Integrity

I really only have one main complaint about Integrity, and that has to do with indication of a build in progress. It essentially doesn't indicate it - it says it hasn't been built yet, but a build may actually be running. The CCMenu/integritray plugin will show you that a build is running, so this mostly solves my problem, but this seems like a core failure. I may have to look at fixing this. And that is a win for Integrity in that it's Ruby, and thus I'll be more apt to go fix this (while I spent MANY years doing heavy Java work, I don't have interest in working in that code anymore for a task like this).

CruiseControl.rb

One thing that remains an advantage for CC.rb is that it has "build artifacts" - i.e. you can create files and such as part of your build and have those known to CC.rb, where it then links to them in the summary of your build. You may or may not need it, but it's also very handy for simply showing you the Git tag you created on a successful build. I just touch a file in the build artifacts dir with the name of the Git tag, and then I don't have to dig through the output of the build to find my Git tag.

That's all the time I have for today, hopefully this is also helpful to others...

Loading mentions Retweet

Recently I switched the DealBase continuous integration server to use Integrity instead of CruiseControl.rb. This happened because I'd been having some sporadic failures under CC.rb that didn't seem explainable (no code would change, and tests would fail at random), and also due to some changes in Cucumber between versions, it all conspired to switch.

It should be noted that, as you might guess, the "random" failures were not exactly random, but suffice it to say that the root cause re-inforced my notion to switch. My main beef with CC.rb has more to do with it being somewhat in bed with Rake and wanting to run your CI build via a rake task, and some of the issues (or impurities?) that come up with that. But I'm boring you...

Anyway, Integrity... Setup is relatively easy, and is covered fairly well on their setup page/docs. But the following is what I did, which I'm documenting for myself and anyone else who may find it useful.

I setup my server to use with Nginx and Passenger. I tried using Integrity with Nginx and Thin, but wasn't able to get Integrity to work right (similar to the results defunkt had when they tried it at GitHub). We've standardized on Nginx+Passenger, so this was good anyway. I nuked the Nginx I had already and proceeded (all this being done on an Ubuntu Hardy VPS system at Slicehost):

sudo gem install passenger
sudo passenger-install-nginx-module

I let it install Nginx, and picked the default location. I then re-added our HTTP Basic Authentication and a few other Nginx tweaks as I had in the prior Nginx configuration. Next up was to install some prerequisites and Integrity itself:

sudo apt-get sqlite3 libsqlite3-dev
sudo gem install integrity do_sqlite3 thin
integrity install /home/ci/integrity --passenger
cd integrity

Next you'll want to tweak config.yml to customize the domain where you'll access your CI server. I left the rest the same. Then setup Integrity's database:

integrity migrate_db /home/ci/integrity/config.yml

I also added the "integritray" plugin, so that I could continue to use CCMenu to monitor my builds. See the integritray GitHub page for simple install.

Now point Nginx/Passenger at your install, by adding the appropriate server block for a Passenger Rails app, such as:

server {
    listen       80;
    server_name  your.ciserver.com;
    root /home/ci/integrity/public;
    passenger_enabled on;
    auth_basic            "Restricted";
    auth_basic_user_file  /opt/nginx/conf/htpasswd;
}

Finally, fire up Nginx, and surf to your CI server domain/URL. You should see something like this:

Uploaded with plasq's Skitch!

Click the "create your first project" link, and enter details about your app. The "Git repository" field should get your GitHub or other git server URL (e.g. your clone URL). For a build script, the following is what I used:

rake log:clear && RAILS_ENV=test rake db:reset && spec --options spec/spec.opts spec/**/*_spec.rb && RAILS_ENV=cucumber rake db:reset && cucumber --strict -q --format pretty features && rake ci_tag

This is one nice thing about Integrity - it's pretty much any command you can give it. Sure, you might want to wrap that up in a Rake task, or a shell script or however you want to do it. I just entered that raw so it's overly obvious exactly what it's doing. I also found this to work better than doing it as a Rake task, as somehow I wasn't getting the environment to switch properly under Rake. The "ci_tag" rake task is my task for tagging/labeling succesful builds in Git, etc.

Next, you'll want to setup a post-commit service hook on GitHub. You can get your Push URL by clicking the edit link for your project in Integiry:

Uploaded with plasq's Skitch!

Note, that works even with HTTP Basic Auth, just add your user credentials to the Push URL before pasting it in on GitHub.

Finally, fire off a build. And note, one downside of Integrity is that it doesn't indicate (in the web UI) that a build is underway. It just says it hasn't been built yet. The integritray item and thus CCMenu will show you that it's building though.

Lastly... some have asked why not XYZ CI server? A few notes on this:

• Hudson: this looks awesome, but also somewhat overkill for our needs. I have one project to build, and I'm doing it on a very inexpensive slice that has only 256MB of RAM - I doubt I could even start Hudson in that little RAM, being it's a Java web app, etc. Secondarily, I'd prefer to have the app be Ruby so I can hack on it (I've made at least very minor tweaks to every CI server I've used to date).


• cruisecontrol.rb - this is what we were using, and it worked well, with minor exception to some random failures and the Rake-oriented build process. I'd really say this is minor though, and would suggest folks try it out. You can of course refer to my previous writeup on setting up CruiseControl.rb :)


• CI Joe/cijoe - this is actually what I started with when I looked at exploring alternatives. I had problems getting the build working properly, which seems odd. In hindsight that may have wound up being due to some problems during our switch to a newer version of Cucumber. But, one thing I didn't like is that there is no state maintained, so if you stop and start cijoe, it loses track of all its previous builds. This may or may not matter much to you, but I didn't like that. I also didn't want to spend time writing notifiers/CCMenu integration type stuff for our needs. I will ay that cijoe setup/install is pretty cool.


• Run Code Run doesn't have a viable plan for us yet, plus I've heard having it run custom rake tasks and such doesn't work (wrong?). Furthermore, I didn't really want our private code on their servers and didn't see a need to outsource this.


• others... either hadn't heard of them, they didn't work well with Rails, or whatever - upshot, Integrity worked, got it up and running fairly fast, and didn't need to spend any more time on this.

Loading mentions Retweet

I too read Al3x's interview the other day, and like John Nunemaker, figured I'd share my setup, as I enjoy reading what others use and often can pick up a few interesting tools or tidbits.

Unlike Mr. Nunemaker, my desk is too messy, IMHO, to photograph right now :) However, many similarities aside from that. On with it...

I use a 17" MacBook Pro with 4GB RAM as my only machine these days. Like Alex and John, I really like having just a single machine, and I no longer work for a corporation where I'd worry about that. DealBase is cool and wouldn't try to make some wacko claim to some work not relevant (and we've explicitly discussed my use of a single machine, etc.). I have my MBP open on a laptop arm from Ergotron, and then my primary monitor is a 30" Dell. Really love the big monitor. I do my main work o the 30", and then the laptop screen has TweetDeck, iChat, Things, some Fluid apps, and other things that I tend to more glance at, and aren't primary work items.

Further, I use a wireless Apple keyboard, and like John, I just love this thing. I can't tell you how long I'd been looking for a keyboard that was just a keyboard (but with arrow keys). I hate normal keyboards that take up so much extra space on the right side (my mouse side) with stuff I rarely use - which only exacerbates problems with having my arm/elbow canitlevered further out to use the mouse, sometimes causing arm strain after long days of coding. I use Logitech MX Revolution cordless mouse, which I like quite a lot.

Transitioning to music... I use JBL Creature speakers, and listen to a variety of things, or nothing. Pandora, via a Fluid app, iTunes (my own playlists, or various Ambient "radio" stations), etc. Either that, or we have a whole-house NuVo Concerto audio system, so sometimes I have that on either with XM satellite radio, or to a playlist from the iPod we have hooked into it. The NuVo setup is nice because it fills my office with sound a bit better (via in-ceiling speakers), but I have more variety via the computer.

As with Alex and John, I am absolutely in love with my iPhone 3G. It is even better than expected. It has essentially replaced my 80GB iPod in the car, typically because it's more up to date, and I like it's UI better; I can remotely work on servers if I have to via iSSH, play games if I'm bored, use InstaPaper to read things I've set for reading later, sync with Address Book and iCal, and of course Twitter, via Tweetie. So, yes, I use Apple's Address Book and iCal, for great sync, simplicity, etc.

Ok, onto dev stuff. My primary work is on Rails-based web-apps, although I dabble with other things as well. DealBase is my day job, and I'm also involved with Bring Light.

Yet again, like Alex and John, I spend the bulk of my time in TextMate, iTerm (a better Terminal, IMHO), and Safari. And actually, I do my development testing in nightly builds of WebKit/Safari, and all my other browsing in standard Safari. I do pull up Firefox for testing, and to use YSlow and sometimes Firebug (although I've been finding the dev tools in WebKit nightlies work well). I've used Emacs - did so for about a year when working with Linux as my desktop. I ditched it back then in favor of Visual SlickEdit, but these days TextMate just rules. I don't get the Emacs passion - why do you want to press two keys for everything, especially the most common things? Yes, I know, you can setup different bindings, etc., but come on the most basic things like saving, opening, copy, paste, etc. should be "single" key (and by single I mean some meta+key) strokes by default. I do fire up vi all the time at the command line on remote servers, and even occasionally on my MBP for some real quick edit. Also, I spend the bulk of my day in my text editor, so yes, appearance matters, and TextMate kills others. I've also used a lot of IDE's in the past, from IDEA, to Eclipse, to Visual Studio. Visual Studio is actually quite good if you have to suffer in that world, but I find Eclipse just plain crappy. IDEA was great for Java, and their Ruby setup will be something to keep an eye on, but generally, the setup I have now works well.

I have all my code for nearly everything I do (e.g. both private and open source/public) on GitHub, and truly love it. Git has been a huge win, and gives me the best of, as well as improving SVN and Perforce. I'm using GitX for most of my commits and history browsing these days.

I use RSpactor for continuously running our RSpec suite, and we also use RSpec stories (but haven't converted to Cucumber yet). I recently added speech output to RSpactor, and that is my preferred notification instead of Growl. We use Pivotal Tracker for tasks/stories/features as well as bug tracking. We used to use Lighthouse, but having it all in one place was nicer, and Tracker wins big time in my opinion. If you want GitHub post-receive hook for Tracker, I recently whipped that up, and its been a real nice addition. We too use Hoptoad for exception notification, and really like it. Also, New Relic is in use at DealBase. I also like viewing Google Analytics with Analytics Reporting Suite, a slick AIR app.

I really like Navicat as a GUI for database stuff. It's proprietary/pay software, but honestly, it's worth it to me. I can do all this stuff command line fine, but the GUI simply makes it a heck of a lot faster to view the results, quickly re-sort on a column, mess around with queries, etc. Also, it has great SSH support, so I can tunnel into all my server's DB's with ease.

I have CruiseControl.rb setups for all my Rails apps, and make use of CCMenu for a nice little status menu item showing me what's going on with those.

I pretty much can't live without LaunchBar. Same goes for 1Password.

Skitch is quite handy for showing sharing and annotating screen shots, and we use Google Docs and Gmail. Speaking of email, I am a huge fan of Mailplane, which is a Mac app for Gmail. Integration is superb, and I can quickly switch around my 15 or so Gmail accounts with ease. I find it superior to a Fluid app for Gmail, since the integration is better and it handles multiple accounts.

I host most of my own web apps on Slicehost, and DealBase is at EngineYard.

I also use Backpack some, although not nearly as much as I used to, and access it about 99% of the time via Packrat. MarsEdit is my blog authoring tool of choice. NetNewsWire is my RSS reader.

All of my photography and photo processing, etc. are done in Adobe Lightroom. I use the Flickr plugin for it as well.

Various other bits:

• TextPander


• WeatherDock


• Pukka


• Flickr


• Del.icio.us


• xScope - a great screen ruler app


• Photoshop CS3 (look for my name in the about box too :)


• JungleDisk - I do some backups with this


• SuperDuper! Still my favorite backup, although I use TimeMachine too


• CSS Edit and XyleScope sometimes


• Last.fm - is running all the time, but I really don't actually make use of it, kinda silly.


• Acrobat Pro and Reader


• XCode (or TextMate) if I'm working on an Objective-C/Cocoa app.


• iStat menus


• YouControl Tunes




p.s. One other bit I can't live without but really isn't computing hardware/software, is my espresso setup. I use an Expobar Brewtus II machine, Macap MC4 stepless doserless grinder and a variety of cups (mostly Nuova Pointe and Illy). I use only totally fresh beans from a variety of places (favorites include Blue Bottle, Ecco Caffe, PT's, 49th Parallel (unfortunately not often, since shipping from Canada makes it a bit cost prohibitive), etc.). Coffelab tamper and Bumper stand and knock box. My espresso bar is kept clean (unlike my desk). The pictures are a bit older, so don't show bottomless portafilter in use these days.

Whew, that's more than plenty. What's your setup?




Loading mentions Retweet
Filed under  //   ContinuousIntegration   CruiseControl   DealBase   environment   Espresso   git   Gmail   iPhone   laptop   Mac   Nuvo   Office   Pivotal Tracker   Rails   RSpactor   Ruby   TextMate  

After discovering that autospec was taking up a lot of CPU while it was "idle", I looked for alternatives. I found RSpactor, which doesn't take as much CPU, and is better since it's a dedicated window with nice GUI results and so. My only gripe was that I'd gotten used to the spoken results output I'd rigged up for Autospec.

I really prefer the spoken results because it is not visually distracting, and doesn't require me to be paying attention to the area of the screen where they pop up (I use a 30" monitor, plus the 17" laptop monitor, so I'm not always looking at the right spot for the Growl notices, and I don't like the monitor-wide growls). Lucky for me, RSpactor is open source and is up on GitHub.

I thought it was an Objective-C Cocoa app, but as it turns out it's a RubyCocoa app. I'd built RubyCocoa apps before, so was familiar with that, plus of course know Ruby. It wouldn't have mattered either way (I'm fine working in ObjC as well), but this did make things a slight bit faster.

Anyway, did a quick bit of work and got a new preferences panel for Speech added, and then rigged that up to test results, so that I now have my desired spoken results. A slight improvement comes along in that it (optionally) speaks the number of passing/failing/pending tests - just insert a question mark in the string/phrase you want spoken for each and it'll say the number at that spot.

I've sent a pull request to RubyPhunk, but no guarantees it will get added to the main line. In the mean time, if you're interested, grab it from my RSpactor fork on GitHub. Update: RubyPhunk integrated my changes into the main RSpactor code.

Loading mentions Retweet

When testing web apps for a general/wide-ranging audience, one must test across a slew of different browsers and operating systems (and different versions of each). This is a real pain, even if you have an army of testers at hand (which most of us don't). I employ various tools to help me with this (CrossBrowserTesting.com, BrowserCam, VM Ware with different VM setups, etc.). But, the reality is that I don't feel like these really stack up to what I need. More specifically, BrowserCam, and its competitors do not. CrossBrowserTesting is actually pretty awesome and does what it advertises quite nicely, it's a favorite tool right now.

What I'm after is a way to see what a survey of pages on my site look like on different browsers and OS's. This is by no means a definitive test of a site, but more of a quick visual inspection of appearance, without having to fire up a dozen different pages across maybe 30 different browsers/OS configurations. We use an automated test suite to test the bulk of other things, but appearance can't be tested that way.

BrowserCam helps, but honestly I find it quite lacking as a tool in this arena. Note, I'm picking on BrowserCam, as that's the one I use, but the others (BrowserShots, Litmus, etc.) all seem to suffer some or all of these problems, and further, they don't seem to have projects or ways to establish a standard suite of tests. Getting on with it, all of this really boils down to two primary things:

• Ability to edit the settings on a project's images. As far as I can tell, once you select the URL's, image size, browsers, etc. for a project, there is no way to change any of that! What if I just want to change the window size? Can't, I have to create a new project (or add new images to the existing one, replicating all my previous work. This seems like an obvious hole.


• Automation. I want to be able to automate regeneration of the project. I'd like something like a simple URL/HTTP API to do this, so that I can, from a command line, use curl or similar to issue a single HTTP request that will regenerate a project's images. Thus, the API would need to use HTTP authentication or similar, and specify which project to regenerate. With this, I would be able to automate requesting a regeneration of the project as part of/after deploying my application.

Finally, if anyone knows of a service similar to BrowserCam that solves these, even if it supports a few less browsers, do let me know. At very minimum it would need to handle Internet Explorer and Firefox, preferably also Safari, and cover Windows XP, Vista, MacOS X 10.5, and one popular Linux flavor. Nice to haves would include Opera and various others.

Loading mentions Retweet

Early warning: this is a hack, which doesn't mean it's bad, just that it's not polished. However, I am documenting my solution for myself thus far, as well as figured others might find it useful...

Update: Added my shell command for doing deploys (see end of this post).

I wanted a way to automate a few things around deployments, and integrate this a bit with my continuous integration server. I use CruiseControl for the CI server, and previously blogged about setting up CC.rb with Git. The goals for this next task, and subject of this blog post are:

• Tag the code on successful deploys. My CI server already tags the code anytime it does a successful build, but since I didn't cover that previously, I'll mention it here as well.


• Notify a list of people via email whenever a new deploy happens.


• Generate a changelog, based on Git commit messages (better make sure they're suitable reading for whoever gets your deploy notices!), and include this changelog in the deploy emails.


• Have the CI tag I want to deploy as the only required piece of info/parameter when issuing a deploy command.

Tagging

desc "Tag the code on successful CI build"
task :ci_tag do
  timestamp = Time.now.strftime("%Y%m%d%H%M%S")
  tag_name = "CI_#{timestamp}"
  # Create an empty file with our tag name, so we can easily go grab the tagname
  # from the CI output page and do deploys, etc.
  system("touch #{File.join(ENV['CC_BUILD_ARTIFACTS'], tag_name)}")
  system("git tag -a -m 'Successful continuous integration build on #{timestamp}' #{tag_name}")
  system("git push --tags")
end

From the above, you can see that I'll get tags of the form: CI_timestamp. Next up, I want to tag a successful deploy to indicate which commit/tag actually got deployed and when. This is handled via an after task in my Capistrano deploy.rb:

after "deploy:restart", "tag_last_deploy"
task :tag_last_deploy do
  set :timestamp, Time.now
  set :tag_name, "deployed_to_#{rails_env}_#{timestamp.to_i}"
  `git tag -a -m "Tagging deploy to #{rails_env} at #{timestamp}" #{tag_name} #{branch}`
  `git push --tags`
  puts "Tagged release with #{tag_name}."
end

This will create tags like, deployed_to_staging_1213223458, and works for both staging and production (or any environment you're targeting - note the use of the rails_env variable - you may need to use something else). One thing to pay particular attention to, is that this tag is actually tagging another tag, as defined by the branch variable (mentioned above). In order for this to work though, you need to ensure that your tags are up to date locally. Thus, somewhere in your workflow you'll need to do a git pull --tags, if like me, your CI server is elsewhere and is generating those tags.

Ok, we're all tagged up, let's move on...

Notification

It turns out there's a nifty new plugin called Cap Gun that will take care of emailing a list of folks on deploy. Setup is covered in their README, but the one bit they don't mention, is that you can include a comment in the email message that goes out. I wanted to include a changelog in these emails, so I tapped into this comment attribute, setting it to the text of my changelog. To use the comment, you can either set it via -s comment="my lovely comment" on your Capistrano deploy command, or you can set the comment variable in your Capistrano deploy.rb or included script. More on that in a minute.

Changelogs

#!/usr/bin/env ruby
require 'rubygems'
require 'mojombo-grit'
include Grit
unless ARGV.length == 2
  puts "Usage: changelog.rb staging|production <commit-or-tag>"
  puts "  where commit-or-tag is the commit ID or tag you are planning to deploy"
  exit -1
end
repo_location = File.expand_path(File.dirname(__FILE__) + '/..')
target = ARGV[0]
about_to_deploy_commit = ARGV[1]
repo = Repo.new(repo_location)
# Find the tag for the last deployed
tags = repo.tags.collect {|tag| tag.name }
tags.delete_if {|tag| !(tag =~ /^deployed_to_#{target}_/)}
tags.sort!
last_deployed_tag = tags[-1]
commits_for_changelog = repo.commits_between(last_deployed_tag, about_to_deploy_commit)
commits_for_changelog.reverse!
puts "Changes since last release:"
commits_for_changelog.each do |commit|
  puts " "
  puts "  #{commit.message}"
end

To run through it briefly, it takes two parameters (and clearly, you can change this for your own deployment targets, etc.): a deployment target, and a tag (which can actually be a tag, a commit ID, branch, etc.). It sets up a repo variable for your Git repository using Grit, and then proceeds to find the last deployed tag for that deployment target. After that, it gets all the commits between that last deployed tag and the tag you specified as the second script argument, and prints out the commit messages.

To integrate this, I added this line to my Capistrano deploy.rb:

set :comment, `script/changelog.rb staging #{branch}`

As you can see, that one is specific to my staging environment, and lives inside my "staging" task in deploy.rb. Same, appropriately edited version goes for production.

Deployment Command

stagemyproject () {
        git pull
        cap -s branch=$1 staging deploy:migrations
}

You would thus have a command line to do a deploy like this:

stagemyproject CI_20080612052417

That's it, and if you've managed to read this far, congrats, and if you've not only managed to read this far, but payed attention and got value out of it, well, cool.

For anyone who uses/adapts this, please do let me know improvements you make, or suggestions, or tweaks/changes, and so on. I've been using this for all of about a half dozen deploys so far. If (more like when) I make improvements, I'll update.

Loading mentions Retweet

[Updated to refer to official ThoughtWorks CC.rb Git repo.]

I have a new Rails project I'm working on and I use Git/GitHub for source control. It was time to setup continuous integration, and my usual weapon of choice for that is CruiseControl.rb. Here's what I did to get my project setup under CruiseControl.rb with Git, on an Ubuntu 7.10 machine...

Setup for accessing GitHub repo

Prerequisites

• I setup a builder@mydomain.com email address which will get used by CruiseControl for sending build related emails/notifications.


• You'll need to determine a port you want CruiseControl to run on, and your strategy for accessing it. For example, I run mine on a port other than port 80, and other than the default 3333. I then proxy that via Nginx, and also use Nginx to password protect access to it (since this is not a public project, etc. This will affect the CC dashboard URL setting specified below. Some notes on this:




• I did my initial Nginx configuration using err's Nginx config generator. However, this makes a lot of path assumptions, and various other things, so you'll definitely want to go through the resulting file closely. I had a few sites on this server, so it was relatively useful to use this as a base starting point, and then just fix up paths to the access and error logs, and the PID file.


• Here's a quicky on how to add password protection to an Nginx server (and a specific location).

Install CruiseControl and Do Site Configuration

1. Cloned the Git version of CruiseControl.rb in location I wanted it (you could also simply download it and expand the tarball): git clone git://github.com/benburkert/cruisecontrolrb.git


2. The DEPENDENCIES file indicated I needed to have the grit and mime-types gems, so installed those.


3. Where your projects get stored for CruiseControl.rb is now defined by the CRUISE_DATA_ROOT environment variable, and if you don't set this, it defaults to $HOME/.cruise. I personally changed this to be /var/cruisecontrolrb.
   
4. Edit the config/site_configuration.rb (probably need to rename the example version accordingly) to set site-wide settings, such as your email config and so on.
   
   
   
   • For email setup, I use Gmail for domains, so I have a block like this:
     

     
     ActionMailer::Base.smtp_settings = {
       :address =>        "smtp.gmail.com",
       :port =>           587,
       :domain =>         "mydomain.com",
       :authentication => :plain,
       :user_name =>      "builder@ mydomain.com",
       :password =>       "password"
     }

   
   
   • You'll want to specify the Configuration.dashboard_url setting so URL's work properly.
   
   
   • There are a variety of other settings available in the file that you may want to tweak.

Add Project and Configure

1. Did the usual usual cruise add command to add my project, but with the Git variant: ./cruise add MyProjectName --git-url git@github.com:mylogin/myproject.git (modify the Git project URL for your Git repo of course). Note that you can see all the options by doing a ./cruise add


2. Create the test database for your project. The easiest way is just to go into $CRUISE_DATA_ROOT/projects/MyProjectName/work and do a rake db:create RAILS_ENV=test. Your first build will have already failed because this hasn't been made, this step hopefully fixes that.


3. If your log directory isn't in Git, you'll need to go mkdir it, so something like:mkdir $CRUISE_DATA_ROOT/projects/MyProjectName/work/log.

Setup CruiseControl.rb Service/Daemon

1. Copy the cruisecontrolrb file into /etc/init.d.


2. I set the port for CruiseControl.rb to run on in the above /etc/init.d/cruisecontrolrb daemon file, by adding "--port 1234" (for example) to the DAEMON_ARGS variable.
   
3. Start the CruiseControl.rb daemon as appropriate for your system (e.g. "sudo /etc/init.d/cruisecontrolrb start").

Finally, surf to your cc.rb site on the web and see how your build has done. If you run into build problems, you'll want to look at the cc.rb build logs (if it was your project test/build that failed) which are in the $CRUISE_DATA_ROOT/projects/MyProjectName directory (or rather, the subdirectory in there for the particular build). And Enjoy!

Loading mentions Retweet

While I haven't had a chance to clean up the code, folks have asked for it, so I'm making my Perforce implementation for CruiseControl available. There are some important notes:

• You need to set up your project manually, you can't do an "add" via CruiseControl.
• Some work needs to be done on the text retrieval for checkin messages, and how that's displayed on the CC.rb results pages.

To install/use it:

• Put the perforce.rb file into your cruisecontrol/app/models directory.
• Manually setup your project:


• Create a directory under the cruisecontrol/projects directory.
• Place a cruise_config.rb file in it. It should contain something like the following in order to use Perforce:
  

Project.configure do |project|
# Use Perforce for source control
project.source_control = Perforce.new(
:port => 'your.perforce.server:1666',
:clientspec => 'clientspec-for-cruisecontrol',
:user => 'buildusername',
:password => 'builduserpassword',
:path => '//depot/path/to/your/rails/app/...')
end

• Sync your code once.
• Fire up CruiseControl, and let the games begin.

Update: I've now given this an official home on GitHub. See the cruisecontrolrb_perforce project there. Fork at will, and please do send me Pull requests if you enhance the code, or at least tell me about your version, and I'll put that in the README or on the wiki, etc.

Loading mentions Retweet