Cobalt Edge

When testing web apps for a general/wide-ranging audience, one must test across a slew of different browsers and operating systems (and different versions of each). This is a real pain, even if you have an army of testers at hand (which most of us don't). I employ various tools to help me with this (CrossBrowserTesting.com, BrowserCam, VM Ware with different VM setups, etc.). But, the reality is that I don't feel like these really stack up to what I need. More specifically, BrowserCam, and its competitors do not. CrossBrowserTesting is actually pretty awesome and does what it advertises quite nicely, it's a favorite tool right now.

What I'm after is a way to see what a survey of pages on my site look like on different browsers and OS's. This is by no means a definitive test of a site, but more of a quick visual inspection of appearance, without having to fire up a dozen different pages across maybe 30 different browsers/OS configurations. We use an automated test suite to test the bulk of other things, but appearance can't be tested that way.

BrowserCam helps, but honestly I find it quite lacking as a tool in this arena. Note, I'm picking on BrowserCam, as that's the one I use, but the others (BrowserShots, Litmus, etc.) all seem to suffer some or all of these problems, and further, they don't seem to have projects or ways to establish a standard suite of tests. Getting on with it, all of this really boils down to two primary things:

• Ability to edit the settings on a project's images. As far as I can tell, once you select the URL's, image size, browsers, etc. for a project, there is no way to change any of that! What if I just want to change the window size? Can't, I have to create a new project (or add new images to the existing one, replicating all my previous work. This seems like an obvious hole.


• Automation. I want to be able to automate regeneration of the project. I'd like something like a simple URL/HTTP API to do this, so that I can, from a command line, use curl or similar to issue a single HTTP request that will regenerate a project's images. Thus, the API would need to use HTTP authentication or similar, and specify which project to regenerate. With this, I would be able to automate requesting a regeneration of the project as part of/after deploying my application.

I'm currently looking into creating my own script to drive the automation, but it's looking non-trivial due to the way BrowserCam's pages work: everything winds up on the same URL, actions/links are all JavaScript that post forms, with a lot of params and a bunch of params that will take some time to ascertain if a) they're required, and b) all the info in the parameter is needed, etc. If someone's already done this, let me know! Also, suggestions for tools to drive the automation? I've used the Ruby Mechanize library in the past for things like this, and this may work if I can determine the params, etc., and if Mechanize can even drive the JavaScript links (not sure it can - anyone?). I can't use something like Selenium or Windmill because this needs to work from a script/command line, and not rely on a browser.

Finally, if anyone knows of a service similar to BrowserCam that solves these, even if it supports a few less browsers, do let me know. At very minimum it would need to handle Internet Explorer and Firefox, preferably also Safari, and cover Windows XP, Vista, MacOS X 10.5, and one popular Linux flavor. Nice to haves would include Opera and various others.