VMware Virtualization for Mac Beta released!

I've been waiting for the ability to use the same VMware guest images on my linux servers as my MacBook Pro. This means I can replicate my production servers on my laptop, removing the need to have direct access to any other computer. Just in time for India!



I was able to point it at copy of an ubuntu image currently running in production and it booted without a problem. Nice! Not all the options for creating a new image are available in this release.

Download it here.

Setting environment variables for mongrel

We were going to use Verisign's PayFlowPro merchant facilities and implemented a solution based around a ruby library called pfpro4r. A problem came up though. We needed to have a shell environment variable set in order for this to work. This was fine when I started mongrel manually but when started via Capistrano the environment variable wasn't being set by the mongrel start script.

You can't seem to set environment variables in the shell Capistrano uses so I simply added a couple of tasks to deploy.rb. These override the tasks that come with mongrel_cluster and set the required variables.

I think it's great that you can add or override tasks on a project by project basis.
I guess someone distributing their app could include a library like deprec and include it from their deploy.rb to make deployment that much easier.

desc <<-DESC Start Mongrel processes on the app server. This uses the :use_sudo variable to determine whether to use sudo or not. By default, :use_sudo is set to true. DESC task :start_mongrel_cluster , :roles => :app do
 set_mongrel_conf
 send(run_method, "sh -c '
 LD_LIBRARY_PATH=/var/www/apps/example_app/current/vendor/libs/pfpro4r/verisign/payflowpro/linux/lib;
 export LD_LIBRARY_PATH;
 PFPRO_CERT_PATH=/var/www/apps/example_app/current/vendor/libs/pfpro4r/verisign/payflowpro/linux/certs;
 export PFPRO_CERT_PATH;
 mongrel_rails cluster::start -C #{mongrel_conf}
 '
 ")
end

desc <<-DESC
Restart the Mongrel processes on the app server by starting and stopping the cluster. This uses the :use_sudo variable to determine whether to use sudo or not. By default, :use_sudo is set to true.
DESC
task :restart_mongrel_cluster , :roles => :app do
  set_mongrel_conf

  send(run_method, "sh -c '
  LD_LIBRARY_PATH=/var/www/apps/example_app/current/vendor/libs/pfpro4r/verisign/payflowpro/linux/lib;
  export LD_LIBRARY_PATH;
  PFPRO_CERT_PATH=/var/www/apps/example_app/current/vendor/libs/pfpro4r/verisign/payflowpro/linux/certs;
  export PFPRO_CERT_PATH;
  mongrel_rails cluster::restart -C #{mongrel_conf}
  '
  ")
end

ssh compression can slow you down

[I've reposted this post on ssh compression to my new blog as I keep using it myself]

When copying some large files between hosts at our rack I noticed some pretty poor transfer speeds. Having recently forked out $70 for a rack mount gigabit switch I wondered what was slowing things.

It seems ssh was trying to help out by compressing everything however compressing the data took more than twice as long as transferring the uncompressed data. I proved this by disabling compression at the commandline and seeing the transfer speed more than triple.

mbailey@modu1:~/vm$ scp -r r03 modu4:vm/
Ubuntu 64-bit-f001.vmdk                         7%  147MB   8.1MB/s   03:54 ETA

mbailey@modu1:~/vm$ scp -r -o 'Compression no'  r03 modu4:vm/
Ubuntu 64-bit-f001.vmdk                       100% 2048MB  28.1MB/s   01:13 ETA

So how can we put this into practice everywhere?

Setting 'Compression no' in /etc/ssh/ssh_config will do the trick for me. There's no need for my vmware hosts to be compressing files I copy between them. I do want compression when copying files over shared/slower links but I can achieve that by initiating connections that benefit from compression on boxes that are configured to use it.

If I want to enable compression I can always use the '-C' switch to enable compression.

deprec - install rails stack and deploy in minutes

Update 2007-06-04: Visit www.deprec.org


Until now, installing a full Rails stack on Ubuntu linux and deploying your app took more than a few minutes. The first time I tried it took hours. Even copying and pasting from my notes took forever.

Now with deprec (deployment recipes for Capistrano) you can install a rails stack and deploy your application with just 6 commands.

cd /path/to/railsapp
deprec --apply-to . --name projectname --domain www.projectname.com
# edit config/deploy.rb to put in details of your subversion repository
cap install_rails_stack
cap deprec_setup
cap deploy_with_migrations
cap restart_apache

That's all it takes to get your Rails app running on a default Ubuntu 6.06 server install. Install the gem with:

sudo gem install deprec -y    # installs what you need
deprec_dotfiles               # patches capistrano + creates your ~/.caprc
cap show_tasks                # now you have deprec tasks included

I'll be putting more details up soon on deprec.rubyforge.org

Deprec was inspired and uses the brilliantly executed Capistrano. Thanks Jamis!

After starting on this project I found myself reading and utilizing a lot of
code by Bradley Taylor, in particular the RailsMachine and VMbuilder gems.
Neil Wilson wrote three capistrano plugins that are used in this project.
I'd like to say a huge thanks to these guys for helping make my work easier!

gem_plugin is clever

gem_plugin is a rubygem with a difference. Zed wrote it to allow people to easily publish plugins for his mongrel httpserver. If you create a gem with mongrel and gem_plugin as dependencies, gem_plugin will load your gem and run its init.rb after mongrel has loaded.

This makes it really easy for end users wanting to use your plugin. They just run

sudo gem install your_plugin__name

and the next time they run mongrel your plugin will be loaded. Very nice. The app doesn't need to be told to load the plugin.

rails code dissection

The first article on The Rails Way reviews the code of Tracks, a Rails site based on GTD.
I took note of the site when I saw it was co-run by Capistrano creator and Rails core team member Jamis Buck.

I really like the formatting and display of the code samples and the layout in general looks nice enough that I'm keen to try out Mephisto. Hopefully the layout sugar comes free with it.

The actual discussion of the code is great. I'm looking forward to future posts!

i made my first rubygem!

In an effort to make the world better for others I made my new recipe library for capistrano into a ruby gem. It took a little doing as there didn't appear to be a concise example on the rubygems site of how to construct the gemspec file. Dave Thomas's Pickaxe book covered it well though.

So here's my quick and dirty guide to creating a gem.

Make a file called projectname.gemspec

require 'rubygems'
SPEC = Gem::Specification.new do |spec|
  spec.name = 'deprec'
  spec.version = '0.1.1'
  spec.summary = 'deployment recipes for capistrano'
  spec.description = <<-EOF
      This project provides libraries of Capistrano tasks and extensions to
      remove the repetative manual work associated with installing services
      on linux servers.
  EOF
  spec.require_path = "lib"
  # spec.platform = Gem::Platform::Ruby
  # spec.required_ruby_version = '>= 1.6.8' # I don't know
  spec.add_dependency('capistrano', '>= 1.2.0')
  spec.files = Dir['lib/*.rb']
end

Then run these commands to build and install the gem

gem build deprec.gemspec
sudo gem install deprec-0.1.1.gem

I applied for a rubyforge project today and aim to publish the gem when it gets approved.

at last! ubuntu server on mac with parallels

After three attempts and several hours wasted I managed to get ubuntu 6.06 LTS server running on my Mac under Parallels. The secret is to download the PC (Intel x86) alternate install CD and select 'server' at the boot prompt. Oh, and select 'Other Linux Kernel 2.6' because Parallels doesn't explicitly support the most popular Linux distro out there at the moment.

RubyOnRails pagination fix

update: Rails Pagination is ugly and will be deprecated.

Here's a fix I made to get around a problem in Rails Pagination.

# pagination links do not support all form objects properly
# a hash with values such as date[year], date[month] and date[day]
# will be included in the pagination link as date=month11day9year2006
#
# putting this function in application_helper.rb and calling it from your
# view will properly construct the pagination links to include your form
# input
#
# <%= pagination_links(@cdr_pages, :params => marshall_objects(params)) %>
#
def marshall_objects(params)
  params.each do |key, val|
    if val.class == HashWithIndifferentAccess
      params[key] = nil
      val.each {|sub_key, sub_val| params[key + '[' + sub_key + ']'] = sub_val}
    end
  end
  params
end

vmware as sysadmins best friend

when doing something new like installing an addon to our most excellent nagios monitoring system i like to take notes. this is so that i can do it faster next time. i tend to take notes in a form that ends up getting turned into a script. if it's worth doing again, it's worth automating. one of the problems i've found in the past is that i don't get a change to test my script. sometimes a leave out a command. or perhaps at the end of solving a problem i decide that some of the things I did just weren't necessary. this is when virtualization and snapshots come in.

before making changes i take a snapshot of the virtual OS. in vmware server this is a simple mouseclick. then i get to work making my changes and writing my script/doc. when i have completed the task and am happy with my script i hit 'revert to snapshot' on vmware and it takes the OS back to the state it was in before i made the changes. now i can run my script, check it worked and repeat until i have automated the process i have spent the afternoon (or weekend!) trying to perfect.

thankyou vmware!