ssh compression can slow you down

[I've reposted this post on ssh compression to my new blog as I keep using it myself]

When copying some large files between hosts at our rack I noticed some pretty poor transfer speeds. Having recently forked out $70 for a rack mount gigabit switch I wondered what was slowing things.

It seems ssh was trying to help out by compressing everything however compressing the data took more than twice as long as transferring the uncompressed data. I proved this by disabling compression at the commandline and seeing the transfer speed more than triple.

mbailey@modu1:~/vm$ scp -r r03 modu4:vm/
Ubuntu 64-bit-f001.vmdk                         7%  147MB   8.1MB/s   03:54 ETA

mbailey@modu1:~/vm$ scp -r -o 'Compression no'  r03 modu4:vm/
Ubuntu 64-bit-f001.vmdk                       100% 2048MB  28.1MB/s   01:13 ETA

So how can we put this into practice everywhere?

Setting 'Compression no' in /etc/ssh/ssh_config will do the trick for me. There's no need for my vmware hosts to be compressing files I copy between them. I do want compression when copying files over shared/slower links but I can achieve that by initiating connections that benefit from compression on boxes that are configured to use it.

If I want to enable compression I can always use the '-C' switch to enable compression.

deprec - install rails stack and deploy in minutes

Update 2007-06-04: Visit www.deprec.org


Until now, installing a full Rails stack on Ubuntu linux and deploying your app took more than a few minutes. The first time I tried it took hours. Even copying and pasting from my notes took forever.

Now with deprec (deployment recipes for Capistrano) you can install a rails stack and deploy your application with just 6 commands.

cd /path/to/railsapp
deprec --apply-to . --name projectname --domain www.projectname.com
# edit config/deploy.rb to put in details of your subversion repository
cap install_rails_stack
cap deprec_setup
cap deploy_with_migrations
cap restart_apache

That's all it takes to get your Rails app running on a default Ubuntu 6.06 server install. Install the gem with:

sudo gem install deprec -y    # installs what you need
deprec_dotfiles               # patches capistrano + creates your ~/.caprc
cap show_tasks                # now you have deprec tasks included

I'll be putting more details up soon on deprec.rubyforge.org

Deprec was inspired and uses the brilliantly executed Capistrano. Thanks Jamis!

After starting on this project I found myself reading and utilizing a lot of
code by Bradley Taylor, in particular the RailsMachine and VMbuilder gems.
Neil Wilson wrote three capistrano plugins that are used in this project.
I'd like to say a huge thanks to these guys for helping make my work easier!

gem_plugin is clever

gem_plugin is a rubygem with a difference. Zed wrote it to allow people to easily publish plugins for his mongrel httpserver. If you create a gem with mongrel and gem_plugin as dependencies, gem_plugin will load your gem and run its init.rb after mongrel has loaded.

This makes it really easy for end users wanting to use your plugin. They just run

sudo gem install your_plugin__name

and the next time they run mongrel your plugin will be loaded. Very nice. The app doesn't need to be told to load the plugin.

rails code dissection

The first article on The Rails Way reviews the code of Tracks, a Rails site based on GTD.
I took note of the site when I saw it was co-run by Capistrano creator and Rails core team member Jamis Buck.

I really like the formatting and display of the code samples and the layout in general looks nice enough that I'm keen to try out Mephisto. Hopefully the layout sugar comes free with it.

The actual discussion of the code is great. I'm looking forward to future posts!

i made my first rubygem!

In an effort to make the world better for others I made my new recipe library for capistrano into a ruby gem. It took a little doing as there didn't appear to be a concise example on the rubygems site of how to construct the gemspec file. Dave Thomas's Pickaxe book covered it well though.

So here's my quick and dirty guide to creating a gem.

Make a file called projectname.gemspec

require 'rubygems'
SPEC = Gem::Specification.new do |spec|
  spec.name = 'deprec'
  spec.version = '0.1.1'
  spec.summary = 'deployment recipes for capistrano'
  spec.description = <<-EOF
      This project provides libraries of Capistrano tasks and extensions to
      remove the repetative manual work associated with installing services
      on linux servers.
  EOF
  spec.require_path = "lib"
  # spec.platform = Gem::Platform::Ruby
  # spec.required_ruby_version = '>= 1.6.8' # I don't know
  spec.add_dependency('capistrano', '>= 1.2.0')
  spec.files = Dir['lib/*.rb']
end

Then run these commands to build and install the gem

gem build deprec.gemspec
sudo gem install deprec-0.1.1.gem

I applied for a rubyforge project today and aim to publish the gem when it gets approved.

at last! ubuntu server on mac with parallels

After three attempts and several hours wasted I managed to get ubuntu 6.06 LTS server running on my Mac under Parallels. The secret is to download the PC (Intel x86) alternate install CD and select 'server' at the boot prompt. Oh, and select 'Other Linux Kernel 2.6' because Parallels doesn't explicitly support the most popular Linux distro out there at the moment.

RubyOnRails pagination fix

update: Rails Pagination is ugly and will be deprecated.

Here's a fix I made to get around a problem in Rails Pagination.

# pagination links do not support all form objects properly
# a hash with values such as date[year], date[month] and date[day]
# will be included in the pagination link as date=month11day9year2006
#
# putting this function in application_helper.rb and calling it from your
# view will properly construct the pagination links to include your form
# input
#
# <%= pagination_links(@cdr_pages, :params => marshall_objects(params)) %>
#
def marshall_objects(params)
  params.each do |key, val|
    if val.class == HashWithIndifferentAccess
      params[key] = nil
      val.each {|sub_key, sub_val| params[key + '[' + sub_key + ']'] = sub_val}
    end
  end
  params
end