mlcastle (a blog)

A Lightweight “Migrations” System for CouchDB

As I’ve mentioned before, I’m working on a Rails project where we’ve dropped ActiveRecord in favor of a CouchDB database.

One thing I miss from ActiveRecord, though, is the ability to do migrations. While SQL migrations in their most conventional use—to modify the schema of our database—are of course not needed in schemaless CouchDB databases, there is still sometimes a need to have a block of code which runs exactly once on every installation of our codebase. For example, we might have a key in our documents which becomes so important that we want to break its values off into new documents, or vice versa, we might have a bunch of documents which should all get combined into attributes of one master document.

What we aren’t doing

The correct way to solve this problem, some people argue, is to simply put some smarts into our model and support versioning there:

   1  class MyClass < CouchRestRails::Document
   2    attribute :version
   3  
   4    def stuff
   5      if self.version && self.version >= 2
   6        Stuff.by_parent_id(:key => this.id)
   7      else
   8        self["stuff"]
   9      end
  10    end
  11  end

No thanks

But that seems like a lot of work to maintain. If our system had millions of models in it and converting them was expensive, we might have no choice but to do it, but I’d rather just convert all existing models to our new schema (or, rather, our new but still schemaless way of being) and write all our code to use the updated way of thinking.

So, I designed a simple, lightweight “migrations” system. It lives entirely in Rake and is only a couple dozen line of code.

Here it is, in its entirety:

   1  desc "Perform unperformed CouchDB migrations"
   2  task :migrate => :environment do
   3    schema = begin
   4               CouchRestRails::Document.get("migration-list")
   5             rescue RestClient::ResourceNotFound
   6               CouchRestRails::Document.new(:_id => "migration-list")
   7             end
   8  
   9    migrations = Rake::Task.tasks.select{ |t| t.name =~ /^migrations:/ }
  10  
  11    begin
  12      migrations.sort{|a, b| a.name <=> b.name}.each do |migration|
  13        unless schema[migration.to_s]
  14          puts
  15          puts "== running #{migration.to_s}"
  16  
  17          migration.invoke
  18  
  19          schema[migration.to_s] = true
  20        end
  21      end
  22    ensure
  23      schema.save
  24    end
  25  end

Download the code and stick it in lib/tasks/migrator.rake. Then, create a new file, lib/tasks/migrations.rake, and create some tasks in the :migrations namespace. Here’s an example file you can modify to suit your needs. If you end up with a lot of migrations, you can of course split them among several files.

Any un-run migrations will be run in lexical order, so you can prefix your tasks with consecutive integers as shown in the example, with timestamps, or with whatever else makes you happy, as long as you are consistent within any one project.

Then, doing rake migrate will run any migrations which have not yet been run against your database.

This system has its limitations of course: among them is the need to stick everything in rake, possibly causing slowness if you end up with too many migrations. But as a lightweight, judiciously used replacement for ActiveRecord migrations, I find that it is sufficient for our needs.