mreq

…a blog about ubuntu, sublime text and the web

While creating a semi-CMS rails skeleton/template (see shle), I've been looking into ways of loading app data during development (and to kick off the production database). Currently, there are no good options for loading complex data structures including relations. That's what led me to writing a custom lib for the task.

Imagine the following models:

# app/models/category.rb
class Category < ActiveRecord::Base
  has_many :pages
end
# app/models/page.rb
class Page < ActiveRecord::Base
  belongs_to :category

  has_many :content_parts,
           -> { where(contentable_type: 'Contentable') },
           dependent: :destroy,
           foreign_key: :contentable_id

  has_many :other_content_parts,
           -> { where(contentable_type: 'PageOther') },
           class_name: ContentPart,
           dependent: :destroy,
           foreign_key: :contentable_id
end
# app/models/content_part.rb
class ContentPart < ActiveRecord::Base
  belongs_to :contentable, polymorphic: true
  has_many :images
end 
# app/models/image.rb
class Image < ActiveRecord::Base
  belongs_to :content_part
  dragonfly_accessor :attachment
end

There's a Page having a content consisting of many ContentParts. Each ContentPart can have multiple Images. Such a structure is hard to seed in a human readable format. Rails has FixtureSets built-in, which are unfortunately not very handy when it comes to relations.

What do I mean by human readable? Imagine the following ymls:

# db/seeds/categories.yml
Category:
  - title: 'the first category'
    foo: 'bar'

  - title: 'the first category'
    foo: 'not bar'

  - title: 'another category title'
    bar: 'foo'

Alright, there's no difference to the regular fixtures. But how about this?

# db/seeds/pages.yml
Page:
  - title: a page
    foo: bar
    category:
      title: 'the first category'
      foo: 'bar'
    content_parts:
      - markdown_text: |
            ## A h2

            paragraph
      - markdown_text: you name it
        images:
          - attachment: 'db/seeds/images/nova-green-energy.jpg'
          - attachment: 'https://placekitten.com/1170/450'
    other_content_parts:
      - text_cs: |
          ## other h2

          whatever

  - title: another page
    foo: bar
    category:
      title: 'another category title'

Note that to reference the has_one relation to Category I simply include some attributes, which are enough to find_by the correct one. The has_many relation is captured by simply providing an array of attributes.

We can create a similar yml for each model that we want to be seeded. To read the ymls and create appropriate data, we'll use the custom YamlFixtureLoader like this:

# db/seeds.rb
Category.destroy_all
Page.destroy_all

models = %w(categories pages)
paths = models.map { |name| Rails.root.join("db/seeds/#{name}.yml") }
YamlFixtureLoader.new.load! paths

The class that I'm about to present handles:

  • regular attributes using column_names
  • file attachments when stored in the attachment attribute - this can be tweaked further - there's a simple adhoc solution for local/remote links, which can be definitely improved (go fork the gist)
  • relations all of has_many, has_one and belongs_to; supports polymorphic versions as well

The code is (I believe) self-explanatory:

require 'open-uri'

class YamlFixtureLoader
  def load!(paths)
    paths.each do |path|
      load_path!(path)
    end
  end

  private

  def load_path!(path)
    yaml_data = YAML::load_file(path)
    CollectionLoader.new(yaml_data).load!
  end

  class CollectionLoader
    def initialize(yaml_data)
      @collection = yaml_data.first.second
      @model = yaml_data.keys.first.constantize
    end

    def load!
      @collection.each do |item|
        ModelLoader.new(item, @model).load!
      end
    end
  end

  class ModelLoader
    def initialize(data, model, build_from = nil)
      @data = data
      @model = model
      if build_from
        @instance = build_from.build
      else
        @instance = @model.new
      end
    end

    def load!(should_save = true)
      load_attributes!
      load_translations!
      load_attachment!
      load_relations!
      @instance.save! if should_save
      @instance
    end

    private

    def load_attributes!
      columns = @model.column_names & @data.keys - ['attachment']
      columns.each do |column|
        @instance[column] = @data[column]
      end
    end

    def load_translations!
      return unless @data.keys.include? 'translations'
      valid_keys = @model.translation_class.column_names
      @data['translations'].each do |translation|
        translation.select! { |key, _| valid_keys.include? key }
        @instance.translations << @model.translation_class.new(translation)
      end
    end

    def load_attachment!
      return unless @data.keys.include? 'attachment'
      path = @data['attachment']
      if path =~ /http/
        attachment = open(path).read
      else
        attachment = File.new(Rails.root.join(path))
      end
      @instance.attachment = attachment
    end

    def load_relations!
      @model.reflect_on_all_associations.each do |reflection|
        case reflection.class.to_s
        when 'ActiveRecord::Reflection::HasManyReflection'
          load_relations_has_many!(reflection)
        when 'ActiveRecord::Reflection::BelongsToReflection'
          load_relations_belongs_to!(reflection)
        when 'ActiveRecord::Reflection::HasOneReflection'
          load_relations_has_one!(reflection)
        end
      end
    end

    def load_relations_has_many!(reflection)
      column = reflection.plural_name
      return unless @data.keys.include? column
      @data[column].each do |related_item|
        model = ModelLoader.new(related_item,
                                reflection.class_name.constantize,
                                eval("@instance.#{column}")).load!(false)
        eval("@instance.#{column} << model")
      end
    end

    def load_relations_belongs_to!(reflection)
      column = reflection.name.to_s
      return unless @data.keys.include? column
      eval("@instance.#{column} = reflection.class_name.constantize.find_by(@data[column])")
    end

    def load_relations_has_one!(reflection)
      column = reflection.name.to_s
      return unless @data.keys.include? column
      eval("@instance.#{column} = ModelLoader.new(@data[column], reflection.class_name.constantize).load!(false)")
    end
  end
end

If you tweak the code further, be sure to drop me a line here or at the gist comments. I'd love to hear from you.