Meta Programming — the most memorable code I’ve written

Meta Programming:

Programs that write programs! Or (more formally), a programming technique in which computer programs have the ability to treat programs as their data.

While working on my startup, I came across an interesting problem. In our web-app, we needed to create Portfolios of our artists (actors, models, photographers, etc.). We were constantly adding new categories (singer, dancer, etc) as well as more and more attributes (experience_level, has_agent, languages) to existing categories.

The solution to this problem is one of my fondest memories in writing code. I believe it’s for moments like this that engineers write software.

TL;DR: Jump to The Magic section later in the article.

Problem Statement:

I need to store attributes associated with a particular Portfolio.

1. Lot of possible values. There are around 100 attribute names (gender, language, hair_color) and each has around 5 attribute values.
2. New attributes are added frequently and some even change over time. I knew I didn’t want to write more code every time a new attribute is added.
3. Some portfolios could have a lot of attributes, some could have very few.
4. Some attribute names are single select (gender) while some are multiple select (language).
5. I need to run queries on the attributes, so I couldn’t store them as a JSON string.
6. The attributes are used in views as well (to display and update). There should be no inconsistency in the database ORM code and view code. Every attribute should in be both places or in neither of them.

Solution:

I knew if I treat attribute names as data instead of code, then I will not have to write new code to accommodate new attributes. I could create a new table portfolio_tags with a column tag to store the value and a portfolio_id to refer to the portfolio. If an entry is present, the user has selected that attribute in their portfolio otherwise they haven’t. This would also solve the problem of running queries on tags (solves #5). I can have any number of attributes as these are SQL rows (solves #3 and helps with #1 and #2). Tags are not a novel concept. A lot of websites (stackoverflow, medium) have them.

Now I have tags.

Now that I have a place to store arbitrary attributes, where should I place the data source for the attribute names and values? The views are already picking up this data from rails’ config/locales/en.yml. I could use this for my rails’ models as well (following the MVC pattern, rails’ models behave as the db ORM). Little unorthodox, but easily doable. The translation file serves as my Single source of truth (vital for point #6 and also helps with #2). I just need to edit this file to add more attributes. If the text of an attribute changes, I just change the value and not the key in the YAML file. Also, I nested all attributes under single_select or multiple_select (solves #4).

Now, I have a Single Source of Truth.

The elephant in the room is the Portfolio.rb class. The class that will actually read from and write to the database. I need methods to read and write these attributes. Like:

def gender
  # ...
end
def gender=(value)
  # ...
end

But I don’t want to write methods by hand!

The Magic!

(To summarize, I have a structured plain text file with a collection of attributes. I need getter and setter methods for each of these attributes.)

Using the meta programming magic of ruby, I write code that dynamically generates these methods for me. This is how:

Iterate over the entries in config/locales/en.yml:

I18n.t('single_select').each do |key, value|
  # ...
end
I18n.t('multiple_select').each do |key, value|
  # ...
end

Call ruby’s define_method inside both the blocks, to generate both getters and setters:

define_method key do |arg = {}|
  # ...
end
define_method (key.to_s + '=').to_sym do |arg|
  # ...
end

(They are implemented a little differently for single_select and multiple_select attributes, but the basics remain the same.)

Voila! I wrote code to generate code.

Honestly, I was terrified of writing something like this and push it to production. It seems so wrong- calling methods that don’t exist. Yet, at the same time, it seems like the only correct way. With the benefit of hindsight, this was the correct way to do it.

It’s easy to shoot myself with such programming features. In the ruby/rails world, these are called Sharp Knives:

Ruby includes a lot of sharp knives in its drawer of features. Not by accident, but by design.

- Rails Doctrine (Sharp Knives)

Code like 2.days.ago blew my mind when I first saw them.

Probably less than 10% of your codebase will use these sharp knives. Using sharp knives is one of pure joys of programming. I’d happily write 1k lines of dumb code (this is the way it should be done, just so you know) so that I can write just 100 lines using sharp knives.

I’m not the only one who thinks of meta programming as magic. Chris does too. I don’t fail to document it either:

Screenshot of original code from Dec ’16

More where this came from

This story is published in Noteworthy, where thousands come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more product & design stories featured by the Journal team.