- Published on
Migrate attr_encrypted to Rails 7 Active Record encrypts
- Austin Miller
Rails 7 has introduced Active Record Encryption, functionality to transparently encrypt and decrypt data before storing it in the database. This is awesome news for any developer who has ever had to encrypt data before storing it.
In this guide, I will walk you through an example to migrate from away from using attr_encrypted gem to the new Rails 7 Active Record encrypts. We will do this using strong migrations and also maintain the ability to perform a database rollback without data loss.
If you are short on time, below is the crux of this article. If you are actually implementing this, I would highly encourage you to read on, this can be a fairly complex migration.
- This article is written on 13 April 2021 - Currently Rails 7 is Edge (aka alpha). This tutorial makes certain assumptions based on that. I will publish updates when Rails 7 is officially released.
- attr_encrypted and Active Record encrypts are not compatible - you’ll need to use a fork of attr_encrypted
- devise - currently needs the
patch-2branch to work with Rails 7
- Upgrade to Rails 7
- Add dynamic attributes to model
- Perform Migrations
- Delete attr_encrypted gem dependency
Most applications at some point in time need to encrypt data before storing it to the database (and conversely decrypt it before using it in the application). Historically there have been 2 gems that were fairly popular for this sort of functionality, namely attr_encrypted and lockbox. I personally have preferred lockbox since its still actively maintained and used less columns, but if you are like me you can’t always choose whats handed to you.
Unfortunately, the attr_encrypted gem is no longer maintained has a lot of name clashes with the Rails 7 Active Record encrypts functionality. To work around this, we had to create a fork and rename many of the function calls and properties (namely encrypt, decrypt, ect.). You too will need to use the PagerTree fork of the attr_encrypted gem during your migration process (but don’t worry, you can delete it after your migration).
Upgrade to Rails 7
You’ll first need to upgrade to Rails 7. As of this writing (13 April 2021), Rails 7 is Edge. This tutorial will use syntax and functionality that is currently in alpha.
In your gem file you’ll need to change:
And then make sure to install the new dependencies, and update any others.
At this point, we now have Rails 7 installed, with a compatible version of attr_encrypted.
Add Rails 7 Active Record encrypts keys
Following encrypts documentation, you’ll need to add some keys to your rails credentials file.
Copy the output YAML and paste it into your credentials file. It should look something like this:
You’ll need to do this once for each environment (normally development, staging, production).
At this point, the Active Record encrypts should be ready to go.
The next steps will be to migrate any data that was previously using attr_encrypted to use the new encrypts methods. Because we want to be secure and also use strong migrations our process should look like this:
- Modify our model to dynamically define attributes to use during our migration
- Create a new temporary column for our old encrypted data (attr_encrypted)
- Copy old encrypted data to new temporary column, and delete old column
- Add a new column for our Rails 7 Active Record encrypts data
- Run a migration to programmatically decrypt attr_encrypted temporary column and put it in Rails 7 Active Record encrypts column
- Delete temporary column
Seems like a lot of overkill, but we do it this way so we don’t perform any dangerous database activity and make strong migrations happy. This process will also keep our migrations backward compatible and prevent data loss in case we ever need to rollback.
I’m going to make the assumption you are fairly dangerous when it comes to coding, and you are relatively familiar with the rails framework. Please use this example as a guide. You’ll need to make modifications to your own code to make this work for you..
Below, is what I will assume is our starting point. We have a
User model that has an attribute called
otp_secret (stands for “one time password secret”, for two factor authentication).
otp_secret property currently uses attr_encrypted. This means in our database we should have the following columns:
We’ll take advantage of the fact that attr_encrypted prefixed its column names with “encrypted”. By copying our data into a temporary column, we can avoid name clashes, and use the encrypts functionality almost transparently (you’ll see below how the names will come full circle).
Modify model to use dynamic attributes
We need to add some extra code to dynamically define attributes. During the migration, only two of these columns will ever exist at a time, making it so that we can migrate our columns without name clashing.
Create a Temporary Column
The temporary column will just hold a copy of our existing attr_encrypted field. We move data here for strong migrations and so the Rails 7 encrypts column doesn’t conflict with the attr_encrypted accessor.
Copy attr_encrypted columns
You’ll want to create a new migration that copies the original attr_encrypted column to the one we just created, but you’ll want to make sure you define both
down so that you can have backward compatibility.
Add Rails 7 Active Record encrypts columns
Now we’ll add a new column, where we will store the Rails 7 Active Record encrypts data.
It’s important that the column be of type
:text. The rails guides specify that the column should be at least 510 bytes.
This takes full advantage of the previous attr_encrypted nomenclature. After the migration, we should be able to access the opt_secret still by using
user.otp_secret. See how that just came full circle ;)
Migration data from attr_encrypted to encrypts
In this step, we generate a migration to move data from the attr_encrypted property to the Rails 7 Active Record encrypts property. We have to do this programmatically (and can’t do a shortcut db command) because it is the rails engine is what is actually doing the
decrypt work for us.
Additionally, we do some special reloading of the
User model because of how we have dynamically defined attributes (Again, this is meant just to be temporary while we migrate).
Remove Temporary Column
Our last step is to remove our temporary column, so our database is kept nice and clean. Again, we define the
down methods in this migration so we are backward compatible, and if for any reason we can go back in time and re-create our data.
Run your migrations
Now you should be able to run all your newly created migrations with one swift command.
If it worked, congrats :) If for some reason it doesn’t work, check the error output. It could be a simple syntax error, or something specific to your setup. Here is where I am counting on you to be dangerous and figure out what could have happened.
Remove attr_encrypted dependancy
You can now safely remove the attr_encrypted dependancy in your gem file. However, be aware that this will break existing process of
rails db:create db:setup (for example in development). You’ll likely want to use
rails db:setup instead so that it loads from the schema file and at some point squash your migrations directory.
I hope you find some value in this tutorial and it can save you time and effort when it comes to migrating away from attr_encrypted. There’s probably a lot I missed on here, so if you have something to add you can reach out to me on twitter and I will update the article with your suggestion.
Some other notes on snags I came across during development.
Problems when creating a database with Devise and Rails 7 Active Record Encrypts
The Rails 7 Active record encrypts seems to break
db:create when used in conjunction with Devise. Didn’t dig too far into this, but Rails complains that the encrypts modifier can’t properly check the database column size. Makes sense since there currently is no database, but it did force me to create a hack on the user model. It didn’t seem to affect other models that didn’t interact with Devise.
I assume this will get fixed at some point and is just a Devise + Edge (alpha) thing.