In order to make sure our customer data is as safe as possible, we decided to implement encryption at rest. Although we tried a few different approaches, we finally settled on encrypting the database file system and everything related to it. As we are already using an Amazon PostgreSQL instance, and Amazon RDS supports database encryption at rest, we chose that option. As it often is in life, you can’t really flip a switch and encrypt a running instance. So we decided to dive into the documentation and find out how to do it as painlessly as possible.
Mandatory disclaimer: this guide is intended for educational purposes only and if you somehow mess up your production data by following it, it’s not my fault.
How do you encrypt an unencrypted Amazon RDS instance then? Well, it’s a pretty simple and quite short procedure (this does depend on the amount of data you have, but it shouldn’t take you more than an hour for a 500GB instance)
Take a few minutes and make yourself a cup of coffee/tea/matcha, then come back and follow along this simple procedure.
- To ensure that you don’t have any “new” data from users or background jobs you should put the application into maintenance mode and turn off all web and worker processes. It is also nice to announce a planned downtime weeks in advance, and do it when there are practically no users using the app. This probably won’t be a Wednesday morning, during business hours. You have your analytics, plan the thing in advance.
- After the maintenance mode is on, and there is no chance of any database connections coming to the old unencrypted database, you can take a snapshot of it. Select the desired database from Amazon RDS > Instances then go to Instance actions > Take snapshot, be sure to give the snapshot a nice name (Daisy does sound nice, but I would go with app-production-decrypted or something descriptive).
- Go to Snapshots and select the snapshot you’ve just created
- Select Snapshot actions then Copy Snapshot
- Select Enable Encryption in the encryption box and select the KMS key for the database you are encrypting (or let the system create an auto managed key, which I don’t recommend, because you will be using a master KMS key for all instances here, and not a per-instance one.)
- Rename the old database instance to a new name, so the name won’t collide with the restored instance.
- Restore the snapshot in the same way as you would do with a normal unencrypted snapshot by going to the Snapshots page, selecting a snapshot and selecting Restore Snapshot under Snapshot Actions. Obviously you want to give it the same name that the old database instance had (you don’t want to update those pesky configuration files with database connection strings).
- Run your tests and assert that everything works, and only then turn on the web/worker services on. Take down the maintenance page, and reap the benefits of having encryption at rest.
While there are many ways to set up encryption at rest, and many variations on what falls under that term, this is just one of them that fits our model best. We did explore a few more encryption options, but all came with certain drawbacks when implemented within our architecture. Stay tuned for more AWS articles in the future.