Keeping Secrets Safe With KMS

Amazon Web Services unveiled a new service during its re:Invent conference in Las Vegas this past November. KMS (Key Management Service) is easy to use, inexpensive and provides a failsafe, additional layer of protection for your data. The uses for KMS are endless and the premise is fairly simple:

KMS creates a Master Encryption Key stored safely within AWS. The Master key is never released, and enables you to encrypt and decrypt data. Another valuable feature is the Data key, an encryption key created by KMS that also comes with a version encrypted with the master key.

Our team recently had the opportunity to work with KMS on a client project. We were to encrypt all secrets and data — specifically SSL keys, Key/Trust stores and passwords for the secrets and row level encryption for the data.

Encrypting secrets is important for the transport and storage of vulnerable data within AWS because machines are provisioning themselves on demand. For our secrets we used the KMS master key to encrypt our file and stored it as binary. The secret was then able to live at rest with other provisioning data.

Getting Started
To get started you’ll need version => 1.6 of the aws CLI or equivalent SDK. Using the CLI you can create a KMS Master Key as follows:

$ aws kms create-key --policy file://path/to/policy

You will absolutely want to include a key policy because otherwise IAM users and roles will not be able to access the key. For the sake of keeping my example simple, I’ll only demonstrate one KMS key and one role. However, it should be noted that it is more secure to use separate keys and roles for different services to limit the blast radius in the event of a compromise.

If you do not provide a policy KMS will create one for you granting access to only the user or role that created the key. You may want to alter this policy to fit your specific needs and use the put-key-policy command to modify. More about KMS policies can be found here.

{
"Version": "2012-10-17",
"Id": "key-default-1",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789370:role/MyAppRole"
},
"Action": "kms:*",
"Resource": "*"
}
]
}

Now For the Fun Stuff!
To encrypt some data with the Master key you will follow a fairly straightforward process.

You must specify the key-id and some plaintext to encrypt. The key-id parameter of this command will accept an ARN, an alias, or the Key ID. These examples will use the Key ID.
$ aws kms encrypt --key-id 69123367-0a7b-4456-885b-a87f08d7b8eb --plaintext file://ssl.pem

The above command will return two items, the KeyId and the CiphertextBlob. The Ciphertext blob is our encrypted data. It’s not clear when reading the documentation that this blob is base64 encoded, we’ve suggested this go in the documentation and hope to see the change soon.

You’ll want to filter out just the Ciphertext and decode that for storage.
$ aws kms encrypt --key-id 69123367-0a7b-4456-885b-a87f08d7b8eb --plaintext file://ssl.pem --query CiphertextBlob --output text | base64 --decode > ssl.pem.encrypted

You’ve now a encrypted file that you can store with your provisioning data.

Decryption
To decrypt this data we’ll use the decrypt command which will send the encrypted file to KMS. KMS does not need to know the key-id to decrypt as the id is stored inside of the encrypted data. When specifying the file you’d like to decrypt use the fileb:// prefix to specify that the file is in binary format. This time we’ll only want the plaintext returned by the command. The information that decrypt sends back is also in Base64 format and will need to be decoded.
$ aws kms decrypt --ciphertext-blob fileb://ssl.pem.encrypted --query Plaintext --output text | base64 --decode

You’ve come all this way to keep your information secret so the best place to store an ssl key on your instance is in memory.

But how do will you tell the configuration file of services to use a key that only lives in memory?

Create a Ramdisk.
A ramdisk is a volume made from memory. There are two primary options here and both come installed on many Linux distributions, tmpfs and ramfs. We’ve chosen to use ramfs, because tmpfs has the capability to write to swap and we never want our secret stored on the disk. Instead, create a ramfs volume with mount.

$ mount -t ramfs -o size=512m ramfs /mnt/ramdisk

Now you can use the decode command above and direct the output to your ramfs volume knowing that your ssl key will never be stored on the EBS volume.

Data Encryption
As for the row level encryption of the data, you would want to encrypt with a data key. When you generate a data key KMS will return you a plaintext encryption key and an encrypted version of the encryption key in the form of a Ciphertext Blob. You can store the Ciphertext Blob along with your encrypted data.

The decryption of this type of data encrypted with a Data key is slightly different. If you present KMS with the Ciphertext Blob, KMS it will decrypt the ciphertext and send you back the data key you used to encrypt your data — you’re responsible for decrypting the actual data set.

Overall KMS was fairly easy to use and a really nice solution for key management as it includes a key rotation feature. This is a step in the right direction for protecting your business in the public cloud.

4 Replies to “Keeping Secrets Safe With KMS”

  1. So once you encrypted your ssl keys where did you store them as encrypted files? I want to use KMS to encrypt pem files like you did but I am unsure where they should be kept. I would think outside version control but if the file is encrypted with KMS then maybe its okay.

  2. Where you keep your encrypted files is mainly based on your teams workflow, source control might work if you have a repository for your configuration, I would not advise storing it with application code. You would also need to be aware of who has permissions to read your source control, are there any contractors etc that might be able to decrypt and have the key, potentially make a mistake with it’s security. Private S3 with server side encryption is not a bad choice because with IAM and bucket policy you can be 100% sure you know who is able to see that key. After the key is decrypted you would want to store it in memory with ramfs. Tmpfs can write out to swap and is the less favorable in memory filesystem for secrets because of that.

Leave a Reply

Your email address will not be published. Required fields are marked *